The Use of Generative Search Engines for Knowledge Work and Complex Tasks
2404.04268
1
0
🖼️
Abstract
Until recently, search engines were the predominant method for people to access online information. The recent emergence of large language models (LLMs) has given machines new capabilities such as the ability to generate new digital artifacts like text, images, code etc., resulting in a new tool, a generative search engine, which combines the capabilities of LLMs with a traditional search engine. Through the empirical analysis of Bing Copilot (Bing Chat), one of the first publicly available generative search engines, we analyze the types and complexity of tasks that people use Bing Copilot for compared to Bing Search. Findings indicate that people use the generative search engine for more knowledge work tasks that are higher in cognitive complexity than were commonly done with a traditional search engine.
Create account to get full access
Overview
- Until recently, search engines were the primary way people accessed online information.
- The emergence of large language models (LLMs) has given machines new capabilities, such as generating text, images, and code.
- This has led to the development of a new tool called a "generative search engine," which combines LLM capabilities with traditional search engine functionality.
- The paper analyzes the types and complexity of tasks that people use the Bing Copilot generative search engine for, compared to Bing Search.
Plain English Explanation
In the past, people mostly relied on search engines like Google or Bing to find information online. However, the rise of advanced artificial intelligence systems called "large language models" (LLMs) has unlocked new abilities for machines, such as generating new digital content. This has led to the creation of a new type of search tool called a "generative search engine," which combines the searching power of traditional search engines with the content-generation capabilities of LLMs.
One example of a generative search engine is Bing Copilot, which was recently made available to the public. The researchers in this paper looked at how people use Bing Copilot compared to a regular search engine like Bing Search. They found that people tend to use the generative search engine for more complex, knowledge-work tasks that require higher-level thinking, rather than just simple lookup tasks. This suggests that generative AI can unlock new ways for people to interact with and make use of online information.
Technical Explanation
The researchers conducted an empirical analysis of Bing Copilot, one of the first publicly available generative search engines. They compared the types of tasks and complexity of queries that people used Bing Copilot for versus a traditional search engine like Bing Search.
The study found that people tended to use the generative search engine, Bing Copilot, for more knowledge work tasks that were higher in cognitive complexity than the tasks commonly performed with a traditional search engine. This suggests that the integration of large language models into search engines can advance the search frontier and enable new ways for people to access and utilize online information.
Additionally, the researchers note that the use of generative search engines may have implications for how AI agents can be leveraged for second language learning and teaching, as the ability to generate contextual responses could be beneficial in that domain.
Critical Analysis
The paper provides a useful initial analysis of how people are using a generative search engine compared to a traditional search engine. However, the research is limited to a single platform, Bing Copilot, and may not be generalizable to other generative search engines or future iterations of the technology.
Additionally, the paper does not delve deeply into the potential challenges or limitations of generative search engines, such as the potential for biases or inaccuracies in the generated content. Further research is needed to explore these aspects and their implications for users.
Overall, the findings provide an interesting initial glimpse into the emerging world of generative search engines and their impact on how people access and utilize online information. However, continued critical analysis and research will be important as this technology continues to evolve.
Conclusion
This paper presents an empirical analysis of how people use a generative search engine, Bing Copilot, compared to a traditional search engine like Bing Search. The key finding is that people tend to use the generative search engine for more complex, knowledge-work tasks that require higher-level thinking, rather than just simple lookup tasks.
This suggests that the integration of large language models into search engines can unlock new ways for people to access and make use of online information, potentially with implications for second language learning and teaching as well. However, further research is needed to explore the potential challenges and limitations of generative search engines as the technology continues to develop.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🛠️
GEO: Generative Engine Optimization
Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik R Narasimhan, Ameet Deshpande
0
0
The advent of large language models (LLMs) has ushered in a new paradigm of search engines that use generative models to gather and summarize information to answer user queries. This emerging technology, which we formalize under the unified framework of generative engines (GEs), can generate accurate and personalized responses, rapidly replacing traditional search engines like Google and Bing. Generative Engines typically satisfy queries by synthesizing information from multiple sources and summarizing them using LLMs. While this shift significantly improves textit{user} utility and textit{generative search engine} traffic, it poses a huge challenge for the third stakeholder - website and content creators. Given the black-box and fast-moving nature of generative engines, content creators have little to no control over textit{when} and textit{how} their content is displayed. With generative engines here to stay, we must ensure the creator economy is not disadvantaged. To address this, we introduce Generative Engine Optimization (GEO), the first novel paradigm to aid content creators in improving their content visibility in GE responses through a flexible black-box optimization framework for optimizing and defining visibility metrics. We facilitate systematic evaluation by introducing GEO-bench, a large-scale benchmark of diverse user queries across multiple domains, along with relevant web sources to answer these queries. Through rigorous evaluation, we demonstrate that GEO can boost visibility by up to 40% in GE responses. Moreover, we show the efficacy of these strategies varies across domains, underscoring the need for domain-specific optimization methods. Our work opens a new frontier in information discovery systems, with profound implications for both developers of GEs and content creators.
5/30/2024
🤖
Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority
Alice Li, Luanne Sinnamon
0
0
This paper reports on an audit study of generative AI systems (ChatGPT, Bing Chat, and Perplexity) which investigates how these new search engines construct responses and establish authority for topics of public importance. We collected system responses using a set of 48 authentic queries for 4 topics over a 7-day period and analyzed the data using sentiment analysis, inductive coding and source classification. Results provide an overview of the nature of system responses across these systems and provide evidence of sentiment bias based on the queries and topics, and commercial and geographic bias in sources. The quality of sources used to support claims is uneven, relying heavily on News and Media, Business and Digital Media websites. Implications for system users emphasize the need to critically examine Generative AI system outputs when making decisions related to public interest and personal well-being.
5/24/2024
💬
A Survey of Generative Search and Recommendation in the Era of Large Language Models
Yongqi Li, Xinyu Lin, Wenjie Wang, Fuli Feng, Liang Pang, Wenjie Li, Liqiang Nie, Xiangnan He, Tat-Seng Chua
0
0
With the information explosion on the Web, search and recommendation are foundational infrastructures to satisfying users' information needs. As the two sides of the same coin, both revolve around the same core research problem, matching queries with documents or users with items. In the recent few decades, search and recommendation have experienced synchronous technological paradigm shifts, including machine learning-based and deep learning-based paradigms. Recently, the superintelligent generative large language models have sparked a new paradigm in search and recommendation, i.e., generative search (retrieval) and recommendation, which aims to address the matching problem in a generative manner. In this paper, we provide a comprehensive survey of the emerging paradigm in information systems and summarize the developments in generative search and recommendation from a unified perspective. Rather than simply categorizing existing works, we abstract a unified framework for the generative paradigm and break down the existing works into different stages within this framework to highlight the strengths and weaknesses. And then, we distinguish generative search and recommendation with their unique challenges, identify open problems and future directions, and envision the next information-seeking paradigm.
4/29/2024
🤖
Advancing the Search Frontier with AI Agents
Ryen W. White
0
0
As many of us in the information retrieval (IR) research community know and appreciate, search is far from being a solved problem. Millions of people struggle with tasks on search engines every day. Often, their struggles relate to the intrinsic complexity of their task and the failure of search systems to fully understand the task and serve relevant results. The task motivates the search, creating the gap/problematic situation that searchers attempt to bridge/resolve and drives search behavior as they work through different task facets. Complex search tasks require more than support for rudimentary fact finding or re-finding. Research on methods to support complex tasks includes work on generating query and website suggestions, personalizing and contextualizing search, and developing new search experiences, including those that span time and space. The recent emergence of generative artificial intelligence (AI) and the arrival of assistive agents, based on this technology, has the potential to offer further assistance to searchers, especially those engaged in complex tasks. There are profound implications from these advances for the design of intelligent systems and for the future of search itself. This article, based on a keynote by the author at the 2023 ACM SIGIR Conference, explores these issues and how AI agents are advancing the frontier of search system capabilities, with a special focus on information interaction and complex task completion.
4/4/2024