Generative AI Search Engines as Arbiters of Public Knowledge: An Audit of Bias and Authority

Overview

• This paper examines how new generative AI search engines like ChatGPT, Bing Chat, and Perplexity construct responses and establish authority on topics of public importance.
• The researchers collected system responses to 48 authentic queries across 4 topics over a 7-day period, then analyzed the data using sentiment analysis, coding, and source classification.
• The study provides insights into the nature of responses from these AI systems, including evidence of sentiment, commercial, and geographic biases, as well as uneven quality of sources used to support claims.

Plain English Explanation

The paper looks at how the latest AI-powered search engines, like ChatGPT, Bing Chat, and Perplexity, construct their responses and establish authority on important public topics. The researchers collected responses from these systems over a week, using real-world questions, and then analyzed the data to understand patterns and potential biases.

The key findings suggest that the AI systems can show biases in their responses, both in terms of sentiment (positive or negative) and in the types of sources they use to back up their claims. For example, the systems tended to rely heavily on news, business, and digital media websites, rather than academic or government sources. This uneven quality of sources could be problematic, especially when people are using these AI systems to make important decisions about public issues or their personal wellbeing.

The researchers recommend that users of these generative AI search engines should critically examine the outputs and not blindly accept them as authoritative. They need to be aware of the potential biases and limitations of these systems, especially when the information is related to matters of public interest.

Technical Explanation

The researchers conducted an audit study to investigate how new generative AI search engines like ChatGPT, Bing Chat, and Perplexity construct responses and establish authority on topics of public importance.

They collected system responses to a set of 48 authentic queries across 4 topics (politics, health, science, and current events) over a 7-day period. The data was then analyzed using sentiment analysis, inductive coding, and source classification techniques.

The results provide an overview of the nature of system responses across these AI models. The researchers found evidence of sentiment bias based on the queries and topics, as well as commercial and geographic biases in the sources used to support claims. The quality of sources relied upon was uneven, with a heavy emphasis on news, media, business, and digital media websites, rather than academic or government sources.

Critical Analysis

The paper acknowledges some limitations of the study, such as the relatively small sample size and the focus on only 4 specific topics. The researchers also note that their analysis of source quality is subjective and may not capture all nuances.

One potential issue not addressed in the paper is the evolving nature of these generative AI systems. As the technology continues to develop, the patterns and biases observed in this study may change over time, requiring ongoing monitoring and evaluation.

Additionally, the paper does not delve deeply into the potential societal implications of these biases in AI-generated content, especially when it comes to the impact on news production and consumption. Further research is needed to understand the broader ramifications of generative AI on information access and decision-making.

Conclusion

This study provides valuable insights into the behavior and potential biases of new generative AI search engines, such as ChatGPT, Bing Chat, and Perplexity. The findings highlight the importance of users critically examining the outputs of these systems, especially when the information is related to matters of public interest or personal well-being.

As the use of generative AI in information access continues to grow, it will be crucial to further investigate the sociotechnical implications and develop strategies to ensure these powerful tools are used responsibly and equitably.