Navigating the Post-API Dilemma | Search Engine Results Pages Present a Biased View of Social Media Data

Overview

• Recent decisions to discontinue social media APIs are having detrimental effects on Internet research and computational social science.
• This lack of access to data has been dubbed the "Post-API era" of Internet research.
• Search engines like Google may provide a solution by crawling and surfacing social media data on their Search Engine Results Pages (SERP).
• The paper investigates whether SERP provides a complete and unbiased sample of social media data, and if it is a viable alternative to direct API access.

Plain English Explanation

Researchers rely on data from social media platforms like Twitter and Reddit to study how people behave and interact online. However, many platforms have recently restricted access to this data, making it much harder for researchers to do their work. This is known as the "Post-API era" of Internet research.

Fortunately, popular search engines like Google have the ability to find and display social media content on their search result pages, which could potentially solve this problem. The researchers in this paper wanted to know if the social media data shown on search results is truly representative and unbiased, or if it has significant gaps and skewed towards certain types of content.

To figure this out, the researchers compared the search results to the actual, unfiltered data from Twitter and Reddit. What they found was that the search results were heavily biased - they favored popular and positive content, and tended to exclude political, explicit, or controversial posts. There were also major gaps in the topics covered compared to the full social media data.

Overall, the researchers concluded that using search engine results is not a good replacement for direct access to social media data. The results are just too limited and skewed to give researchers an accurate picture of what's really happening online.

Technical Explanation

The paper conducts a comparative analysis between data obtained directly from Reddit and Twitter/X, and the social media content surfaced on Google's Search Engine Results Pages (SERP).

The researchers first obtained a complete, unfiltered dataset from Reddit and Twitter/X. This "ground truth" dataset represents the full, unbiased sample of social media activity.

They then collected the top search results from Google for a set of carefully chosen queries designed to surface relevant social media content. This SERP dataset was then compared to the ground truth in several ways:

1. Popularity Bias: The SERP results were found to be heavily skewed towards highly popular social media posts, neglecting less viral content.

2. Sentiment Bias: The sentiment expressed in the SERP results was more positive on average compared to the ground truth dataset.

3. Topical Gaps: Significant gaps were observed in the topics covered by the SERP results versus the full social media data.

4. Content Filtering: The SERP results were found to systematically exclude political, pornographic, and profane content that was present in the ground truth data.

Through this comprehensive analysis, the researchers conclude that SERP is not a viable replacement for direct access to social media data via APIs. The biases and gaps present in the SERP results make it an incomplete and unreliable source for conducting computational social science research.

Critical Analysis

The paper provides a thorough and rigorous analysis of the limitations of using search engine results as a proxy for social media data. The researchers carefully designed their experiment to enable a direct comparison between SERP and the ground truth datasets.

However, one potential limitation of the study is the use of a single search engine (Google) and the reliance on the top search results. It's possible that other search engines or different ways of surfacing social media content on SERPs could yield different results.

Additionally, the paper does not delve into the specific mechanisms or algorithms used by search engines to select and rank social media content. A deeper understanding of these processes could provide further insights into the origins of the observed biases.

Another area for further research could be investigating ways to mitigate the biases, such as techniques for retrieving a more representative sample of social media data from search engines. This could potentially make SERP a more useful resource for computational social science research.

Conclusion

The discontinuation of social media APIs has created significant challenges for Internet researchers, but using search engine results as an alternative is not a viable solution. This paper demonstrates that the social media content surfaced on SERPs is heavily biased, with significant gaps and systematic exclusion of certain types of content.

These findings highlight the critical importance of maintaining direct access to social media data for researchers in the field of computational social science. Without unbiased data, our understanding of online behavior and social dynamics will be incomplete and potentially skewed.

The issues raised in this paper underscore the need for search engines, social media platforms, and the research community to work together to find new ways to provide researchers with the data they need to advance our knowledge of the digital world.