The Influencer Next Door: How Misinformation Creators Use GenAI

Overview

• This paper explores how the increasing use of generative AI (GenAI) by non-expert users is changing the landscape of content creation and misinformation.
• The researchers conducted ethnographic research with misinformation creators and consumers between 2022-2023 to understand how GenAI is being used in practice.
• The key findings highlight how GenAI is being used as a productivity tool by non-experts to create large volumes of content, often with misinformative elements, and to deploy marketing tactics to expand engagement and monetize their content.

Plain English Explanation

The paper examines how the rise of generative AI is impacting the way people create and share content online. Traditionally, the concern has been about "expert" disinformation creators and faulty AI models deceiving "ordinary" users. However, this research found that GenAI is actually enabling more non-experts to become content creators themselves.

Participants in the study used GenAI as a tool to quickly generate large volumes of content, often with misinformative elements, to meet their personal needs and desires. They were driven by a "get rich quick" narrative of becoming an "influencer millionaire" through content creation. GenAI lowered the barriers to entry, making it easier for consumers to become creators across different mediums.

Additionally, participants leveraged GenAI to learn and apply marketing tactics to expand their audience engagement and monetize their content. This suggests a shift from analyzing the public as passive consumers of AI-generated content to seeing them as active "bricoleurs" who use GenAI creatively, even if they don't fully understand the underlying technology.

The research highlights how these emergent uses of GenAI can produce new or accelerated misinformation harms, which has important implications for the design of AI products, platforms, and policies.

Technical Explanation

The paper presents findings from longitudinal ethnographic research conducted between 2022-2023 with misinformation creators and consumers. This approach departs from the typical assumption in existing literature that "expert" organized disinformation creators and flawed AI models are the primary sources of deception targeting "ordinary" users.

The key findings from the research are:

1. Participants primarily used GenAI for content creation, rather than truth-seeking.
2. A spreading "influencer millionaire" narrative drove participants to become content creators, using GenAI as a productivity tool to generate large volumes of (often misinformative) content.
3. GenAI lowered the barrier to entry for content creation across modalities, encouraging consumers to become creators and significantly increasing existing creators' output.
4. Participants used GenAI to learn and deploy marketing tactics to expand engagement and monetize their content.

The researchers argue for a shift in analysis from the public as consumers of AI content to "bricoleurs" who use GenAI creatively, often without a detailed understanding of its underlying technology. They analyze how these understudied emergent uses of GenAI can produce new or accelerated misinformation harms, and the implications for the design of AI products, platforms, and policies.

Critical Analysis

The paper provides valuable insights into how the rise of generative AI is reshaping content creation and the spread of misinformation. By shifting the focus from expert disinformation creators to non-expert "bricoleurs," the researchers uncover significant changes in how people are using these technologies.

However, the paper does not delve deeply into the technical details of the GenAI systems being used by participants, nor does it explore the specific types of misinformation being generated. Further research may be needed to understand the technical capabilities and limitations of the GenAI tools being employed, as well as the content and narratives being produced.

Additionally, the paper's focus on ethnographic research with a limited sample of participants raises questions about the generalizability of the findings. Expanding the study to include a broader range of creators and consumers could help validate the observed trends and uncover additional insights.

Overall, the paper makes a valuable contribution by highlighting the need to consider the evolving and often unexpected ways in which people are using generative AI in their daily lives. This shift in perspective can inform the development of more effective strategies for addressing the challenges posed by misinformation in the digital age.

Conclusion

This research challenges the traditional narrative around generative AI and misinformation, revealing how non-expert users are leveraging these technologies to become active content creators and bricoleurs. The key findings highlight how GenAI is being used as a productivity tool to generate large volumes of often misinformative content, driven by a narrative of becoming an "influencer millionaire."

These emergent uses of GenAI have important implications for the design of AI products, platforms, and policies, as they can produce new or accelerated misinformation harms. The paper calls for a shift in analysis from the public as passive consumers to active, creative users of GenAI, even if they don't fully understand the underlying technology.

Ultimately, this research underscores the need for a more nuanced understanding of how people are engaging with generative AI in their daily lives, and the potential consequences of these practices for the wider digital ecosystem.