Decentralised Moderation for Interoperable Social Networks: A Conversation-based Approach for Pleroma and the Fediverse
2404.03048
0
0
Abstract
The recent development of decentralised and interoperable social networks (such as the fediverse) creates new challenges for content moderators. This is because millions of posts generated on one server can easily spread to another, even if the recipient server has very different moderation policies. An obvious solution would be to leverage moderation tools to automatically tag (and filter) posts that contravene moderation policies, e.g. related to toxic speech. Recent work has exploited the conversational context of a post to improve this automatic tagging, e.g. using the replies to a post to help classify if it contains toxic speech. This has shown particular potential in environments with large training sets that contain complete conversations. This, however, creates challenges in a decentralised context, as a single conversation may be fragmented across multiple servers. Thus, each server only has a partial view of an entire conversation because conversations are often federated across servers in a non-synchronized fashion. To address this, we propose a decentralised conversation-aware content moderation approach suitable for the fediverse. Our approach employs a graph deep learning model (GraphNLI) trained locally on each server. The model exploits local data to train a model that combines post and conversational information captured through random walks to detect toxicity. We evaluate our approach with data from Pleroma, a major decentralised and interoperable micro-blogging network containing 2 million conversations. Our model effectively detects toxicity on larger instances, exclusively trained using their local post information (0.8837 macro-F1). Our approach has considerable scope to improve moderation in decentralised and interoperable social networks such as Pleroma or Mastodon.
Create account to get full access
Overview
- This paper proposes a decentralized moderation system for interoperable social networks, focusing on Pleroma and the Fediverse.
- The approach is based on conversation-based moderation, where users can collectively discuss and vote on content moderation decisions.
- The system aims to enable self-governance and community-driven moderation, in contrast to centralized content moderation models.
Plain English Explanation
The paper describes a new way for online communities to moderate their own content and discussions. Rather than having a centralized authority decide what's allowed, the communities would collectively discuss and vote on moderation decisions.
Imagine an online forum where users can raise concerns about posts they find problematic. Other users can then join the discussion, share their perspectives, and ultimately vote on whether the content should be removed or left up. This collaborative process allows the community to self-govern and shape their own norms, rather than having an outside party make those choices for them.
The researchers focus on implementing this approach within Pleroma and the broader Fediverse network, which are decentralized social media platforms. By decentralizing moderation as well, the goal is to empower users and communities to take more control over their online spaces.
Technical Explanation
The paper proposes a conversation-based moderation system for Pleroma and the Fediverse. When a user flags content as potentially problematic, it triggers a moderation conversation thread. Other users can join this thread, share their views, and ultimately vote on whether the content should be removed or left up.
The system is built on top of the ActivityPub protocol, which enables interoperability between different Fediverse servers. This allows moderation conversations to span multiple servers and communities. The researchers also describe mechanisms for handling conflicts between local and federated moderation decisions.
The paper outlines the system architecture, including components for managing moderation conversations, tracking user reputations, and aggregating votes. It also discusses potential attack vectors, such as coordinated efforts to game the moderation process, and how the system can be designed to mitigate these risks.
Critical Analysis
The paper presents a compelling vision for decentralized, community-driven content moderation. By empowering users to collectively discuss and decide on moderation issues, the approach aims to foster more transparent and accountable online spaces.
However, the researchers acknowledge several challenges and limitations. Maintaining active participation in moderation conversations may be difficult, particularly for smaller communities. There are also open questions around how to handle power dynamics, prevent manipulation, and ensure fair representation in the voting process.
Additionally, the paper does not fully address the potential for moderation decisions to become politicized or for communities to develop harmful norms. Further research is needed to understand how these systems might evolve and whether they can consistently uphold principles of free speech and open discourse.
Conclusion
This paper introduces a novel approach to content moderation that seeks to decentralize and democratize the process. By enabling conversation-based moderation, the researchers aim to empower online communities to self-govern and shape their own norms.
While the proposal raises intriguing possibilities, it also highlights the inherent complexities and challenges of moderating large-scale, decentralized social networks. Ongoing research and experimentation will be crucial to refine these systems and ensure they foster healthy, inclusive online spaces.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
💬
Can Language Model Moderators Improve the Health of Online Discourse?
Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May
0
0
Conversational moderation of online communities is crucial to maintaining civility for a constructive environment, but it is challenging to scale and harmful to moderators. The inclusion of sophisticated natural language generation modules as a force multiplier to aid human moderators is a tantalizing prospect, but adequate evaluation approaches have so far been elusive. In this paper, we establish a systematic definition of conversational moderation effectiveness grounded on moderation literature and establish design criteria for conducting realistic yet safe evaluation. We then propose a comprehensive evaluation framework to assess models' moderation capabilities independently of human intervention. With our framework, we conduct the first known study of language models as conversational moderators, finding that appropriately prompted models that incorporate insights from social science can provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation.
5/7/2024
Content-Agnostic Moderation for Stance-Neutral Recommendation
Nan Li, Bo Kang, Tijl De Bie
0
0
Personalized recommendation systems often drive users towards more extreme content, exacerbating opinion polarization. While (content-aware) moderation has been proposed to mitigate these effects, such approaches risk curtailing the freedom of speech and of information. To address this concern, we propose and explore the feasibility of emph{content-agnostic} moderation as an alternative approach for reducing polarization. Content-agnostic moderation does not rely on the actual content being moderated, arguably making it less prone to forms of censorship. We establish theoretically that content-agnostic moderation cannot be guaranteed to work in a fully generic setting. However, we show that it can often be effectively achieved in practice with plausible assumptions. We introduce two novel content-agnostic moderation methods that modify the recommendations from the content recommender to disperse user-item co-clusters without relying on content features. To evaluate the potential of content-agnostic moderation in controlled experiments, we built a simulation environment to analyze the closed-loop behavior of a system with a given set of users, recommendation system, and moderation approach. Through comprehensive experiments in this environment, we show that our proposed moderation methods significantly enhance stance neutrality and maintain high recommendation quality across various data scenarios. Our results indicate that achieving stance neutrality without direct content information is not only feasible but can also help in developing more balanced and informative recommendation systems without substantially degrading user engagement.
5/30/2024
How Decentralization Affects User Agency on Social Platforms
Aditya Surve, Aneesh Shamraj, Swapneel Mehta
0
0
Mainstream social media platforms function as walled garden ecosystems that restrict user agency, control, and data portability. They have demonstrated a lack of transparency that contributes to a multitude of online harms. Our research investigates how decentralization might present promise as an alternative model to walled garden platforms. Specifically, we describe the user-driven content moderation through blocks as an expression of agency on Bluesky, a decentralized social platform. We examine the impact of providing users with more granular control over their online experiences, including what they post, who can see it, and whose content they are exposed to. We describe the patterns identified in user-driven content moderation and suggest directions for further research.
6/14/2024
🗣️
Decentralized Social Networks and the Future of Free Speech Online
Tao Huang
0
0
Decentralized social networks like Mastodon and BlueSky are trending topics that have drawn much attention and discussion in recent years. By devolving powers from the central node to the end users, decentralized social networks aim to cure existing pathologies on the centralized platforms and have been viewed by many as the future of the Internet. This article critically and systematically assesses the decentralization project's prospect for communications online. It uses normative theories of free speech to examine whether and how the decentralization design could facilitate users' freedom of expression online. The analysis shows that both promises and pitfalls exist, highlighting the importance of value-based design in this area. Two most salient issues for the design of the decentralized networks are: how to balance the decentralization ideal with constant needs of centralization on the network, and how to empower users to make them truly capable of exercising their control. The article then uses some design examples, such as the shared blocklist and the opt-in search function, to illustrate the value considerations underlying the design choices. Some tentative proposals for law and policy interventions are offered to better facilitate the design of the new network. Rather than providing clear answers, the article seeks to map the value implications of the design choices, highlight the stakes, and point directions for future research.
6/12/2024