How to Train Your YouTube Recommender to Avoid Unwanted Videos

Read original: arXiv:2307.14551 - Published 4/3/2024 by Alexander Liu, Siqi Wu, Paul Resnick

🎲

Overview

YouTube provides features for users to indicate disinterest in recommended videos, such as the "Not interested" and "Don't recommend channel" buttons.
These buttons are meant to help correct mistakes made by YouTube's recommendation system.
However, little is known about the effectiveness of these buttons or how aware users are of them.

Plain English Explanation

YouTube's recommendation system tries to suggest new videos that users might enjoy based on their viewing history. However, sometimes the recommendations may not be accurate or relevant. To address this, YouTube provides users with buttons they can click to indicate that they are not interested in a particular video or channel.

The researchers in this study wanted to understand how well these "disinterest" buttons work in practice. They created fake YouTube accounts (called "sock puppet agents") and had them watch videos on specific topics. Then, the agents tried to remove recommendations related to those topics using different strategies, such as clicking the "Not interested" button or deleting the video from their watch history.

The researchers found that the "Not interested" button was the most effective at reducing recommendations for the topic the agents had watched. On average, it removed 88% of those recommendations. However, the buttons did not have much effect on the recommendations shown on individual video pages.

The researchers also surveyed real YouTube users to see how aware they were of these disinterest buttons and how effective they found them to be. They found that 44% of users were not even aware the "Not interested" button existed. But those who did know about it often used it and thought it was moderately effective at removing unwanted recommendations.

Technical Explanation

The researchers conducted two main parts to their study:

Simulation experiment: They created "sock puppet" YouTube accounts and had them go through a "stain" phase, where the accounts watched many videos on a specific topic. Then, in a "scrub" phase, the accounts tried to remove recommendations related to that topic using different strategies, such as clicking the "Not interested" button or deleting the watched videos from their history.
User survey: The researchers also surveyed 300 adult YouTube users in the US to assess their awareness and perceived effectiveness of the "Not interested" and "Don't recommend channel" buttons.

In the simulation experiment, the researchers found that the "stain" phase successfully increased the fraction of recommended videos related to the assigned topic on the user's homepage. However, using the "Not interested" button in the "scrub" phase was the most effective strategy, removing an average of 88% of those topic-related recommendations.

The survey results showed that 44% of users were not aware the "Not interested" button existed. But among those who were aware of it, 82.8% reported using it to remove unwanted recommendations, and they rated its effectiveness as moderately high (3.42 out of 5).

Critical Analysis

The researchers acknowledge some limitations of their work. For example, the simulation experiment may not fully capture the complexity of real YouTube user behavior and the recommendation system. Additionally, the survey results could be biased, as participants may not accurately remember or report their usage of the disinterest buttons.

One aspect not addressed in the paper is the potential for the disinterest buttons to be misused or gamed by users. It's possible that some users could repeatedly click the "Not interested" button on videos they actually want to see, in an attempt to further customize their recommendations.

The paper also does not delve into the underlying algorithms and data used by YouTube's recommendation system. A deeper understanding of these technical details could provide more insight into why the disinterest buttons have the observed effects (or lack thereof) on video recommendations.

Conclusion

This study sheds light on the empirical effectiveness of YouTube's disinterest buttons and users' awareness and perceptions of them. The results suggest that the "Not interested" button can be a useful tool for users to refine their recommendations, but its impact is limited, especially on individual video pages.

The findings highlight the need for YouTube to improve the transparency and effectiveness of its recommendation system, so users can better control the content they see. Ongoing research and collaboration between platforms, researchers, and users will be crucial to address these challenges and ensure recommendation systems better serve the needs of all.

This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

🎲

How to Train Your YouTube Recommender to Avoid Unwanted Videos

Alexander Liu, Siqi Wu, Paul Resnick

YouTube provides features for users to indicate disinterest when presented with unwanted recommendations, such as the Not interested and Don't recommend channel buttons. These buttons purportedly allow the user to correct mistakes made by the recommendation system. Yet, relatively little is known about the empirical efficacy of these buttons. Neither is much known about users' awareness of and confidence in them. To address these gaps, we simulated YouTube users with sock puppet agents. Each agent first executed a stain phase, where it watched many videos of an assigned topic; it then executed a scrub phase, where it tried to remove recommendations from the assigned topic. Each agent repeatedly applied a single scrubbing strategy, either indicating disinterest in one of the videos visited in the stain phase (disliking it or deleting it from the watch history), or indicating disinterest in a video recommended on the homepage (clicking the not interested or don't recommend channel button or opening the video and clicking the dislike button). We found that the stain phase significantly increased the fraction of the recommended videos dedicated to the assigned topic on the user's homepage. For the scrub phase, using the Not interested button worked best, significantly reducing such recommendations in all topics tested, on average removing 88% of them. Neither the stain phase nor the scrub phase, however, had much effect on videopage recommendations. We also ran a survey (N = 300) asking adult YouTube users in the US whether they were aware of and used these buttons before, as well as how effective they found these buttons to be. We found that 44% of participants were not aware that the Not interested button existed. Those who were aware of it often used it to remove unwanted recommendations (82.8%) and found it to be modestly effective (3.42 out of 5).

4/3/2024

✨

Leveraging Recommender Systems to Reduce Content Gaps on Peer Production Platforms

Mo Houtti, Isaac Johnson, Morten Warncke-Wang, Loren Terveen

Peer production platforms like Wikipedia commonly suffer from content gaps. Prior research suggests recommender systems can help solve this problem, by guiding editors towards underrepresented topics. However, it remains unclear whether this approach would result in less relevant recommendations, leading to reduced overall engagement with recommended items. To answer this question, we first conducted offline analyses (Study 1) on SuggestBot, a task-routing recommender system for Wikipedia, then did a three-month controlled experiment (Study 2). Our results show that presenting users with articles from underrepresented topics increased the proportion of work done on those articles without significantly reducing overall recommendation uptake. We discuss the implications of our results, including how ignoring the article discovery process can artificially narrow recommendations on peer production platforms.

4/11/2024

Harm Mitigation in Recommender Systems under User Preference Dynamics

Jerry Chee, Shankar Kalyanaraman, Sindhu Kiranmai Ernala, Udi Weinsberg, Sarah Dean, Stratis Ioannidis

We consider a recommender system that takes into account the interplay between recommendations, the evolution of user interests, and harmful content. We model the impact of recommendations on user behavior, particularly the tendency to consume harmful content. We seek recommendation policies that establish a tradeoff between maximizing click-through rate (CTR) and mitigating harm. We establish conditions under which the user profile dynamics have a stationary point, and propose algorithms for finding an optimal recommendation policy at stationarity. We experiment on a semi-synthetic movie recommendation setting initialized with real data and observe that our policies outperform baselines at simultaneously maximizing CTR and mitigating harm.

6/17/2024

Better Late Than Never: Formulating and Benchmarking Recommendation Editing

Chengyu Lai, Sheng Zhou, Zhimeng Jiang, Qiaoyu Tan, Yuanchen Bei, Jiawei Chen, Ningyu Zhang, Jiajun Bu

Recommendation systems play a pivotal role in suggesting items to users based on their preferences. However, in online platforms, these systems inevitably offer unsuitable recommendations due to limited model capacity, poor data quality, or evolving user interests. Enhancing user experience necessitates efficiently rectify such unsuitable recommendation behaviors. This paper introduces a novel and significant task termed recommendation editing, which focuses on modifying known and unsuitable recommendation behaviors. Specifically, this task aims to adjust the recommendation model to eliminate known unsuitable items without accessing training data or retraining the model. We formally define the problem of recommendation editing with three primary objectives: strict rectification, collaborative rectification, and concentrated rectification. Three evaluation metrics are developed to quantitatively assess the achievement of each objective. We present a straightforward yet effective benchmark for recommendation editing using novel Editing Bayesian Personalized Ranking Loss. To demonstrate the effectiveness of the proposed method, we establish a comprehensive benchmark that incorporates various methods from related fields. Codebase is available at https://github.com/cycl2018/Recommendation-Editing.

6/10/2024