The MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems
2
Sign in to get full access
Overview
The paper describes the creation of the MovieLens Beliefs Dataset, which collects pre-choice data from users of a movie recommendation system. This dataset aims to provide insights into users' beliefs and preferences before they make a choice, which can help improve recommendation algorithms. The dataset includes information on user beliefs about movie attributes, attitudes towards recommendations, and intended actions.
Plain English Explanation
When you use a movie recommendation service, the system tries to suggest films it thinks you'll enjoy. But how does the system know what you like? Typically, it looks at the movies you've watched in the past and uses that information to make new recommendations. However, this approach has limitations - it only considers your past behavior, not your underlying beliefs and preferences.
The researchers behind this paper wanted to get a better understanding of users' thoughts and feelings
By collecting this pre-choice data, the researchers hope to develop better recommendation algorithms that can more accurately predict what users will actually enjoy, rather than just relying on their past viewing history. This could lead to more personalized and satisfying movie recommendations in the future.
Technical Explanation
The researchers conducted a study with users of the MovieLens movie recommendation platform. When users visited the platform, they were shown a set of movie options and asked to provide information about their beliefs, attitudes, and intended actions related to those movies
Specifically, the dataset includes:
- Users' beliefs about various attributes of the movies, such as how entertaining, educational, or relevant the films were to the user's interests.
- Users' attitudes towards the movie recommendation system, including their trust in the system and willingness to accept its suggestions.
- Users' intended actions, such as whether they planned to watch the recommended movies.
This pre-choice data provides insights into the underlying factors that influence users' decision-making, going beyond just their past viewing history. The researchers hope that incorporating this information into recommendation algorithms will lead to more personalized and effective movie recommendations.
Critical Analysis
The MovieLens Beliefs Dataset represents an important step forward in understanding user preferences for recommender systems. By collecting data on users' beliefs, attitudes, and intended actions
However, the dataset does have some limitations. The study was conducted on the MovieLens platform, so the findings may not generalize to other recommendation domains or user populations. Additionally, the dataset only captures a single interaction between the user and the recommendation system, rather than tracking how user beliefs and preferences evolve over time.
Future research could explore ways to longitudinally collect pre-choice data, or to apply the approach to other recommendation domains, such as music recommendations or conversational movie recommendations. Researchers could also investigate ways to preserve user privacy while collecting this type of pre-choice data.
Additionally, the dataset could be used to study selection bias in recommender systems, as the pre-choice data provides a more complete picture of user preferences than just their past behavior.
Conclusion
The MovieLens Beliefs Dataset represents an important step forward in understanding the factors that influence user decision-making in recommender systems. By collecting pre-choice data on user beliefs, attitudes, and intended actions, the researchers hope to develop more personalized and effective recommendation algorithms that can better predict what users will actually enjoy.
While the dataset has some limitations, it opens up new avenues for research in areas such as multi-modal item recommendation, privacy-preserving data collection, and the mitigation of selection bias in recommender systems. Overall, this work highlights the value of going beyond just past user behavior to understand the broader factors that shape user preferences.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
2
The MovieLens Beliefs Dataset: Collecting Pre-Choice Data for Online Recommender Systems
Guy Aridor, Duarte Goncalves, Ruoyan Kong, Daniel Kluver, Joseph Konstan
An increasingly important aspect of designing recommender systems involves considering how recommendations will influence consumer choices. This paper addresses this issue by introducing a method for collecting user beliefs about un-experienced items - a critical predictor of choice behavior. We implemented this method on the MovieLens platform, resulting in a rich dataset that combines user ratings, beliefs, and observed recommendations. We document challenges to such data collection, including selection bias in response and limited coverage of the product space. This unique resource empowers researchers to delve deeper into user behavior and analyze user choices absent recommendations, measure the effectiveness of recommendations, and prototype algorithms that leverage user belief data, ultimately leading to more impactful recommender systems. The dataset can be found at https://grouplens.org/datasets/movielens/ml_belief_2024/.
Read more8/6/2024
0
What Are We Optimizing For? A Human-centric Evaluation of Deep Learning-based Movie Recommenders
Ruixuan Sun, Xinyi Wu, Avinash Akella, Ruoyan Kong, Bart Knijnenburg, Joseph A. Konstan
In the past decade, deep learning (DL) models have gained prominence for their exceptional accuracy on benchmark datasets in recommender systems (RecSys). However, their evaluation has primarily relied on offline metrics, overlooking direct user perception and experience. To address this gap, we conduct a human-centric evaluation case study of four leading DL-RecSys models in the movie domain. We test how different DL-RecSys models perform in personalized recommendation generation by conducting survey study with 445 real active users. We find some DL-RecSys models to be superior in recommending novel and unexpected items and weaker in diversity, trustworthiness, transparency, accuracy, and overall user satisfaction compared to classic collaborative filtering (CF) methods. To further explain the reasons behind the underperformance, we apply a comprehensive path analysis. We discover that the lack of diversity and too much serendipity from DL models can negatively impact the consequent perceived transparency and personalization of recommendations. Such a path ultimately leads to lower summative user satisfaction. Qualitatively, we confirm with real user quotes that accuracy plus at least one other attribute is necessary to ensure a good user experience, while their demands for transparency and trust can not be neglected. Based on our findings, we discuss future human-centric DL-RecSys design and optimization strategies.
Read more5/3/2024
0
Debiased Contrastive Representation Learning for Mitigating Dual Biases in Recommender Systems
Zhirong Huang, Shichao Zhang, Debo Cheng, Jiuyong Li, Lin Liu, Guixian Zhang
In recommender systems, popularity and conformity biases undermine recommender effectiveness by disproportionately favouring popular items, leading to their over-representation in recommendation lists and causing an unbalanced distribution of user-item historical data. We construct a causal graph to address both biases and describe the abstract data generation mechanism. Then, we use it as a guide to develop a novel Debiased Contrastive Learning framework for Mitigating Dual Biases, called DCLMDB. In DCLMDB, both popularity bias and conformity bias are handled in the model training process by contrastive learning to ensure that user choices and recommended items are not unduly influenced by conformity and popularity. Extensive experiments on two real-world datasets, Movielens-10M and Netflix, show that DCLMDB can effectively reduce the dual biases, as well as significantly enhance the accuracy and diversity of recommendations.
Read more8/20/2024
0
Large Language Models as Conversational Movie Recommenders: A User Study
Ruixuan Sun, Xinyi Li, Avinash Akella, Joseph A. Konstan
This paper explores the effectiveness of using large language models (LLMs) for personalized movie recommendations from users' perspectives in an online field experiment. Our study involves a combination of between-subject prompt and historic consumption assessments, along with within-subject recommendation scenario evaluations. By examining conversation and survey response data from 160 active users, we find that LLMs offer strong recommendation explainability but lack overall personalization, diversity, and user trust. Our results also indicate that different personalized prompting techniques do not significantly affect user-perceived recommendation quality, but the number of movies a user has watched plays a more significant role. Furthermore, LLMs show a greater ability to recommend lesser-known or niche movies. Through qualitative analysis, we identify key conversational patterns linked to positive and negative user interaction experiences and conclude that providing personal context and examples is crucial for obtaining high-quality recommendations from LLMs.
Read more5/1/2024