Simple but Efficient: A Multi-Scenario Nearline Retrieval Framework for Recommendation on Taobao
0
Sign in to get full access
Overview
- This paper presents a simple but efficient multi-scenario nearline retrieval framework for recommendation on the Taobao platform.
- The framework combines multiple retrieval models to handle different recommendation scenarios, addressing the challenges of scale and efficiency.
- Key components include a multi-view user representation, a multi-scenario retrieval model, and an online-offline hybrid architecture.
Plain English Explanation
The paper describes a recommendation system used on the Taobao e-commerce platform. Recommendation systems help online stores suggest products that customers might be interested in buying.
The authors developed a framework that combines multiple recommendation models to handle different types of recommendation scenarios. This helps the system work efficiently at a large scale, even with many users and products.
The key ideas are:
-
Multi-View User Representation: The system creates a comprehensive user profile by combining different types of information about the user, such as their browsing history, purchases, and interactions with the platform.
-
Multi-Scenario Retrieval Model: The framework uses several different recommendation models, each tailored to a specific recommendation scenario, such as suggesting new products, re-engaging users, or cross-selling related items. This allows the system to provide relevant recommendations in various contexts.
-
Online-Offline Hybrid Architecture: The system has both an online component that responds quickly to user requests, and an offline component that periodically updates the recommendation models with the latest data. This hybrid approach balances speed and accuracy.
By using this multi-faceted framework, the Taobao recommendation system can provide personalized and relevant product suggestions to its large user base efficiently.
Technical Explanation
The paper introduces a Multi-Scenario Nearline Retrieval Framework for recommendation on the Taobao platform. The key components of the framework are:
-
Multi-View User Representation: The system creates a comprehensive user profile by combining information from different sources, such as browsing history, purchases, and user interactions. This adaptive utilization of cross-scenario information allows the recommendation models to have a more complete understanding of user preferences.
-
Multi-Scenario Retrieval Model: The framework includes multiple retrieval models, each designed for a specific recommendation scenario, such as new product discovery, re-engagement, and cross-selling. This comprehensive approach to retrieval methods allows the system to provide relevant recommendations in various contexts.
-
Online-Offline Hybrid Architecture: The system has both an online component that quickly responds to user requests, and an offline component that periodically updates the recommendation models with the latest data. This personalized and federated multi-scenario multi-task approach balances speed and accuracy.
The authors evaluate the framework on the Taobao platform and demonstrate its LLM-enhanced performance in terms of efficiency and effectiveness across multiple recommendation scenarios.
Critical Analysis
The paper presents a well-designed and practical recommendation framework that addresses the challenges of scale and efficiency in a large-scale e-commerce platform like Taobao. The authors' use of a multi-view user representation, multi-scenario retrieval models, and an online-offline hybrid architecture is a sensible approach to handling the complexity of real-world recommendation scenarios.
However, the paper does not provide much detail on the specific machine learning techniques and algorithms used in the retrieval models. Additionally, the evaluation is mostly focused on the system-level performance, and there is limited discussion of the individual model components and their relative contributions to the overall performance.
Further research could explore the performance of the individual retrieval models, as well as the impact of different user features and model architectures on the recommendation quality. Additionally, the framework could be tested on other e-commerce platforms or recommendation domains to assess its generalizability.
Conclusion
This paper introduces a simple but efficient multi-scenario nearline retrieval framework for recommendation on the Taobao platform. By combining a multi-view user representation, multi-scenario retrieval models, and an online-offline hybrid architecture, the system is able to provide personalized and relevant product recommendations at scale.
The framework's practical approach and demonstrated performance on the Taobao platform suggest that it could be a valuable model for other large-scale recommendation systems facing similar challenges of efficiency and scalability.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Simple but Efficient: A Multi-Scenario Nearline Retrieval Framework for Recommendation on Taobao
Yingcai Ma, Ziyang Wang, Yuliang Yan, Jian Wu, Yuning Jiang, Longbin Li, Wen Chen, Jianhang Huang
In recommendation systems, the matching stage is becoming increasingly critical, serving as the upper limit for the entire recommendation process. Recently, some studies have started to explore the use of multi-scenario information for recommendations, such as model-based and data-based approaches. However, the matching stage faces significant challenges due to the need for ultra-large-scale retrieval and meeting low latency requirements. As a result, the methods applied at this stage (collaborative filtering and two-tower models) are often designed to be lightweight, hindering the full utilization of extensive information. On the other hand, the ranking stage features the most sophisticated models with the strongest scoring capabilities, but due to the limited screen size of mobile devices, most of the ranked results may not gain exposure or be displayed. In this paper, we introduce an innovative multi-scenario nearline retrieval framework. It operates by harnessing ranking logs from various scenarios through Flink, allowing us to incorporate finely ranked results from other scenarios into our matching stage in near real-time. Besides, we propose a streaming scoring module, which selects a crucial subset from the candidate pool. Implemented on the Guess You Like (homepage of the Taobao APP), China's premier e-commerce platform, our method has shown substantial improvements-most notably, a 5% uptick in product transactions. Furthermore, the proposed approach is not only model-free but also highly efficient, suggesting it can be quickly implemented in diverse scenarios and demonstrate promising performance.
Read more8/7/2024
0
Adaptive Utilization of Cross-scenario Information for Multi-scenario Recommendation
Xiufeng Shu, Ruidong Han, Xiang Li, Wei Lin
Recommender system of the e-commerce platform usually serves multiple business scenarios. Multi-scenario Recommendation (MSR) is an important topic that improves ranking performance by leveraging information from different scenarios. Recent methods for MSR mostly construct scenario shared or specific modules to model commonalities and differences among scenarios. However, when the amount of data among scenarios is skewed or data in some scenarios is extremely sparse, it is difficult to learn scenario-specific parameters well. Besides, simple sharing of information from other scenarios may result in a negative transfer. In this paper, we propose a unified model named Cross-Scenario Information Interaction (CSII) to serve all scenarios by a mixture of scenario-dominated experts. Specifically, we propose a novel method to select highly transferable features in data instances. Then, we propose an attention-based aggregator module, which can adaptively extract relative knowledge from cross-scenario. Experiments on the production dataset verify the superiority of our method. Online A/B test in Meituan Waimai APP also shows a significant performance gain, leading to an average improvement in GMV (Gross Merchandise Value) of 1.0% for overall scenarios.
Read more7/30/2024
0
A Unified Search and Recommendation Framework Based on Multi-Scenario Learning for Ranking in E-commerce
Jinhan Liu, Qiyu Chen, Junjie Xu, Junjie Li, Baoli Li, Sulong Xu
Search and recommendation (S&R) are the two most important scenarios in e-commerce. The majority of users typically interact with products in S&R scenarios, indicating the need and potential for joint modeling. Traditional multi-scenario models use shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of individual tasks. This coarse-grained modeling approach does not effectively capture the differences between S&R scenarios. Furthermore, this approach does not sufficiently exploit the information across the global label space. These issues can result in the suboptimal performance of multi-scenario models in handling both S&R scenarios. To address these issues, we propose an effective and universal framework for Unified Search and Recommendation (USR), designed with S&R Views User Interest Extractor Layer (IE) and S&R Views Feature Generator Layer (FG) to separately generate user interests and scenario-agnostic feature representations for S&R. Next, we introduce a Global Label Space Multi-Task Layer (GLMT) that uses global labels as supervised signals of auxiliary tasks and jointly models the main task and auxiliary tasks using conditional probability. Extensive experimental evaluations on real-world industrial datasets show that USR can be applied to various multi-scenario models and significantly improve their performance. Online A/B testing also indicates substantial performance gains across multiple metrics. Currently, USR has been successfully deployed in the 7Fresh App.
Read more6/13/2024
0
A Comprehensive Survey on Retrieval Methods in Recommender Systems
Junjie Huang, Jizheng Chen, Jianghao Lin, Jiarui Qin, Ziming Feng, Weinan Zhang, Yong Yu
In an era dominated by information overload, effective recommender systems are essential for managing the deluge of data across digital platforms. Multi-stage cascade ranking systems are widely used in the industry, with retrieval and ranking being two typical stages. Retrieval methods sift through vast candidates to filter out irrelevant items, while ranking methods prioritize these candidates to present the most relevant items to users. Unlike studies focusing on the ranking stage, this survey explores the critical yet often overlooked retrieval stage of recommender systems. To achieve precise and efficient personalized retrieval, we summarize existing work in three key areas: improving similarity computation between user and item, enhancing indexing mechanisms for efficient retrieval, and optimizing training methods of retrieval. We also provide a comprehensive set of benchmarking experiments on three public datasets. Furthermore, we highlight current industrial applications through a case study on retrieval practices at a specific company, covering the entire retrieval process and online serving, along with practical implications and challenges. By detailing the retrieval stage, which is fundamental for effective recommendation, this survey aims to bridge the existing knowledge gap and serve as a cornerstone for researchers interested in optimizing this critical component of cascade recommender systems.
Read more8/1/2024