Valuing an Engagement Surface using a Large Scale Dynamic Causal Model
0
Sign in to get full access
Overview
- Utilizes a large-scale, dynamic causal model to assess the value of an "engagement surface" - a set of features, content, and interactions that aim to engage users
- Focuses on quantifying the causal impact of the engagement surface on key business metrics like user retention, monetization, and growth
- Aims to provide a framework for investment decisions and optimizing the engagement surface to drive desired outcomes
Plain English Explanation
The paper describes a way to evaluate the impact of an "engagement surface" - a set of features, content, and interactions that are designed to keep users engaged with a product or service. The researchers use a large-scale, dynamic causal model to quantify how much the engagement surface causally affects important business metrics like user retention, monetization, and growth.
The key idea is to go beyond just looking at correlations and instead try to understand the underlying causal relationships. This can help guide investment decisions and allow the engagement surface to be optimized to drive the desired outcomes for the business.
Technical Explanation
The paper presents a dynamic causal model that captures the complex, time-varying relationships between the engagement surface and key business metrics. The model incorporates both observable features of the engagement surface as well as unobserved latent factors.
The researchers use a large-scale dataset spanning multiple products, channels, and user cohorts to estimate the model parameters. This allows them to quantify the causal impacts of different engagement surface elements on outcomes like retention, monetization, and growth.
The dynamic nature of the model enables understanding how these causal relationships evolve over time as users interact with the engagement surface. This provides insights into the long-term value of investment in the engagement surface.
Critical Analysis
The paper presents a sophisticated and data-driven approach to valuing an engagement surface. However, the reliance on observational data means there may be unobserved confounding factors that are not accounted for, potentially biasing the causal estimates.
Additionally, the large-scale nature of the model and dataset introduces challenges around computational complexity and interpretability. The authors acknowledge these limitations and suggest areas for further research, such as incorporating experimental interventions and developing more parsimonious modeling approaches.
Conclusion
Overall, this paper provides a valuable framework for quantifying the causal impact of an engagement surface on key business metrics. The dynamic causal modeling approach can help guide investment decisions and optimization of the engagement surface to drive desired outcomes. While the method has some limitations, it represents an important step forward in understanding the value of user engagement strategies.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Valuing an Engagement Surface using a Large Scale Dynamic Causal Model
Abhimanyu Mukerji, Sushant More, Ashwin Viswanathan Kannan, Lakshmi Ravi, Hua Chen, Naman Kohli, Chris Khawand, Dinesh Mandalapu
With recent rapid growth in online shopping, AI-powered Engagement Surfaces (ES) have become ubiquitous across retail services. These engagement surfaces perform an increasing range of functions, including recommending new products for purchase, reminding customers of their orders and providing delivery notifications. Understanding the causal effect of engagement surfaces on value driven for customers and businesses remains an open scientific question. In this paper, we develop a dynamic causal model at scale to disentangle value attributable to an ES, and to assess its effectiveness. We demonstrate the application of this model to inform business decision-making by understanding returns on investment in the ES, and identifying product lines and features where the ES adds the most value.
Read more8/23/2024
📉
0
Unboxing Engagement in YouTube Influencer Videos: An Attention-Based Approach
Prashant Rajaram, Puneet Manchanda
Influencer marketing videos have surged in popularity, yet significant gaps remain in understanding the relationship between video features and engagement. This challenge is intensified by the complexities of interpreting unstructured data. While deep learning models effectively leverage unstructured data to predict business outcomes, they often function as black boxes with limited interpretability, particularly when human validation is hindered by the absence of a known ground truth. To address this issue, the authors develop an interpretable deep learning framework that not only makes good out-of-sample predictions using unstructured data but also provides insights into the captured relationships. Inspired by visual attention in print advertising, the interpretation approach uses measures of model attention to video features, eliminating spurious associations through a two-step process and shortlisting relationships for formal causal testing. This method is applicable across well-known attention mechanisms - additive attention, scaled dot-product attention, and gradient-based attention - when analyzing text, audio, or video image data. Validated using simulations, this approach outperforms benchmark feature selection methods. This framework is applied to YouTube influencer videos, linking video features to measures of shallow and deep engagement developed based on the dual-system framework of thinking. The findings guide influencers and brands in prioritizing video features associated with deep engagement.
Read more8/27/2024
0
A General Model for Detecting Learner Engagement: Implementation and Evaluation
Somayeh Malekshahi, Javad M. Kheyridoost, Omid Fatemi
Considering learner engagement has a mutual benefit for both learners and instructors. Instructors can help learners increase their attention, involvement, motivation, and interest. On the other hand, instructors can improve their instructional performance by evaluating the cumulative results of all learners and upgrading their training programs. This paper proposes a general, lightweight model for selecting and processing features to detect learners' engagement levels while preserving the sequential temporal relationship over time. During training and testing, we analyzed the videos from the publicly available DAiSEE dataset to capture the dynamic essence of learner engagement. We have also proposed an adaptation policy to find new labels that utilize the affective states of this dataset related to education, thereby improving the models' judgment. The suggested model achieves an accuracy of 68.57% in a specific implementation and outperforms the studied state-of-the-art models detecting learners' engagement levels.
Read more5/8/2024
0
Uplift Modeling Under Limited Supervision
George Panagopoulos, Daniele Malitesta, Fragkiskos D. Malliaros, Jun Pang
Estimating causal effects in e-commerce tends to involve costly treatment assignments which can be impractical in large-scale settings. Leveraging machine learning to predict such treatment effects without actual intervention is a standard practice to diminish the risk. However, existing methods for treatment effect prediction tend to rely on training sets of substantial size, which are built from real experiments and are thus inherently risky to create. In this work we propose a graph neural network to diminish the required training set size, relying on graphs that are common in e-commerce data. Specifically, we view the problem as node regression with a restricted number of labeled instances, develop a two-model neural architecture akin to previous causal effect estimators, and test varying message-passing layers for encoding. Furthermore, as an extra step, we combine the model with an acquisition function to guide the creation of the training set in settings with extremely low experimental budget. The framework is flexible since each step can be used separately with other models or treatment policies. The experiments on real large-scale networks indicate a clear advantage of our methodology over the state of the art, which in many cases performs close to random, underlining the need for models that can generalize with limited supervision to reduce experimental risks.
Read more9/4/2024