Performative Prediction with Bandit Feedback: Learning through Reparameterization

Overview

• Performative prediction is a framework for studying social prediction where the data distribution changes in response to the deployment of a model.
• Existing work in this field often makes assumptions that are easily violated in practice.
• This paper explores performative prediction problems without these assumptions.

Plain English Explanation

Performative prediction is a way of looking at how AI models can affect the data they are trained on. Normally, we assume the data stays the same no matter what model is used. But in some cases, the model itself can change the real-world data it is trying to predict.

For example, imagine an AI model that predicts the popularity of songs. If the model becomes very accurate, and music companies start using it to decide which songs to promote, that could actually change which songs become popular in the future. The model has "performed" and altered the data it was trying to predict.

Existing research on this topic usually relies on three key assumptions:

1. The "performative risk" (how the model affects the data) is a simple, convex function.
2. The relationship between the model and data distribution is known in advance.
3. The model has access to detailed information about how it affects the data.

But in the real world, these assumptions are often violated. This paper explores new ways to do performative prediction without needing those assumptions to hold.

Technical Explanation

The paper proposes a reparameterization framework that reformulates the performative prediction objective as a function of the induced data distribution, rather than the model parameters directly. This allows the researchers to transform the non-convex objective into a convex one, and develop a two-level optimization procedure to provably optimize it.

The first level of optimization iterates on the distribution parameter space, while the second level learns the model that induces a particular target distribution at each iteration. Under mild conditions, this approach achieves sublinear regret - meaning the model's performance improves over time and eventually gets very close to optimal.

Crucially, this method does not require the three assumptions that often limit existing performative prediction research. The authors show their approach can work even when the performative risk is non-convex, the relationship between model and data is unknown, and only zeroth-order (black-box) information about the performative risk is available.

Critical Analysis

The paper makes important theoretical contributions by expanding the scope of performative prediction to more realistic settings. However, the analysis is mostly theoretical, focusing on regret bounds and convexity properties. More empirical validation would be needed to assess the practical benefits and limitations of the proposed approach.

Additionally, the paper does not address some key challenges in performative prediction, such as how to handle distributional shift over time, or how to ensure the model converges to a socially desirable outcome when there are multiple stable equilibria. Further research is needed to tackle these important real-world concerns.

Conclusion

This paper takes a significant step forward in the field of performative prediction by developing a new optimization framework that does not rely on restrictive assumptions. By reparameterizing the objective and using a two-level optimization procedure, the authors show it's possible to achieve provable performance guarantees without requiring detailed knowledge of how the model affects the data distribution.

While more empirical validation is needed, this work opens up new possibilities for applying performative prediction to a wider range of real-world scenarios where the assumptions of prior research do not hold. As AI systems become increasingly pervasive, understanding and accounting for their performative effects will only grow more important.