A structured regression approach for evaluating model performance across intersectional subgroups
0
↗️
Sign in to get full access
Overview
- Disaggregated evaluation is a crucial task in assessing the fairness of AI systems, where the goal is to measure performance across different subgroups defined by demographic or sensitive attributes.
- The standard approach involves stratifying the evaluation data and computing performance metrics separately for each subgroup, but this becomes challenging when dealing with small subgroups in moderately-sized datasets.
- This paper introduces a structured regression approach to disaggregated evaluation that can provide reliable system performance estimates even for very small subgroups.
- The paper also explores how goodness-of-fit testing can help identify the key factors that drive differences in performance across intersectional groups.
Plain English Explanation
When evaluating the fairness of an AI system, it's important to look at how the system performs across different subgroups of people, such as those defined by age, gender, or race. The typical way to do this is to split the evaluation data into these different subgroups and calculate the system's performance metrics (like accuracy or error rate) for each one.
However, even with a moderately-sized dataset, the sample sizes for some subgroups (especially those that combine multiple attributes, like "women over 65") can get very small. This makes it difficult to get reliable estimates of the system's performance for those subgroups.
The researchers in this paper propose a new approach that uses a structured regression model to estimate the system's performance across subgroups, even when the subgroup sizes are tiny. Their method can provide more accurate performance estimates than the standard approach, especially for those small subgroups.
Additionally, the paper shows how you can use statistical tests to understand the factors that are driving differences in the system's performance between different subgroups. This can help identify the key causes of unfairness in the system.
Technical Explanation
The paper introduces a structured regression approach to disaggregated evaluation of AI systems. The key idea is to model the relationship between the system's performance metric (e.g., accuracy) and the demographic/sensitive attributes used to define the subgroups.
Specifically, the researchers use a generalized additive model (GAM) to capture the main effects of each attribute, as well as any interactions between them. This structured model allows for reliable performance estimates even when some subgroup sample sizes are very small.
The paper also demonstrates how goodness-of-fit testing on the GAM can provide insights into the structure of fairness-related harms experienced by intersectional groups. For example, the tests can help identify which specific attribute combinations are driving the largest performance disparities.
The researchers evaluate their approach on two public datasets, as well as several variants of semi-synthetic data. The results show that their method outperforms the standard stratification approach, especially for small subgroups. The goodness-of-fit analysis also provides useful information about the nature of the fairness issues in the tested systems.
Critical Analysis
A key strength of this paper is the structured regression approach, which can overcome the limitations of the standard stratification method when dealing with small subgroups. By modeling the relationships between performance and demographic attributes, the method can provide more reliable estimates of system fairness, even in challenging data settings.
However, the paper does acknowledge some limitations. The GAM approach relies on certain assumptions about the functional form of the relationships, which may not always hold in practice. Additionally, the goodness-of-fit testing can identify relevant factors, but does not directly suggest ways to improve fairness or mitigate harms.
Further research could explore the robustness of the approach to model misspecification, as well as develop methods to translate the insights from the fairness analysis into actionable steps for improving the AI system.
Conclusion
This paper presents a structured regression approach to disaggregated evaluation of AI system fairness that can provide more reliable performance estimates, especially for small subgroups. The method also enables the use of goodness-of-fit testing to identify the key factors driving fairness-related disparities.
While the approach has some limitations, it represents an important step forward in addressing the challenges of intersectional fairness assessment. By providing a more robust and informative disaggregated evaluation framework, this work can help AI developers and researchers better understand and mitigate unfairness in their systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
↗️
0
A structured regression approach for evaluating model performance across intersectional subgroups
Christine Herlihy, Kimberly Truong, Alexandra Chouldechova, Miroslav Dudik
Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups defined by combinations of demographic or other sensitive attributes. The standard approach is to stratify the evaluation data across subgroups and compute performance metrics separately for each group. However, even for moderately-sized evaluation datasets, sample sizes quickly get small once considering intersectional subgroups, which greatly limits the extent to which intersectional groups are included in analysis. In this work, we introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups. We provide corresponding inference strategies for constructing confidence intervals and explore how goodness-of-fit testing can yield insight into the structure of fairness-related harms experienced by intersectional groups. We evaluate our approach on two publicly available datasets, and several variants of semi-synthetic data. The results show that our method is considerably more accurate than the standard approach, especially for small subgroups, and demonstrate how goodness-of-fit testing helps identify the key factors that drive differences in performance.
Read more5/15/2024
📊
0
Synthetic Data Generation for Intersectional Fairness by Leveraging Hierarchical Group Structure
Gaurav Maheshwari, Aur'elien Bellet, Pascal Denis, Mikaela Keller
In this paper, we introduce a data augmentation approach specifically tailored to enhance intersectional fairness in classification tasks. Our method capitalizes on the hierarchical structure inherent to intersectionality, by viewing groups as intersections of their parent categories. This perspective allows us to augment data for smaller groups by learning a transformation function that combines data from these parent groups. Our empirical analysis, conducted on four diverse datasets including both text and images, reveals that classifiers trained with this data augmentation approach achieve superior intersectional fairness and are more robust to ``leveling down'' when compared to methods optimizing traditional group fairness metrics.
Read more5/24/2024
0
Quantifying the Cross-sectoral Intersecting Discrepancies within Multiple Groups Using Latent Class Analysis Towards Fairness
Yingfang Yuan, Kefan Chen, Mehdi Rizvi, Lynne Baillie, Wei Pang
The growing interest in fair AI development is evident. The ''Leave No One Behind'' initiative urges us to address multiple and intersecting forms of inequality in accessing services, resources, and opportunities, emphasising the significance of fairness in AI. This is particularly relevant as an increasing number of AI tools are applied to decision-making processes, such as resource allocation and service scheme development, across various sectors such as health, energy, and housing. Therefore, exploring joint inequalities in these sectors is significant and valuable for thoroughly understanding overall inequality and unfairness. This research introduces an innovative approach to quantify cross-sectoral intersecting discrepancies among user-defined groups using latent class analysis. These discrepancies can be used to approximate inequality and provide valuable insights to fairness issues. We validate our approach using both proprietary and public datasets, including EVENS and Census 2021 (England & Wales) datasets, to examine cross-sectoral intersecting discrepancies among different ethnic groups. We also verify the reliability of the quantified discrepancy by conducting a correlation analysis with a government public metric. Our findings reveal significant discrepancies between minority ethnic groups, highlighting the need for targeted interventions in real-world AI applications. Additionally, we demonstrate how the proposed approach can be used to provide insights into the fairness of machine learning.
Read more7/12/2024
📈
0
Cross-model Fairness: Empirical Study of Fairness and Ethics Under Model Multiplicity
Kacper Sokol, Meelis Kull, Jeffrey Chan, Flora Salim
While data-driven predictive models are a strictly technological construct, they may operate within a social context in which benign engineering choices entail implicit, indirect and unexpected real-life consequences. Fairness of such systems -- pertaining both to individuals and groups -- is one relevant consideration in this space; algorithms can discriminate people across various protected characteristics regardless of whether these properties are included in the data or discernible through proxy variables. To date, this notion has predominantly been studied for a fixed model, often under different classification thresholds, striving to identify and eradicate undesirable, discriminative and possibly unlawful aspects of its operation. Here, we backtrack on this fixed model assumption to propose and explore a novel definition of cross-model fairness where individuals can be harmed when one predictor is chosen ad hoc from a group of equally well performing models, i.e., in view of utility-based model multiplicity. Since a person may be classified differently across models that are otherwise considered equivalent, this individual could argue for a predictor granting them the most favourable outcome, employing which may have adverse effects on other people. We introduce this scenario with a two-dimensional example and linear classification; then, we present a comprehensive empirical study based on real-life predictive models and data sets that are popular with the algorithmic fairness community; finally, we investigate analytical properties of cross-model fairness and its ramifications in a broader context. Our findings suggest that such unfairness can be readily found in real life and it may be difficult to mitigate by technical means alone as doing so is likely to degrade predictive performance.
Read more7/11/2024