A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
2402.13636
0
0
Abstract
Vision-language models (VLMs) have gained widespread adoption in both industry and academia. In this study, we propose a unified framework for systematically evaluating gender, race, and age biases in VLMs with respect to professions. Our evaluation encompasses all supported inference modes of the recent VLMs, including image-to-text, text-to-text, text-to-image, and image-to-image. Additionally, we propose an automated pipeline to generate high-quality synthetic datasets that intentionally conceal gender, race, and age information across different professional domains, both in generated text and images. The dataset includes action-based descriptions of each profession and serves as a benchmark for evaluating societal biases in vision-language models (VLMs). In our comparative analysis of widely used VLMs, we have identified that varying input-output modalities lead to discernible differences in bias magnitudes and directions. Additionally, we find that VLM models exhibit distinct biases across different bias attributes we investigated. We hope our work will help guide future progress in improving VLMs to learn socially unbiased representations. We will release our data and code.
Create account to get full access
Overview
- Proposes a unified framework and dataset for assessing gender bias in vision-language models
- Introduces a new dataset called Social Counterfactuals that includes visual scenes with diverse gender and racial representations
- Evaluates several state-of-the-art vision-language models on the dataset to uncover biases
Plain English Explanation
This research paper presents a new way to measure gender bias in AI systems that combine vision and language, such as image captioning or visual question answering models. The researchers created a dataset called Social Counterfactuals that contains images depicting diverse people in various scenarios.
By evaluating how well-known vision-language models perform on this dataset, the researchers were able to identify biases in how the models perceive and describe people of different genders. For example, the models might be more likely to associate women with domestic tasks and men with leadership roles.
The paper provides a framework for probing and understanding these biases, which is an important step towards building more equitable and inclusive AI systems. The Uncovering Bias in Large Vision-Language Models and Think Before You Act papers also address the important issue of bias in vision-language models.
Technical Explanation
The researchers introduce a new dataset called Social Counterfactuals that contains over 100,000 images depicting diverse people in a variety of scenarios. This dataset was designed to probe for intersectional biases related to gender, race, and social roles.
They then evaluate several state-of-the-art vision-language models, including CLIP, VinVL, and VisualBERT, on this dataset. The models are tasked with generating captions that describe the visual scenes. By analyzing the generated captions, the researchers are able to identify biases in how the models perceive and describe people of different genders.
The Uncovering Bias in Large Vision-Language Models and No Filter papers also explore techniques for uncovering and mitigating biases in vision-language models.
Critical Analysis
The researchers acknowledge that their dataset and evaluation framework have some limitations. For example, the dataset primarily focuses on gender and racial biases, but does not address other forms of social bias, such as those related to age, class, or disability.
Additionally, the researchers note that the vision-language models they evaluated were not specifically trained to avoid gender biases. Future work could explore techniques for probing and mitigating intersectional biases during the training process.
Overall, this research represents an important step towards understanding and addressing gender biases in vision-language models, which is crucial for developing more equitable and inclusive AI systems.
Conclusion
This paper presents a unified framework and dataset for assessing gender bias in vision-language models. By evaluating state-of-the-art models on the Social Counterfactuals dataset, the researchers were able to uncover biases in how these models perceive and describe people of different genders.
The findings from this research can inform the development of more equitable and inclusive AI systems, which is an important goal for the field. Continued work in this area, such as probing and mitigating intersectional biases, will be crucial for ensuring that AI technologies benefit all members of society.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model
Jie Zhang, Sibo Wang, Xiangkui Cao, Zheng Yuan, Shiguang Shan, Xilin Chen, Wen Gao
0
0
The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence. However, these advancements are tempered by the outputs that often reflect biases, a concern not yet extensively investigated. Existing benchmarks are not sufficiently comprehensive in evaluating biases due to their limited data scale, single questioning format and narrow sources of bias. To address this problem, we introduce VLBiasBench, a benchmark aimed at evaluating biases in LVLMs comprehensively. In VLBiasBench, we construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status). To create a large-scale dataset, we use Stable Diffusion XL model to generate 46,848 high-quality images, which are combined with different questions to form 128,342 samples. These questions are categorized into open and close ended types, fully considering the sources of bias and comprehensively evaluating the biases of LVLM from multiple perspectives. We subsequently conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models. Our benchmark is available at https://github.com/Xiangkui-Cao/VLBiasBench.
6/21/2024
🎲
SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples
Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwalla, Vasudev Lal
0
0
While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect to social attributes such as gender and race. Prior studies have primarily focused on probing such bias attributes individually while ignoring biases associated with intersections between social attributes. This could be due to the difficulty of collecting an exhaustive set of image-text pairs for various combinations of social attributes. To address this challenge, we employ text-to-image diffusion models to produce counterfactual examples for probing intersectional social biases at scale. Our approach utilizes Stable Diffusion with cross attention control to produce sets of counterfactual image-text pairs that are highly similar in their depiction of a subject (e.g., a given occupation) while differing only in their depiction of intersectional social attributes (e.g., race & gender). Through our over-generate-then-filter methodology, we produce SocialCounterfactuals, a high-quality dataset containing 171k image-text pairs for probing intersectional biases related to gender, race, and physical characteristics. We conduct extensive experiments to demonstrate the usefulness of our generated dataset for probing and mitigating intersectional social biases in state-of-the-art VLMs.
4/11/2024
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals
Phillip Howard, Kathleen C. Fraser, Anahita Bhiwandiwalla, Svetlana Kiritchenko
0
0
With the advent of Large Language Models (LLMs) possessing increasingly impressive capabilities, a number of Large Vision-Language Models (LVLMs) have been proposed to augment LLMs with visual inputs. Such models condition generated text on both an input image and a text prompt, enabling a variety of use cases such as visual question answering and multimodal chat. While prior studies have examined the social biases contained in text generated by LLMs, this topic has been relatively unexplored in LVLMs. Examining social biases in LVLMs is particularly challenging due to the confounding contributions of bias induced by information contained across the text and visual modalities. To address this challenging problem, we conduct a large-scale study of text generated by different LVLMs under counterfactual changes to input images. Specifically, we present LVLMs with identical open-ended text prompts while conditioning on images from different counterfactual sets, where each set contains images which are largely identical in their depiction of a common subject (e.g., a doctor), but vary only in terms of intersectional social attributes (e.g., race and gender). We comprehensively evaluate the text produced by different models under this counterfactual generation setting at scale, producing over 57 million responses from popular LVLMs. Our multi-dimensional analysis reveals that social attributes such as race, gender, and physical characteristics depicted in input images can significantly influence the generation of toxic content, competency-associated words, harmful stereotypes, and numerical ratings of depicted individuals. We additionally explore the relationship between social bias in LVLMs and their corresponding LLMs, as well as inference-time strategies to mitigate bias.
5/31/2024
Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts
Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, Yi Fang
0
0
Large vision-language models (LVLMs) have recently achieved significant progress, demonstrating strong capabilities in open-world visual understanding. However, it is not yet clear how LVLMs address demographic biases in real life, especially the disparities across attributes such as gender, skin tone, and age. In this paper, we empirically investigate emph{visual fairness} in several mainstream LVLMs and audit their performance disparities across sensitive demographic attributes, based on public fairness benchmark datasets (e.g., FACET). To disclose the visual bias in LVLMs, we design a fairness evaluation framework with direct questions and single-choice question-instructed prompts on visual question-answering/classification tasks. The zero-shot prompting results indicate that, despite enhancements in visual understanding, both open-source and closed-source LVLMs exhibit prevalent fairness issues across different instruct prompts and demographic attributes.
6/27/2024