Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models

Overview

• Large language models (LLMs) are widely used, but can encode societal biases, leading to harmful effects on users.
• Most bias research focuses on Western contexts, with less attention to biases in the Global South.
• This paper quantifies stereotypical biases in popular LLMs using an Indian-centric dataset called Indian-BhED.

Plain English Explanation

Large language models are AI systems that can generate human-like text. These models are now used by millions of people every day. However, these models can pick up on and perpetuate societal biases, which can lead to harmful effects on the people who use them.

Most of the research on bias in language models has focused on biases in Western countries, like gender and racial biases. Comparatively less attention has been paid to biases in the Global South, such as biases related to caste and religion in India.

In this paper, the researchers quantify the level of stereotypical biases in popular language models using a new dataset called Indian-BhED. This dataset contains examples of stereotypical and anti-stereotypical beliefs about caste and religion in India.

The researchers found that many of the language models they tested, including GPT-2, GPT-2 Large, and GPT 3.5, have a strong tendency to generate stereotypical outputs, especially when it comes to caste (63-79%) and religion (69-72%) in the Indian context. This is higher than the biases traditionally studied in Western contexts, like gender and race.

The paper investigates possible reasons for these harmful biases in language models and suggests ways to reduce both stereotypical and anti-stereotypical biases. The findings highlight the need to include more diverse perspectives when studying fairness in AI and evaluating language models.

Technical Explanation

The researchers used a new dataset called Indian-BhED to quantify stereotypical biases in popular large language models (LLMs). Indian-BhED contains examples of stereotypical and anti-stereotypical beliefs about caste and religion in India.

The researchers tested several widely-used LLMs, including GPT-2, GPT-2 Large, and GPT 3.5. They measured the models' propensity to output stereotypical versus anti-stereotypical statements for each axis of bias (caste and religion).

The results show that the majority of LLMs tested have a strong tendency to generate stereotypical outputs, especially for caste (63-79%) and religion (69-72%). This is notably higher than biases traditionally studied in Western contexts, such as gender and race.

The researchers investigate potential causes for these harmful biases, including the models' training data and architectures. They also propose intervention techniques, such as targeted debiasing and adversarial training, to reduce both stereotypical and anti-stereotypical biases in LLMs.

Critical Analysis

The paper provides valuable insights into the presence of stereotypical biases in LLMs, particularly in the context of the Global South. However, the research is limited to a single dataset (Indian-BhED) and a handful of popular LLMs. Additional datasets and models from diverse cultural contexts would be needed to fully understand the extent and manifestations of bias in LLMs globally.

The authors acknowledge that their dataset may not capture the full complexity of social biases in India, and that further research is needed to develop more comprehensive benchmarks. They also note that their intervention techniques require further testing and refinement to be effective at reducing harmful biases in real-world applications.

It would be valuable for future research to explore the societal impacts of these biases, such as how they may contribute to the perpetuation of systemic discrimination. Additionally, more work is needed to understand the role of LLM developers and users in addressing bias and promoting fairness in AI systems.

Conclusion

This paper highlights the need for a more inclusive and global approach to studying and mitigating biases in large language models. The findings demonstrate that popular LLMs exhibit strong stereotypical biases, particularly in the context of caste and religion in India, which have been underexplored compared to biases in Western contexts.

The researchers' work underscores the importance of incorporating diverse perspectives and experiences when developing AI systems and evaluating their fairness. By expanding the scope of bias research to include the Global South, the field can work towards creating more inclusive and equitable language models that better serve the needs of all users.