Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review

Overview

• This scoping review examines how biases related to Generative Artificial Intelligence (GAI) models like ChatGPT have been researched and discussed in the context of higher education institutions (HEIs).
• The study categorizes the potential societal biases that GAI might cause in higher education, and highlights the need for more in-depth studies and empirical work to understand the specific biases that these models might introduce or amplify.

Plain English Explanation

Generative AI models like ChatGPT are becoming increasingly popular in higher education, with students, faculty, and staff using them for a variety of tasks. However, these models may inherit or amplify biases that exist in the data they are trained on, which could have significant implications in educational settings.

This review aimed to understand how researchers have been looking at the issue of bias in the use of GAI models in higher education. The researchers categorized the different types of societal biases that these models might introduce or worsen, such as biases related to race, gender, or socioeconomic status.

The review found that while there is meaningful discussion about bias and discrimination in the broader AI field, most articles addressing the use of GAI in higher education only touch on the issue superficially. There is a lack of detailed, empirical research on the specific biases that might arise in different educational contexts, such as in the humanities, social sciences, medicine, or engineering.

The review highlights the need for more in-depth studies to understand the potential biases introduced by GAI models in higher education, so that these issues can be addressed and more ethical AI applications can be developed for use in educational settings.

Technical Explanation

The scoping review analyzed recent academic publications in English, Chinese, and Japanese across four main databases, focusing on the use of GAI in higher education and the associated biases. The researchers categorized the potential societal biases that GAI might cause in the field of higher education, such as biases related to race, gender, socioeconomic status, and other demographic factors.

The findings reveal that while there is meaningful scholarly discussion around bias and discrimination concerning large language models (LLMs) in the broader AI field, most articles addressing higher education only approach the issue superficially. Few articles identify specific types of bias under different circumstances, and there is a notable lack of empirical research.

Most papers in the review focus primarily on educational and research fields related to medicine and engineering, with some addressing English education. However, there is almost no discussion regarding the potential biases in the humanities and social sciences. Additionally, a significant portion of the current discourse is in English and primarily addresses English-speaking contexts.

Critical Analysis

The review highlights the need for more in-depth, empirical studies to understand the specific biases that GAI models like ChatGPT might introduce or amplify in different educational settings, such as the humanities, social sciences, and non-English-speaking contexts.

While the review provides a useful summary of the current state of research in this area, it also acknowledges the limitations of the existing literature. The lack of detailed, context-specific studies makes it challenging to develop comprehensive strategies for addressing the ethical implications of GAI use in higher education.

Furthermore, the review does not delve into the potential societal biases that may be reflected in the training data used to develop these GAI models, as discussed in other research. Addressing these underlying biases may be crucial for mitigating the issues identified in the higher education context.

Conclusion

This scoping review highlights the urgent need for more research and empirical investigation into the ethical implications of using Generative Artificial Intelligence models like ChatGPT in higher education institutions. The potential for these models to inherit or amplify societal biases poses significant challenges that must be addressed to ensure the equitable and responsible use of AI in educational settings.

The review calls for a deeper understanding of the specific biases that may arise in different academic disciplines and contexts, as well as the development of strategies and guidelines to help higher education institutions navigate the ethical considerations around the use of these powerful AI technologies.