Predicting Sustainable Development Goals Using Course Descriptions -- from LLMs to Conventional Foundation Models

Overview

• This paper explores using language models to predict Sustainable Development Goals (SDGs) from course descriptions.
• The researchers compare the performance of large language models (LLMs) like GPT-3 to conventional foundation models on this task.
• They analyze the strengths and limitations of different model types in capturing SDG-relevant information from course descriptions.

Plain English Explanation

The paper looks at using artificial intelligence (AI) models to predict the Sustainable Development Goals (SDGs) that are relevant to the content of university course descriptions. SDGs are a set of global goals established by the United Nations to address challenges like poverty, inequality, and climate change.

The researchers tested different types of AI models, including the very large language models (LLMs) like GPT-3 that have become well-known in recent years, as well as more conventional "foundation" models that are trained on a broader set of data. They wanted to see how well these models could identify the SDGs associated with the topics covered in course descriptions.

By comparing the performance of the different model types, the researchers aim to understand the strengths and weaknesses of each approach when it comes to capturing information about sustainable development in educational content. This could help inform the use of AI systems for tasks like curriculum planning, student advising, or automated assessment of educational materials.

Technical Explanation

The paper presents a study that uses language models to predict the Sustainable Development Goals (SDGs) associated with university course descriptions. The researchers compare the performance of large language models (LLMs) like GPT-3 to more conventional foundation models on this task.

The dataset used consists of course descriptions from a national longitudinal dataset on skills taught in US higher education. The researchers fine-tuned the LLM and foundation model approaches on this data to build predictive models for the 17 SDG categories.

Through their experiments, the authors find that the LLMs generally outperform the foundation models in predicting the SDGs associated with course content. However, they also identify some limitations of the LLMs, such as difficulties in accurately classifying courses related to certain SDGs. The results suggest that a combination of model types may be most effective for this application.

Critical Analysis

The paper provides a thorough evaluation of using language models to predict SDGs from course descriptions. The researchers acknowledge limitations in their dataset, such as potential biases in the course descriptions, that could impact model performance.

One area that could be explored further is the interpretability of the models' SDG predictions. Understanding the specific textual cues the models use to associate courses with SDGs could provide insights into the models' reasoning and potentially lead to improvements.

Additionally, the paper does not deeply examine the potential societal implications of using AI systems for this task. There could be concerns around algorithmic bias or unintended consequences that should be considered, especially given the importance of the SDGs.

Overall, the study represents a valuable contribution to research on applying language models to educational datasets. The findings could inform the development of intelligent systems to support sustainable development initiatives in higher education.

Conclusion

This paper investigates the use of language models, including both large language models and more conventional foundation models, to predict the Sustainable Development Goals (SDGs) associated with university course descriptions. The researchers find that the large language models generally outperform the foundation models on this task, but also identify limitations in the LLMs' ability to accurately classify certain SDG categories.

The results highlight the potential of language models to support tasks related to sustainable development in education, such as curriculum planning, student advising, and content evaluation. However, the authors also note the need to further explore issues of model interpretability and potential societal impacts.

Overall, this work contributes valuable insights into the strengths and weaknesses of different AI modeling approaches for extracting sustainability-relevant information from educational data. As AI systems become more widely adopted in educational settings, research like this will be crucial for ensuring these technologies are developed and deployed responsibly.