Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers

Overview

• Student evaluations are the primary way to provide feedback to academics on their teaching
• However, for large classes, the volume of feedback makes these tools impractical
• This paper explores using open-source generative AI to synthesize summaries of student feedback and actionable items for instructors

Plain English Explanation

Professors often rely on student feedback to improve their teaching, usually in the form of end-of-semester evaluations. However, for large classes with many students, sorting through all the feedback can be overwhelming. This paper looks at using generative AI models to automatically summarize the student feedback and provide actionable suggestions for the professor.

The researchers had 742 student responses across 75 different computer science courses. For each course, they used the AI to create a summary of the feedback and highlight specific areas the professor could work on to enhance their teaching. The results show this is a promising approach to give professors useful insights in a more efficient way, compared to manually reviewing all the individual student comments. This demonstrates how open-source language models can be leveraged to support educators' development and improve teaching practices in the classroom.

Technical Explanation

The researchers had access to 742 student evaluation responses across 75 courses in a Computer Science department. They used open-source generative AI models to synthesize a summary of the feedback for each course, including both factual observations and actionable items for the instructor.

The key steps were:

1. Collecting the raw student feedback data
2. Feeding this data into the generative AI models to produce a concise summary
3. Reviewing the summaries to ensure they were factual, actionable, and appropriate

The results indicate this approach can provide a cost-effective way to give instructors meaningful feedback, which is especially valuable for large classes where manually reviewing all the responses is impractical. The ability of language models to generate nuanced and contextual feedback is a core part of what enables this application.

Critical Analysis

The paper demonstrates the potential of using generative AI to streamline the process of gathering and summarizing student feedback on teaching. However, it does not address some important considerations:

• The summaries were not validated against the original student comments to ensure accuracy and completeness. More rigorous evaluation is needed.
• The generative models could introduce biases or inappropriate language that undermines the goal of providing constructive, actionable feedback. Careful prompt engineering and monitoring is required.
• The study only looked at a single department. Broader testing across different disciplines and class sizes would strengthen the generalizability of the findings.

Overall, this is a promising initial exploration, but further research is needed to fully assess the efficacy and limitations of using generative AI for this purpose.

Conclusion

This paper explores a novel application of open-source generative AI to streamline the process of providing feedback to instructors based on student evaluations of their teaching. The results suggest this approach holds promise as a cost-effective way to give professors actionable insights, especially for large classes.

However, the validity and reliability of the AI-generated summaries requires more rigorous evaluation. Careful consideration of potential biases and appropriate use of language is also crucial. With further refinement and testing, this technique could become a valuable tool to support the professional development of educators and improve teaching practices in the classroom.