Understanding Student and Academic Staff Perceptions of AI Use in Assessment and Feedback

Overview

• This study examines student and academic staff experiences with Artificial Intelligence (AI) and Generative AI (GenAI) tools in higher education, focusing on their familiarity and comfort with current and potential future applications in learning and assessment.
• The study collected data from 35 academic staff and 282 students across universities in Vietnam and Singapore through an online survey.
• Key findings include generally low familiarity with GenAI among both groups, negative perceptions of GenAI feedback, but more positive views when combined with instructor feedback, and differences in acceptance of GenAI text detection tools between academic staff and students.

Plain English Explanation

This study looks at how students and university staff are dealing with the rise of artificial intelligence (AI) and a specific type of AI called generative AI in higher education. The researchers wanted to understand how familiar these groups are with these technologies and how comfortable they feel using them for things like grading assignments, providing feedback, and checking student knowledge.

The researchers surveyed 35 university staff and 282 students at universities in Vietnam and Singapore. They found that both groups generally don't know much about generative AI yet. Students tended to view feedback from generative AI negatively, but they felt better about it when it was combined with feedback from their professors. University staff were more open to using AI tools to detect if students were cheating by copying text, and they were more comfortable making adjustments to student grades based on the results of these AI detectors.

The study also found that there was a lot of confusion and variability in how people had experienced these AI detectors so far, and people had mixed feelings about how generative AI might impact education assessments in the future. These findings suggest that universities need to develop clear policies and practices for using AI and generative AI in student assessments and feedback.

Technical Explanation

The study used an online survey to collect data from 35 academic staff and 282 students across two universities in Vietnam and one in Singapore. The survey examined the participants' familiarity with generative AI, their perceptions of using generative AI for assessment marking and feedback, their views on using it for knowledge checking and student participation, and their experiences with generative AI text detection tools.

Descriptive statistics and thematic analysis of the survey responses revealed that both academic staff and students generally had low familiarity with generative AI. Participants viewed generative AI feedback negatively, but had more positive perceptions when it was combined with feedback from instructors. Academic staff were more accepting of using generative AI text detection tools and making grade adjustments based on the results, compared to students.

The qualitative analysis identified three key themes: 1) unclear understanding of text detection tools, 2) variability in experiences with generative AI detectors, and 3) mixed feelings about the future impact of generative AI on educational assessment. These findings have significant implications for developing policies and practices around using generative AI-enabled assessment and feedback in higher education.

Critical Analysis

The study provides valuable insights into the current state of student and academic staff experiences with AI and generative AI tools in higher education assessment. However, it is important to note that the sample size, particularly for the academic staff, is relatively small, which may limit the generalizability of the findings.

Additionally, the study was conducted in Vietnam and Singapore, and the experiences of students and staff in other regions may differ. Further research is needed to explore how these issues play out in a wider range of educational contexts and cultures.

The study also did not delve deeply into the specific reasons behind the participants' perceptions and experiences. Understanding the underlying factors that shape these views would be helpful in developing more effective policies and practices for the use of AI and generative AI in assessment.

Overall, this study is an important step in addressing the critical gap in understanding the experiences and concerns of students and academic staff as higher education grapples with the rise of AI and generative AI. The findings highlight the need for continued research and dialogue to ensure that the implementation of these technologies in assessment is done in a way that supports learning and maintains academic integrity.

Conclusion

This study provides valuable insights into the current state of student and academic staff experiences with AI and generative AI tools in higher education assessment. The key findings include generally low familiarity with generative AI, negative perceptions of generative AI feedback, but more positive views when combined with instructor feedback, and differences in acceptance of generative AI text detection tools between academic staff and students.

These findings have significant implications for the development of policies and practices around the use of generative AI-enabled assessment and feedback in higher education. As AI and generative AI continue to evolve, it is crucial that universities work to address the concerns and needs of both students and academic staff to ensure that these technologies are implemented in a way that supports learning and maintains academic integrity.