Validating an Instrument for Teachers' Acceptance of Artificial Intelligence in Education

Read original: arXiv:2406.10506 - Published 6/18/2024 by Shuchen Guo, Lehong Shi, Xiaoming Zhai
Total Score

0

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • This paper describes the development and validation of an instrument to measure teachers' acceptance of artificial intelligence (AI) in education.
  • The researchers used a rigorous process, including focus groups, expert reviews, and statistical analyses, to create a valid and reliable measure of teachers' perceptions of AI in the classroom.
  • The resulting instrument can be used to better understand teachers' readiness and attitudes towards incorporating AI technologies into their teaching practices.

Plain English Explanation

The paper focuses on creating a tool to measure how willing and open teachers are to using artificial intelligence (AI) in their classrooms. The researchers recognized that as AI technologies continue to advance, it's important to understand teachers' perspectives on this. They wanted to develop a reliable way to assess teachers' acceptance of AI, including factors like their beliefs about its usefulness, ease of use, and potential impacts.

To create this measurement tool, the researchers went through several steps. First, they held discussions with teachers to get their input on important issues related to AI in education. Then, they had experts review the draft instrument to make sure it was well-designed. Finally, they used statistical analysis to refine and validate the survey, ensuring it accurately captured teachers' views.

This validated instrument can now be used by researchers and schools to better understand how teachers feel about incorporating AI into their teaching. This knowledge can then inform efforts to help teachers feel more comfortable and confident using these new technologies in their classrooms.

Technical Explanation

The researchers followed a multi-stage process to develop and validate an instrument for measuring teachers' acceptance of AI in education. First, they conducted focus groups with pre-service teachers to identify key factors influencing TAAI. This informed the initial item pool for the survey instrument.

Next, the researchers had the draft instrument reviewed by a panel of experts, who provided feedback to improve the content and wording of the items. They then administered the revised survey to a sample of 366 pre-service teachers and conducted exploratory and confirmatory factor analyses.

The analyses revealed a four-factor structure underlying TAAI: Perceived Usefulness, Perceived Ease of Use, Social Influence, and Facilitating Conditions. The final 20-item instrument demonstrated good reliability and validity, supporting its use as a measure of teachers' acceptance of AI in educational contexts.

Critical Analysis

The researchers took a comprehensive and rigorous approach to developing and validating the TAAI instrument. By grounding the item generation in focus group discussions with teachers, they ensured the survey captured salient factors influencing acceptance. The expert review and statistical analyses further strengthened the instrument's content and construct validity.

However, the sample was limited to pre-service teachers, so the instrument's applicability to in-service teachers remains to be seen. Additionally, the study did not explore how TAAI might differ across subject areas or grade levels, which could provide valuable insights for targeted professional development efforts.

Future research could also investigate the relationship between TAAI and actual AI adoption and integration in classrooms. This would help determine the predictive validity of the instrument and its usefulness for identifying potential barriers to AI implementation.

Conclusion

This study provides a well-validated instrument for measuring teachers' acceptance of AI in education. The four-factor structure offers a comprehensive framework for understanding the key drivers of TAAI, which can inform the design of AI-based educational tools and training programs to better support teachers' needs and concerns. As AI continues to evolve in the classroom, this instrument can be a valuable tool for researchers and educators to assess and address teachers' readiness to adopt these emerging technologies.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

Total Score

0

Validating an Instrument for Teachers' Acceptance of Artificial Intelligence in Education

Shuchen Guo, Lehong Shi, Xiaoming Zhai

As artificial intelligence (AI) receives wider attention in education, examining teachers' acceptance of AI (TAAI) becomes essential. However, existing instruments measuring TAAI reported limited reliability and validity evidence and faced some design challenges, such as missing informed definitions of AI to participants. This study aimed to develop and validate a TAAI instrument, with providing sufficient evidence for high psychometric quality. Based on the literature, we first identified five dimensions of TAAI, including perceived usefulness, perceived ease of use, behavioral intention, self-efficacy, and anxiety, and then developed items to assess each dimension. We examined the face and content validity using expert review and think-aloud with pre-service teachers. Using the revised instrument, we collected responses from 274 pre-service teachers and examined the item discriminations to identify outlier items. We employed the confirmatory factor analysis and Cronbach's alpha to examine the construct validity, convergent validity, discriminant validity, and reliability. Results confirmed the dimensionality of the scale, resulting in 27 items distributed in five dimensions. The study exhibits robust validity and reliability evidence for TAAI, thus affirming its usefulness as a valid measurement instrument.

Read more

6/18/2024

👀

Total Score

0

The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges

Okan Bulut, Maggie Beiting-Parrish, Jodi M. Casabianca, Sharon C. Slater, Hong Jiao, Dan Song, Christopher M. Ormerod, Deborah Gbemisola Fabiyi, Rodica Ivan, Cole Walsh, Oscar Rios, Joshua Wilson, Seyma N. Yildirim-Erbasli, Tarid Wongvorachan, Joyce Xinle Liu, Bin Tan, Polina Morilova

The integration of artificial intelligence (AI) in educational measurement has revolutionized assessment methods, enabling automated scoring, rapid content analysis, and personalized feedback through machine learning and natural language processing. These advancements provide timely, consistent feedback and valuable insights into student performance, thereby enhancing the assessment experience. However, the deployment of AI in education also raises significant ethical concerns regarding validity, reliability, transparency, fairness, and equity. Issues such as algorithmic bias and the opacity of AI decision-making processes pose risks of perpetuating inequalities and affecting assessment outcomes. Responding to these concerns, various stakeholders, including educators, policymakers, and organizations, have developed guidelines to ensure ethical AI use in education. The National Council of Measurement in Education's Special Interest Group on AI in Measurement and Education (AIME) also focuses on establishing ethical standards and advancing research in this area. In this paper, a diverse group of AIME members examines the ethical implications of AI-powered tools in educational measurement, explores significant challenges such as automation bias and environmental impact, and proposes solutions to ensure AI's responsible and effective use in education.

Read more

6/28/2024

🤔

Total Score

0

Understanding Student and Academic Staff Perceptions of AI Use in Assessment and Feedback

Jasper Roe (James Cook University Singapore), Mike Perkins (British University Vietnam), Daniel Ruelle (VinUniversity)

The rise of Artificial Intelligence (AI) and Generative Artificial Intelligence (GenAI) in higher education necessitates assessment reform. This study addresses a critical gap by exploring student and academic staff experiences with AI and GenAI tools, focusing on their familiarity and comfort with current and potential future applications in learning and assessment. An online survey collected data from 35 academic staff and 282 students across two universities in Vietnam and one in Singapore, examining GenAI familiarity, perceptions of its use in assessment marking and feedback, knowledge checking and participation, and experiences of GenAI text detection. Descriptive statistics and reflexive thematic analysis revealed a generally low familiarity with GenAI among both groups. GenAI feedback was viewed negatively; however, it was viewed more positively when combined with instructor feedback. Academic staff were more accepting of GenAI text detection tools and grade adjustments based on detection results compared to students. Qualitative analysis identified three themes: unclear understanding of text detection tools, variability in experiences with GenAI detectors, and mixed feelings about GenAI's future impact on educational assessment. These findings have major implications regarding the development of policies and practices for GenAI-enabled assessment and feedback in higher education.

Read more

6/26/2024

🤖

Total Score

0

Generative AI Adoption in Classroom in Context of Technology Acceptance Model (TAM) and the Innovation Diffusion Theory (IDT)

Aashish Ghimire, John Edwards

The burgeoning development of generative artificial intelligence (GenAI) and the widespread adoption of large language models (LLMs) in educational settings have sparked considerable debate regarding their efficacy and acceptability.Despite the potential benefits, the assimilation of these cutting-edge technologies among educators exhibits a broad spectrum of attitudes, from enthusiastic advocacy to profound skepticism.This study aims to dissect the underlying factors influencing educators' perceptions and acceptance of GenAI and LLMs.We conducted a survey among educators and analyzed the data through the frameworks of the Technology Acceptance Model (TAM) and Innovation Diffusion Theory (IDT). Our investigation reveals a strong positive correlation between the perceived usefulness of GenAI tools and their acceptance, underscoring the importance of demonstrating tangible benefits to educators. Additionally, the perceived ease of use emerged as a significant factor, though to a lesser extent, influencing acceptance. Our findings also show that the knowledge and acceptance of these tools is not uniform, suggesting that targeted strategies are required to address the specific needs and concerns of each adopter category to facilitate broader integration of AI tools.in education.

Read more

6/26/2024