Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course
0
Sign in to get full access
Overview
- Examines the appropriation of large language models (LLMs) in an undergraduate programming course
- Draws insights from social shaping theory and the technology appropriation model
- Explores how students' perceptions and usage of LLMs evolve over time
Plain English Explanation
This paper investigates how students in an undergraduate programming course appropriated and used large language models (LLMs) like ChatGPT. The researchers were interested in understanding how students' views and usage of these AI tools changed over the course of the semester.
The researchers used social shaping theory and the technology appropriation model as a framework to analyze the students' experiences. Social shaping theory suggests that the way people use technology is shaped by their social and cultural contexts. The technology appropriation model explains how users make sense of and integrate new technologies into their existing practices.
By observing the students and gathering their feedback, the researchers were able to see how the students' perceptions and usage of LLMs evolved. They found that as the students became more familiar with the capabilities and limitations of the LLMs, they started to use them in more strategic and productive ways, tailoring the tools to their specific needs.
The study provides valuable insights into how students can effectively leverage large language models in their learning and problem-solving processes. It also highlights the importance of understanding the social and contextual factors that shape the way people interact with emerging technologies like LLMs.
Technical Explanation
The researchers conducted a qualitative study to explore how students in an undergraduate programming course appropriated and used large language models (LLMs) like ChatGPT over the course of a semester. They drew on social shaping theory and the technology appropriation model as a theoretical framework to analyze the students' experiences.
Through observations, focus group discussions, and individual interviews, the researchers collected data on the students' perceptions, usage patterns, and challenges encountered with LLMs. The study followed a longitudinal approach, with data collected at different points throughout the semester to capture the evolving nature of the students' interactions with the technology.
The findings suggest that as the students became more familiar with the capabilities and limitations of the LLMs, they started to use them in more strategic and productive ways. Initially, the students expressed concerns about the ethical and academic integrity implications of using LLMs. However, over time, they developed a more nuanced understanding of the tools and began to tailor their usage to specific tasks and problem-solving needs.
The researchers also observed that the students' self-efficacy and confidence in their programming abilities increased as they learned to effectively leverage the LLMs to complement their own skills and knowledge. This aligns with the findings of other studies that have explored the potential of large language models to enhance and empower students in educational settings.
The study provides valuable insights into the social and contextual factors that shape the way people appropriate and use emerging technologies like large language models. The researchers suggest that understanding these dynamics can inform the development and implementation of LLMs in educational contexts to ensure they are used effectively and ethically.
Critical Analysis
The study provides a nuanced and contextualized understanding of how students appropriate and use large language models in an educational setting. The researchers' use of social shaping theory and the technology appropriation model as a theoretical framework offers a valuable lens for examining the complex interplay between the students, the technology, and the broader social and academic environment.
One potential limitation of the study is the relatively small sample size and the fact that it was conducted in a single undergraduate programming course. While the longitudinal approach and qualitative methods employed provide rich insights, the findings may not be fully generalizable to other educational contexts or student populations. Further research could explore the appropriation of LLMs in a wider range of courses and institutions to validate and expand the insights from this study.
Additionally, the paper does not delve deeply into the specific strategies and techniques the students developed for effectively leveraging LLMs in their programming tasks. More detailed examination of the students' problem-solving approaches and the specific ways they integrated the LLMs could provide additional practical guidance for educators and students.
Overall, the study represents an important contribution to the growing body of research on the educational applications of large language models and the need to understand the social and contextual factors that shape their appropriation and use.
Conclusion
This paper presents a nuanced exploration of how students in an undergraduate programming course appropriated and used large language models (LLMs) like ChatGPT over the course of a semester. Drawing on social shaping theory and the technology appropriation model, the researchers provide valuable insights into the evolving perceptions and usage patterns of the students as they engaged with these AI-powered tools.
The study highlights the importance of understanding the social and contextual factors that shape the way people interact with emerging technologies like LLMs. By observing the students' experiences, the researchers were able to see how the students' initial concerns and reservations gave way to more strategic and productive uses of the LLMs as they developed a deeper understanding of the tools' capabilities and limitations.
The findings suggest that effectively leveraging large language models in educational settings can enhance student self-efficacy and complement their existing skills and knowledge. This aligns with other research exploring the potential of LLMs to empower students and advance research in various domains.
Overall, this study provides a valuable contribution to the ongoing discourse on the integration of large language models in educational contexts, highlighting the importance of a nuanced, context-sensitive approach to the appropriation and use of these powerful AI tools.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course
Aadarsh Padiyath, Xinying Hou, Amy Pang, Diego Viramontes Vargas, Xingjian Gu, Tamara Nelson-Fromm, Zihan Wu, Mark Guzdial, Barbara Ericson
The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM's technical capabilities. Using the social shaping of technology theory as a guiding framework, our study explores how students' social perceptions influence their own LLM usage. We then examine the correlation of self-reported LLM usage with students' self-efficacy and midterm performances in an undergraduate programming course. Triangulating data from an anonymous end-of-course student survey (n = 158), a mid-course self-efficacy survey (n=158), student interviews (n = 10), self-reported LLM usage on homework, and midterm performances, we discovered that students' use of LLMs was associated with their expectations for their future careers and their perceptions of peer usage. Additionally, early self-reported LLM usage in our context correlated with lower self-efficacy and lower midterm scores, while students' perceived over-reliance on LLMs, rather than their usage itself, correlated with decreased self-efficacy later in the course.
Read more6/11/2024
🛸
0
Analyzing LLM Usage in an Advanced Computing Class in India
Anupam Garg, Aryaman Raina, Aryan Gupta, Jaskaran Singh, Manav Saini, Prachi Iiitd, Ronit Mehta, Rupin Oberoi, Sachin Sharma, Samyak Jain, Sarthak Tyagi, Utkarsh Arora, Dhruv Kumar
This study examines the use of large language models (LLMs) by undergraduate and graduate students for programming assignments in advanced computing classes. Unlike existing research, which primarily focuses on introductory classes and lacks in-depth analysis of actual student-LLM interactions, our work fills this gap. We conducted a comprehensive analysis involving 411 students from a Distributed Systems class at an Indian university, where they completed three programming assignments and shared their experiences through Google Form surveys. Our findings reveal that students leveraged LLMs for a variety of tasks, including code generation, debugging, conceptual inquiries, and test case creation. They employed a spectrum of prompting strategies, ranging from basic contextual prompts to advanced techniques like chain-of-thought prompting and iterative refinement. While students generally viewed LLMs as beneficial for enhancing productivity and learning, we noted a concerning trend of over-reliance, with many students submitting entire assignment descriptions to obtain complete solutions. Given the increasing use of LLMs in the software industry, our study highlights the need to update undergraduate curricula to include training on effective prompting strategies and to raise awareness about the benefits and potential drawbacks of LLM usage in academic settings.
Read more7/29/2024
👨🏫
0
CS1-LLM: Integrating LLMs into CS1 Instruction
Annapurna Vadaparty, Daniel Zingaro, David H. Smith IV, Mounika Padala, Christine Alvarado, Jamie Gorson Benario, Leo Porter
The recent, widespread availability of Large Language Models (LLMs) like ChatGPT and GitHub Copilot may impact introductory programming courses (CS1) both in terms of what should be taught and how to teach it. Indeed, recent research has shown that LLMs are capable of solving the majority of the assignments and exams we previously used in CS1. In addition, professional software engineers are often using these tools, raising the question of whether we should be training our students in their use as well. This experience report describes a CS1 course at a large research-intensive university that fully embraces the use of LLMs from the beginning of the course. To incorporate the LLMs, the course was intentionally altered to reduce emphasis on syntax and writing code from scratch. Instead, the course now emphasizes skills needed to successfully produce software with an LLM. This includes explaining code, testing code, and decomposing large problems into small functions that are solvable by an LLM. In addition to frequent, formative assessments of these skills, students were given three large, open-ended projects in three separate domains (data science, image processing, and game design) that allowed them to showcase their creativity in topics of their choosing. In an end-of-term survey, students reported that they appreciated learning with the assistance of the LLM and that they interacted with the LLM in a variety of ways when writing code. We provide lessons learned for instructors who may wish to incorporate LLMs into their course.
Read more6/26/2024
0
Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study
Wenhan Lyu (Rachel), Yimeng Wang (Rachel), Tingting (Rachel), Chung, Yifan Sun, Yixuan Zhang
The integration of AI assistants, especially through the development of Large Language Models (LLMs), into computer science education has sparked significant debate. An emerging body of work has looked into using LLMs in education, but few have examined the impacts of LLMs on students in entry-level programming courses, particularly in real-world contexts and over extended periods. To address this research gap, we conducted a semester-long, between-subjects study with 50 students using CodeTutor, an LLM-powered assistant developed by our research team. Our study results show that students who used CodeTutor (the experimental group) achieved statistically significant improvements in their final scores compared to peers who did not use the tool (the control group). Within the experimental group, those without prior experience with LLM-powered tools demonstrated significantly greater performance gain than their counterparts. We also found that students expressed positive feedback regarding CodeTutor's capability, though they also had concerns about CodeTutor's limited role in developing critical thinking skills. Over the semester, students' agreement with CodeTutor's suggestions decreased, with a growing preference for support from traditional human teaching assistants. Our analysis further reveals that the quality of user prompts was significantly correlated with CodeTutor's response effectiveness. Building upon our results, we discuss the implications of our findings for integrating Generative AI literacy into curricula to foster critical thinking skills and turn to examining the temporal dynamics of user engagement with LLM-powered tools. We further discuss the discrepancy between the anticipated functions of tools and students' actual capabilities, which sheds light on the need for tailored strategies to improve educational outcomes.
Read more5/6/2024