How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course

Overview

• This study explores how novice programmers in an introductory computer science course use and experience the AI language model ChatGPT when solving programming exercises.
• Researchers conducted interviews with students to understand their perceptions, strategies, and challenges when using ChatGPT for programming tasks.
• Key findings include the ways students leverage ChatGPT's capabilities, the potential benefits and drawbacks, and implications for programming education.

Plain English Explanation

The researchers were interested in how beginner programmers in an introductory computer science class use and feel about the AI assistant ChatGPT when working on programming assignments. They interviewed students to learn about their experiences, including how they use ChatGPT, what they find helpful or problematic about it, and what this means for teaching programming.

Some of the main ways students used ChatGPT included:

• Getting help with understanding programming concepts and syntax
• Generating code snippets or entire programs to solve problems
• Explaining code and debugging issues

Students appreciated ChatGPT's ability to provide explanations and generate code quickly. However, they also had concerns, such as over-reliance on the AI, difficulty verifying the correctness of the output, and uncertainty about academic integrity policies. The researchers discuss the implications of these findings for supporting students' learning and development as programmers.

Technical Explanation

The researchers conducted a qualitative study to investigate how novice programmers in an introductory computer science course used and experienced the language model ChatGPT when solving programming exercises. They interviewed 15 students about their perceptions, strategies, and challenges when using ChatGPT for programming tasks.

The study found that students leveraged ChatGPT in several ways, including:

• Understanding programming concepts and syntax
• Generating code snippets or entire programs to solve problems
• Explaining code and debugging issues

Students generally found ChatGPT helpful, as it could quickly provide explanations and generate code. However, they also expressed concerns, such as:

• Over-relying on the AI and not fully understanding the solutions
• Difficulty verifying the correctness of ChatGPT's outputs
• Uncertainty about academic integrity policies and guidelines for using ChatGPT

The researchers discuss the implications of these findings for programming education, suggesting the need to support students in developing productive ways of using large language models like ChatGPT, while also addressing potential challenges and risks.

Critical Analysis

The study provides valuable insights into how novice programmers are using and experiencing ChatGPT when solving programming exercises. The qualitative approach allows for a rich, nuanced understanding of the students' perspectives and strategies.

One potential limitation is the relatively small sample size of 15 students. While the in-depth interviews yielded detailed data, a larger and more diverse sample could strengthen the generalizability of the findings.

Additionally, the study focuses on a single introductory computer science course, so the experiences of students in different educational contexts or with varying levels of programming expertise may differ. Further research could explore ChatGPT use in a broader range of programming courses and levels.

The researchers acknowledge the need to address the potential risks and challenges associated with students' use of ChatGPT, such as academic integrity concerns and over-reliance on the AI. Exploring effective ways to integrate large language models like ChatGPT into programming education, while fostering students' fundamental programming skills and understanding, remains an important area for further investigation.

Conclusion

This study provides valuable insights into how novice programmers use and experience the AI language model ChatGPT when solving programming exercises in an introductory computer science course. The findings highlight the ways students leverage ChatGPT's capabilities, as well as the potential benefits and drawbacks they encounter.

The research suggests that as large language models continue to advance, it will be crucial for educators to support students in developing productive and responsible strategies for using these tools in programming education. By understanding the nuances of student experiences, educators can better integrate ChatGPT and similar technologies to enhance learning outcomes while addressing potential challenges.

Overall, this study contributes to the growing body of research on the use of AI assistants in programming education, offering insights that can inform both teaching practices and future technological developments in this domain.