Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals

Overview

• Current automated essay scoring (AES) methods show high agreement with human raters, but their scoring mechanisms are not fully explored.
• The proposed method uses counterfactual intervention assisted by Large Language Models (LLMs) to reveal that BERT-like models primarily focus on sentence-level features, while LLMs are attuned to conventions, language complexity, and organization.
• This approach improves understanding of neural AES methods and can also apply to other domains seeking transparency in model-driven decisions.

Plain English Explanation

Automated essay scoring (AES) systems are computer programs that can grade essays, often with results that closely match human graders. However, the inner workings of these systems are not always clear. Researchers have developed a new method that uses large language models (LLMs) and "counterfactual interventions" to better understand how AES systems make their decisions.

By making small changes to essays and seeing how the scores change, the researchers found that BERT-like models tend to focus mainly on features at the sentence level, such as word choice and grammar. In contrast, the more advanced LLMs also consider higher-level aspects like the overall structure, complexity, and conventions of the writing.

This new approach provides a more comprehensive view of how these AES systems work, which could lead to improvements in their transparency and performance. The researchers believe it could also be applied to other areas where machine learning models make important decisions and need to be better understood.

Technical Explanation

The researchers proposed a method using counterfactual intervention assisted by Large Language Models (LLMs) to investigate the scoring mechanisms of automated essay scoring (AES) systems.

They found that BERT-like models, which are commonly used in AES, primarily focus on sentence-level features such as word choice and grammar when scoring essays. In contrast, the more advanced LLMs are attuned to higher-level aspects of writing, including conventions, language complexity, and organization.

By making small, targeted changes to essays and observing the resulting score changes, the researchers were able to gain insights into how these AES models arrive at their decisions. This approach revealed a more comprehensive alignment between the LLMs' scoring and typical essay scoring rubrics, suggesting that LLMs may be better equipped to provide meaningful feedback to students.

Additionally, the researchers found that LLMs can discern when counterfactual interventions have been made during the feedback process, indicating their potential for providing more transparent and informative scoring.

Critical Analysis

The researchers acknowledge that their study is limited to a specific set of essays and AES systems, and that further research is needed to validate their findings across a wider range of writing samples and model architectures.

Additionally, while the use of counterfactual interventions provides valuable insights, the researchers note that the process of generating and evaluating these interventions can be time-consuming and complex. There may be a need for more efficient or automated methods to make this approach more practical for real-world applications.

It is also worth considering the potential biases or limitations that may be inherent in the training data or model architectures used by the AES systems, and how these factors may influence the scoring decisions. The researchers do not address these issues in depth, but investigating model biases could be an important area for future research.

Conclusion

This study presents a novel approach to understanding the scoring mechanisms of automated essay scoring systems, using counterfactual intervention and Large Language Models. The findings suggest that LLMs may be better equipped to provide comprehensive and transparent feedback, aligning more closely with typical essay scoring rubrics.

The researchers have made their code and data publicly available, which could encourage further exploration and refinement of these techniques. As machine learning models play an increasingly important role in educational and other decision-making domains, understanding their inner workings will be crucial for ensuring fairness, transparency, and accountability.