nicolay-r at SemEval-2024 Task 3: Using Flan-T5 for Reasoning Emotion Cause in Conversations with Chain-of-Thought on Emotion States

Overview

• The paper explores the use of large language models for emotion expression analysis in conversations.
• It proposes a three-hop reasoning approach (THOR) to perform instruction-tuning for answering questions about emotion states and the emotion caused by one speaker to another.
• The THOR-cause model is equipped with reasoning revision (rr) to refine the reasoning path during fine-tuning, using annotated speaker emotion states.
• The final model, based on Flan-T5-base and a rule-based span correction technique, achieved top-3 and top-5 results in a competition.

Plain English Explanation

Conversations often involve the expression of emotions, which can be related to the speakers themselves or influenced by what the other person says. The researchers in this study looked at the different reasons behind these emotional reactions, such as the conversation history or the speakers' emotional states.

Inspired by recent advances in chain-of-thought reasoning, the researchers used a three-hop approach called THOR to train large language models to answer two types of questions: identifying the emotion states of the speakers (THOR-state) and determining the emotion caused by one speaker to the other (THOR-cause).

For the THOR-cause model, the researchers added a "reasoning revision" (rr) component, which uses the annotated emotion states of the speakers to refine the reasoning path during the fine-tuning process. This helps the model better understand the causal relationships between speakers' statements and the emotions they evoke.

The final model, built on the Flan-T5-base language model and a rule-based span correction technique, was tested in a competition and achieved impressive results, ranking 3rd, 4th, and 5th among 15 participating teams.

Technical Explanation

The researchers leveraged the THOR (Three-Hop Reasoning) approach to perform instruction-tuning of large language models for two tasks: identifying the emotion states of speakers (THOR-state) and determining the emotion caused by one speaker to the other (THOR-cause).

For the THOR-cause task, the researchers added a "reasoning revision" (rr) component, which uses the annotated emotion states of the speakers to refine the reasoning path during the fine-tuning process. This helps the model better understand the causal relationships between speakers' statements and the emotions they evoke.

The final model was built on the Flan-T5-base (250M) language model and used a rule-based span correction technique. It was first tuned with the THOR-state task and then fine-tuned with the THOR-cause-rr task, using the competition training data.

The results of this approach were impressive, with the final model achieving 3rd, 4th, and 5th places among 15 participating teams in the competition, based on F1-proportional and F1-strict metrics.

Critical Analysis

The paper presents a novel approach to emotion expression analysis in conversations by leveraging the THOR reasoning framework and incorporating a reasoning revision component for the THOR-cause task. This combination of techniques appears to have been effective in improving the model's performance on the competition tasks.

However, the paper does not provide much detail on the specific architectures or implementation details of the THOR-state and THOR-cause-rr models. Additionally, the dataset used for training and evaluation is not described in depth, which makes it difficult to assess the generalizability of the approach.

Furthermore, the paper does not discuss any potential limitations or caveats of the proposed method. For example, it would be interesting to understand how the THOR-cause-rr approach handles cases where the emotion caused by one speaker is not directly evident from the conversation history or the annotated emotion states.

Nonetheless, the researchers have made their THOR implementation fork publicly available, which allows other researchers to build upon their work and explore potential improvements or extensions.

Conclusion

This research demonstrates the potential of large language models, combined with advanced reasoning techniques, to tackle the challenge of emotion expression analysis in conversations. The THOR-state and THOR-cause-rr models, built on the Flan-T5-base architecture, achieved impressive results in a competition, showcasing the effectiveness of this approach.

While the paper leaves room for further exploration of the method's limitations and potential improvements, the researchers have made a valuable contribution to the field of emotion recognition and understanding in natural language processing. Their publicly available implementation provides a solid foundation for future work in this area.