Audeering

wav2vec2-large-robust-12-ft-emotion-msp-dim

audeering

77

The wav2vec2-large-robust-12-ft-emotion-msp-dim model is a fine-tuned version of the Wav2Vec2-Large-Robust model for dimensional speech emotion recognition. This model was created by audeering and fine-tuned on the MSP-Podcast dataset. It expects a raw audio signal as input and outputs predictions for arousal, dominance and valence in a range of approximately 0 to 1. The model also provides the pooled states of the last transformer layer. Similar models include the wav2vec2-large-960h-lv60-self model from Facebook, which was pre-trained and fine-tuned on 960 hours of speech data for speech recognition, as well as the wav2vec2-lg-xlsr-en-speech-emotion-recognition model, which was fine-tuned for speech emotion recognition. Model inputs and outputs Inputs Raw audio signal**: The model expects a raw audio signal as input, which must be sampled at the appropriate rate. Outputs Arousal, dominance and valence predictions**: The model outputs predictions for arousal, dominance and valence in a range of approximately 0 to 1. Pooled states**: The model also provides the pooled states of the last transformer layer. Capabilities The wav2vec2-large-robust-12-ft-emotion-msp-dim model is capable of dimensional speech emotion recognition, meaning it can predict the arousal, dominance, and valence of a given speech sample. This can be useful for applications that require understanding the emotional state of the speaker, such as customer service, mental health monitoring, or human-robot interaction. What can I use it for? You can use this model for various applications that require dimensional speech emotion recognition, such as: Customer service**: Analyze customer calls to better understand their emotional state and provide more personalized and empathetic support. Mental health monitoring**: Track the emotional state of patients or study participants over time to detect changes that may indicate mental health issues. Human-robot interaction**: Enable robots to better understand and respond to the emotional state of humans they interact with, leading to more natural and engaging interactions. Things to try One interesting thing to try with this model is to explore how the arousal, dominance, and valence predictions change across different speech samples or scenarios. You could, for example, analyze how the emotional state of a speaker varies during a conversation, or compare the emotional responses of different speakers to the same stimulus. Additionally, you could experiment with using the pooled states from the model as features in other machine learning tasks, such as speaker identification or acoustic event detection.

Read more