Voice Passing : a Non-Binary Voice Gender Prediction System for evaluating Transgender voice transition

Overview

• This paper presents a software tool that can describe voices using a continuous Voice Femininity Percentage (VFP)
• The system is intended to help transgender speakers during their voice transition and voice therapists supporting them
• The researchers recorded a corpus of 41 French cis- and transgender speakers and had 57 participants estimate the VFP for each voice
• Machine learning models were trained on external data to predict gender and were then calibrated to estimate VFP, achieving higher accuracy than models based on pitch or vocal tract length

Plain English Explanation

The researchers have developed a software tool that can analyze someone's voice and provide a numerical estimate of how "feminine" or "masculine" it sounds on a continuous scale. This "Voice Femininity Percentage" (VFP) is intended to be useful for transgender individuals going through a voice transition, as well as the speech therapists who work with them.

To create this system, the researchers recorded voices from 41 French speakers, both cisgender (non-transgender) and transgender. They then had 57 human participants listen to the voices and estimate each one's VFP on a scale. The researchers then trained machine learning models on other datasets to predict the gender of voices. They calibrated these models to output the VFP values, and found them to be more accurate than simpler models based just on pitch or vocal tract length.

The accuracy of the VFP estimates was affected by factors like the speaking style and age of the speakers. This highlights the importance of considering social and cultural aspects of gender when building AI systems to represent these concepts.

Technical Explanation

The key elements of this paper are:

1. Corpus Collection: The researchers recorded a corpus of 41 French cis- and transgender speakers, which was used to train and evaluate their models.

2. Perceptual Evaluation: 57 participants listened to the voice recordings and provided estimates of the Voice Femininity Percentage (VFP) for each one on a continuous scale.

3. Gender Prediction Models: The researchers trained binary gender classification models on external gender-balanced datasets. They then used these models on overlapping windows of the voice recordings to obtain average gender prediction scores, which were calibrated to output the VFP values.

4. Model Evaluation: The researchers found that the calibrated gender prediction models achieved higher accuracy in estimating VFP compared to simpler models based on features like fundamental frequency (F0) or vocal tract length.

5. Impact of Factors: The researchers observed that the accuracy of the VFP estimates was affected by factors like the speaking style and age of the speakers, highlighting the importance of considering social and cultural aspects of gender when building such AI systems.

Critical Analysis

The paper acknowledges some important limitations and areas for further research. For example, the corpus was limited to French speakers, so the generalization to other languages and cultures is unclear. Additionally, the perceptual evaluation involved a relatively small number of raters, and their own biases and conceptions of gender may have influenced the VFP labels.

One potential issue not addressed in the paper is the ethical implications of such a system. While it may be useful for transgender individuals and their therapists, there are concerns about the potential for misuse, such as in surveillance or discrimination. The researchers could have discussed safeguards or guidelines for the responsible development and deployment of this technology.

Furthermore, the paper does not delve deeply into the sociocultural factors that shape perceptions of voice femininity. A more thorough exploration of these complex issues could provide valuable insights and inform the design of more inclusive and equitable voice analysis systems.

Conclusion

This paper presents a novel software tool that can estimate a continuous Voice Femininity Percentage (VFP) for voice recordings. The system is intended to support transgender individuals and their voice therapists during the transition process. The researchers found that machine learning models calibrated to predict VFP outperformed simpler approaches based on acoustic features alone.

The study highlights the importance of considering social and cultural factors, such as speaking style and age, when building AI systems to represent concepts like gender. Further research is needed to address the ethical implications of such technology and to more deeply explore the sociocultural complexities underlying perceptions of voice femininity.

Overall, this work represents an important step towards developing more inclusive and nuanced voice analysis tools, with potential applications in speech-to-speech and text-to-speech systems, as well as voice privacy and multimodal applications.