VietMed: A Dataset and Benchmark for Automatic Speech Recognition of Vietnamese in the Medical Domain

Overview

• This paper presents VietMed, a large-scale Vietnamese speech recognition dataset in the medical domain.
• VietMed includes 16 hours of labeled medical speech, 1000 hours of unlabeled medical speech, and 1200 hours of unlabeled general-domain speech.
• VietMed is the world's largest public medical speech recognition dataset in multiple aspects, including total duration, number of speakers, diseases covered, recording conditions, speaker roles, unique medical terms, and accents.
• The authors also release pre-trained and fine-tuned speech recognition models for Vietnamese, including the first public large-scale models for medical ASR.

Plain English Explanation

Speech recognition technology is becoming increasingly important in the medical field, as it can help healthcare providers more efficiently document patient information and interactions. However, a lack of publicly available datasets in the medical domain has posed a challenge for developing accurate speech recognition models.

To address this, the researchers created VietMed, a large-scale dataset of Vietnamese medical speech. This dataset includes over 16 hours of labeled medical speech, 1000 hours of unlabeled medical speech, and 1200 hours of unlabeled general-domain speech.

What makes VietMed unique is its size and comprehensiveness. It is the largest public medical speech recognition dataset in the world, covering a wide range of medical topics, speaker roles, recording conditions, and accents. This diversity helps ensure the models trained on VietMed can handle the real-world variability found in medical settings.

In addition to the dataset, the researchers also released pre-trained and fine-tuned speech recognition models for Vietnamese, including models specifically adapted for the medical domain. These models can serve as a valuable starting point for others working on speech recognition in the Vietnamese language, particularly in medical applications.

Technical Explanation

The researchers created the VietMed dataset to address the lack of publicly available speech recognition datasets in the medical domain, particularly for the Vietnamese language. VietMed includes 16 hours of labeled medical speech, 1000 hours of unlabeled medical speech, and 1200 hours of unlabeled general-domain speech.

Compared to other publicly available datasets, VietMed is the largest in several key aspects:

• Total duration: VietMed has over 2,200 hours of audio, far exceeding other medical speech datasets.
• Number of speakers: VietMed includes speech from thousands of speakers, providing greater diversity.
• Diseases covered: VietMed covers all 22 disease groups in the ICD-10 classification system.
• Recording conditions: The dataset includes speech recorded in various clinical settings, such as hospitals and clinics.
• Speaker roles: VietMed includes speech from healthcare providers, patients, and administrative staff.
• Unique medical terms: The dataset contains a large vocabulary of specialized medical terminology.
• Accents: VietMed represents all major accents within Vietnam, ensuring the models can handle diverse pronunciation.

In addition to the dataset, the researchers released pre-trained speech recognition models for Vietnamese, including w2v2-Viet and XLSR-53-Viet. They also fine-tuned these models on the VietMed dataset, creating specialized medical ASR models.

Interestingly, the researchers found that even without any medical data in the unsupervised pre-training, the XLSR-53-Viet model was able to generalize well to the medical domain. On the VietMed test set, XLSR-53-Viet achieved a word error rate (WER) of 29.6%, outperforming the base XLSR-53 model's WER of 51.8% - a relative reduction of over 40%.

Critical Analysis

The researchers have done an impressive job in creating the VietMed dataset and releasing high-quality pre-trained and fine-tuned speech recognition models. The dataset's breadth and depth are particularly noteworthy, making it a valuable resource for the research community.

However, the paper does not provide much insight into the specific challenges or limitations encountered during the data collection process. For example, it would be helpful to know more about the distribution of speaker demographics, the quality control measures used, or any biases present in the data.

Additionally, while the researchers demonstrate the effectiveness of their models on the VietMed test set, it would be informative to see how the models perform on other medical speech recognition benchmarks, such as those used in comprehensive studies of language models for clinical and biomedical applications. This would give a better sense of the models' generalization capabilities.

Overall, the VietMed dataset and associated models are a significant contribution to the field of medical speech recognition, particularly for the Vietnamese language. With continued research and refinement, these resources have the potential to greatly improve the efficiency and accuracy of medical documentation and patient-provider interactions.

Conclusion

In this work, the researchers have presented VietMed, a large-scale Vietnamese speech recognition dataset focused on the medical domain. VietMed is the world's largest public medical speech dataset, covering a wide range of medical topics, speaker roles, recording conditions, and accents.

The researchers have also released pre-trained and fine-tuned speech recognition models for Vietnamese, including the first public large-scale models for medical ASR. These models demonstrate impressive performance on the VietMed dataset, highlighting the potential of this resource to advance speech recognition technology in the medical field.

Overall, the VietMed dataset and associated models are a significant contribution to the research community, providing a valuable tool for developing more accurate and robust medical speech recognition systems, particularly for the Vietnamese language.