To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages

Overview

• This paper evaluates different approaches for cross-lingual transfer (XLT) using multilingual language models (mLMs) and machine translation (MT).
• The researchers investigate whether recent advances in massively multilingual MT can replace XLT with mLMs for low-resource language tasks.
• They systematically evaluate existing and propose new translation-based XLT approaches, showing that these dramatically outperform zero-shot XLT with mLMs.
• The researchers also explore ways to further improve translation-based XLT, such as adding reliable translations to the training data and proposing an effective strategy for languages not supported by the MT system.

Plain English Explanation

When you want to use a machine learning model to understand or generate text in a language that has limited data available, you can try cross-lingual transfer (XLT). This involves using a multilingual language model (mLM) that has been trained on many languages to perform the task in the low-resource language.

However, recent progress in machine translation (MT) has been so significant that the researchers wondered if it could completely replace the need for XLT with mLMs. They evaluated different approaches that use MT to help with low-resource language tasks, and found that these "translation-based" methods dramatically outperformed the zero-shot XLT approach with mLMs.

The researchers also discovered some ways to further improve the translation-based XLT methods. For example, adding high-quality translations of data from other languages to the training set can boost performance. They even came up with a way to do translation-based XLT for languages that aren't supported by the MT system being used.

Overall, the findings suggest that using MT-based approaches is a more robust and effective way to handle low-resource language tasks, compared to relying solely on multilingual language models.

Technical Explanation

The paper systematically evaluates existing and proposes new translation-based cross-lingual transfer (XLT) approaches for transferring to low-resource languages. The researchers compare these techniques to the standard zero-shot XLT approach using multilingual language models (mLMs).

The key translation-based XLT approaches evaluated include:

1. Round-trip translation of the source-language training data
2. Translation of the target-language test instances at inference time
3. Adding reliable translations of high-resource language data to the training set

The experiments show that all the translation-based methods dramatically outperform zero-shot XLT with mLMs. The combination of round-trip translation of the source data and translating the target test instances is generally the most effective approach.

The researchers also propose a strategy for doing translation-based XLT for languages not directly supported by the machine translation (MT) system. Additionally, they find that model selection for XLT can be improved by using target-language validation data obtained via MT, rather than relying solely on the source-language data.

Critical Analysis

The paper provides a thorough and systematic evaluation of translation-based approaches for cross-lingual transfer (XLT), which is an important problem in multilingual natural language processing. The key finding that translation-based methods significantly outperform zero-shot XLT with multilingual language models (mLMs) is a valuable insight.

However, the paper does not delve into the potential limitations or caveats of the translation-based approaches. For example, the quality of the machine translation (MT) system used could be a critical factor, and the performance gains may not generalize to low-resource language pairs not covered by the MT system.

Additionally, the paper does not address potential issues with the training data quality and translation reliability when adding translated high-resource language data to the training set. This could be an area for further research.

Overall, the findings warrant further exploration and a broader inclusion of translation-based baselines in XLT research. However, it would be valuable for future work to examine the limitations and robustness of these approaches, particularly for truly low-resource language settings.

Conclusion

This paper presents a comprehensive evaluation of translation-based approaches for cross-lingual transfer (XLT), demonstrating that they dramatically outperform the standard zero-shot XLT method using multilingual language models (mLMs). The researchers also propose several ways to further improve the translation-based XLT strategies, such as augmenting the training data with reliable translations and devising approaches for languages not directly supported by the machine translation (MT) system.

These findings suggest that translation-based techniques should be more widely adopted as robust baselines for XLT research, especially as massively multilingual MT systems continue to advance. However, further work is needed to fully understand the limitations and generalization of these approaches, particularly for truly low-resource language settings.