Why Not Transform Chat Large Language Models to Non-English?

Overview

• The scarcity of non-English data limits the development of non-English large language models (LLMs).
• Transforming English-centric LLMs to non-English has been identified as an effective and resource-efficient method.
• Previous works start from base LLMs and perform knowledge distillation (KD) with data generated by stronger LLMs, e.g., GPT-4.
• Chat LLMs are further optimized for advanced abilities, e.g., multi-turn conversation and human preference alignment, and thus more powerful in both helpfulness and safety.
• Transforming a chat LLM involves two critical issues: (1) How can we effectively transfer advanced abilities without their supervised data? (2) How can we prevent the original knowledge from catastrophic forgetting during transformation?

Plain English Explanation

Large language models (LLMs) are powerful AI systems that can generate human-like text, answer questions, and perform a variety of language-related tasks. However, the majority of these models have been developed using English data, which limits their ability to work with other languages.

One solution to this problem is to take an existing English-based LLM and transform it to work with a different language, such as Thai. This is a more efficient approach than building a brand-new model from scratch for each language.

Previous methods have taken a base LLM and used a technique called knowledge distillation to teach it new abilities, using data generated by even more powerful LLMs like GPT-4. However, these methods have focused on transforming "base" LLMs, which are not as advanced as the "chat" LLMs that have been further optimized for tasks like multi-turn conversations and aligning with human preferences.

Transforming a chat LLM presents two key challenges: [1] How can we effectively transfer the advanced abilities of the chat LLM without access to the supervised data used to train those abilities? [2] How can we prevent the original knowledge of the LLM from being forgotten during the transformation process?

Technical Explanation

The researchers introduce a framework called TransLLM to address these challenges. For the first issue, TransLLM breaks down the transfer problem into common sub-tasks, using translation as a bridge between the English and non-English models. The performance of these sub-tasks is further enhanced using publicly available data.

For the second issue, TransLLM uses two synergistic components: low-rank adaptation and recovery knowledge distillation (KD). Low-rank adaptation allows the model to be trained while maintaining the original LLM parameters. Recovery KD then utilizes data generated by the chat LLM itself to recover the original knowledge from these frozen parameters.

In experiments, the researchers transformed the LLaMA-2-chat-7B model to the Thai language. Their method, using only single-turn data, outperformed strong baselines and even ChatGPT on a multi-turn benchmark called MT-bench. Furthermore, their method, without any safety data, rejected more harmful queries on a safety benchmark called AdvBench than both ChatGPT and GPT-4.

Critical Analysis

The researchers acknowledge that their method relies on the availability of some single-turn data in the target language, which may not always be the case. Additionally, they note that their approach focuses on transforming a chat LLM, but it may not be as effective for transforming other types of LLMs, such as those optimized for specific tasks like question answering or language generation.

While the researchers demonstrate impressive results on the MT-bench and AdvBench benchmarks, it would be valuable to see the model's performance on a wider range of tasks and in real-world applications to better understand its capabilities and limitations.

Conclusion

This research presents a promising framework, called TransLLM, for transforming English-centric chat LLMs to work with non-English languages. By addressing the challenges of transferring advanced abilities and preventing catastrophic forgetting, the researchers have developed a method that outperforms strong baselines and even some of the most advanced language models currently available.

This work could have significant implications for making powerful language models more accessible to users around the world, regardless of their primary language. As the field of natural language processing continues to evolve, techniques like those introduced in this paper will be crucial for building truly multilingual and inclusive AI systems.