Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

Overview

• This paper introduces a technique called DARE (Decoupled Alignment and Robust Embedding) that allows language models (LMs) to acquire new capabilities by assimilating parameters from similar models without retraining or specialized hardware.
• The authors show that the differences (delta parameters) between fine-tuned and pre-trained LMs are typically small and redundant, and DARE can effectively eliminate 90% or even 99% of these parameters without affecting the model's abilities.
• DARE can be used as a versatile plug-in to merge multiple task-specific LMs into a single model with diverse capabilities, which is especially pronounced in large-scale LMs.
• The merged LM can sometimes surpass the performance of any of the source models, providing a new discovery.

Plain English Explanation

Acquiring New Capabilities Without Retraining The paper explains how language models can learn new skills by incorporating parameters from similar models, without having to go through the entire retraining process. This is done using a technique called DARE, which can efficiently remove most of the differences (delta parameters) between the fine-tuned and pre-trained versions of a model, without affecting its performance. link to DARE paper

Merging Multiple Language Models The researchers also show how DARE can be used to combine several task-specific language models into a single model that has a diverse set of capabilities. This is particularly powerful for large-scale language models, where the merged model can sometimes outperform any of the individual source models. link to paper on abilities of large language models

Potential for Efficient Model Scaling This discovery suggests that there may be an efficient way to scale up language models by merging specialized models, rather than having to retrain a single large model from scratch. This could lead to significant improvements in the capabilities of AI systems without the need for massive computational resources. link to paper on teaching languages to large language models

Technical Explanation

The paper introduces a technique called DARE (Decoupled Alignment and Robust Embedding) that allows language models (LMs) to acquire new capabilities by assimilating parameters from similar, or "homologous," models without retraining or specialized hardware like GPUs.

The authors first show that the differences (delta parameters) between fine-tuned and pre-trained LMs are typically small, within a range of 0.002, and exhibit extreme redundancy. They then propose DARE, which Drops delta parameters with a ratio p and REscales the remaining ones by 1 / (1 - p) to approximate the original embeddings. This effectively eliminates 90% or even 99% of the delta parameters without affecting the model's abilities.

The researchers then use DARE as a versatile plug-in to sparsify the delta parameters of multiple task-specific SFT (Supervised Fine-Tuning) homologous models, and merge them into a single model by parameter fusing. link to paper on robust plug-and-play adaptation

The experiments show that this phenomenon is more pronounced in large-scale LMs, where the merged model can sometimes surpass the performance of any of the source models, providing a new discovery. The authors also utilize DARE to create a merged LM that ranks first among models with 7 billion parameters on the Open LLM Leaderboard. link to paper on expansion of spoken language understanding

Critical Analysis

The paper presents an intriguing approach for efficiently scaling up language models by merging specialized models, rather than retraining a single large model from scratch. This could lead to significant improvements in the capabilities of AI systems without the need for massive computational resources.

However, the authors do not address potential limitations or caveats of their approach. For example, it's unclear how well the merged model would perform on a wide range of tasks compared to a model trained from scratch on a diverse dataset. Additionally, the paper does not explore the effects of this approach on model robustness, fairness, or safety. link to paper on debiasing algorithm through model adaptation

Further research is needed to understand the broader implications and potential issues with this technique, as well as its applicability to other types of AI models beyond language models. It will be important for the research community to critically examine the findings and consider the long-term consequences of such model merging approaches.

Conclusion

This paper introduces a novel technique called DARE that enables language models to acquire new capabilities by assimilating parameters from similar models, without the need for retraining or specialized hardware. The authors demonstrate that DARE can effectively merge multiple task-specific language models into a single model with diverse capabilities, particularly for large-scale language models.

This discovery suggests that there may be an efficient way to scale up language models by leveraging existing specialized models, rather than having to retrain a single large model from scratch. If further research can address the potential limitations and implications of this approach, it could lead to significant advancements in the capabilities and accessibility of AI systems.