Language Model Decoding as Direct Metrics Optimization

Overview

• Current language models struggle to generate texts that align with human texts across different aspects.
• Sampling-based methods produce less-repetitive but disjunctive texts, while search-based methods maintain topic coherence at the cost of increased repetition.
• The paper frames decoding from a language model as an optimization problem to strictly match the expected performance with human texts across multiple metrics simultaneously.

Plain English Explanation

Despite the impressive capabilities of modern language models, the texts they generate often fall short of matching the attributes of human-written texts. Sampling-based methods can produce more varied output, but the resulting text may lack coherence and flow. Conversely, search-based methods can maintain topic consistency, but at the expense of increased repetition.

This paper proposes a new approach that frames the text generation process as an optimization problem. The goal is to generate text that closely matches human-written text across a range of desired qualities, such as fluency, coherence, and creativity. This is achieved by defining a sequence-level energy function that incorporates multiple metrics simultaneously, and then using this function to scale the input language model distribution.

The key insight is that this induced distribution is guaranteed to improve the perplexity on human texts, suggesting it is a better approximation of the underlying distribution of human-written language. To make the sampling process tractable, the researchers adopt the Sampling-Importance-Resampling technique.

Technical Explanation

The paper proposes a novel decoding method that frames the text generation process as an optimization problem. The goal is to generate text that strictly matches the expected performance with human texts, as measured by multiple metrics of desired aspects simultaneously.

The authors define a sequence-level energy function based on these metrics and use it to scale the input language model distribution. This induced distribution enjoys an analytical solution and is proven to improve the perplexity on human texts, implying a better approximation of the underlying distribution of human-written language.

To facilitate tractable sampling from this globally normalized distribution, the researchers adopt the Sampling-Importance-Resampling (SIR) technique. Experiments on various domains and model scales demonstrate the superiority of their method in metrics alignment with human texts and human evaluation over strong baselines, such as SpeechAlign and SED.

Critical Analysis

The paper presents a compelling approach to text generation that aims to address the shortcomings of existing methods. By formulating the problem as an optimization task and deriving an analytically-solved distribution, the authors demonstrate a principled way to generate text that aligns with human-written text across multiple metrics.

However, the paper does not discuss the computational complexity of the proposed method, which could be a practical concern, especially for large-scale language models. Additionally, the paper does not explore the robustness of the method to different types of human texts or its performance on tasks beyond text generation, such as summarization or dialogue.

Further research could investigate the generalization of the approach to other language-related tasks, as well as its scalability and efficiency in real-world applications. It would also be valuable to understand the specific tradeoffs between the proposed method and existing techniques, perhaps through sample-efficient human evaluation or other comparative analyses.

Conclusion

This paper presents a novel optimization-based approach to text generation that aims to closely match the qualities of human-written text. By defining a sequence-level energy function based on multiple metrics and scaling the input language model distribution accordingly, the authors demonstrate a principled way to generate text that is more aligned with human preferences.

The key contribution of this work is the guaranteed improvement in perplexity on human texts, suggesting the induced distribution is a better approximation of the underlying human language distribution. While the computational complexity and generalization of the method require further investigation, this research represents an important step towards generating text that is more coherent, fluent, and engaging from the perspective of human readers.