Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models

Overview

• Large language models (LLMs) can exhibit positional bias, meaning they use context differently depending on the order of elements, which complicates listwise ranking tasks.
• The researchers propose a method called "permutation self-consistency" to address this issue by marginalizing out order biases in the prompt.
• The approach involves shuffling the list in the prompt, passing it through the LLM, and then aggregating the resulting rankings to produce an order-independent ranking.
• Experiments on list-ranking datasets show significant improvements over conventional inference, surpassing previous state-of-the-art methods.

Plain English Explanation

Large language models (LLMs) like GPT-3.5 and LLaMA v2 are powerful AI systems that can generate human-like text. However, these models can sometimes be biased by the order of the information they're given. This is a particular problem when they're asked to rank a list of items, as their rankings may be influenced by the order in which the items are presented.

To address this issue, the researchers developed a technique called "permutation self-consistency." The key idea is to repeatedly shuffle the order of the items in the prompt, pass each version through the LLM, and then combine the resulting rankings into a single, order-independent ranking. This helps to neutralize the positional biases that the LLM might otherwise exhibit.

Consolidating Ranking Relevance Predictions in Large Language Models and PICO: Peer-Review LLMs via Consistency Optimization are two related techniques that also aim to improve the consistency and reliability of LLM outputs.

The researchers tested their permutation self-consistency approach on several list-ranking datasets, including tasks like sorting and passage reranking. They found that it could improve the rankings produced by GPT-3.5 and LLaMA v2 by up to 7-18% and 8-16%, respectively, outperforming previous state-of-the-art methods.

Technical Explanation

The researchers start by observing that large language models (LLMs) can exhibit positional bias, meaning they use context differently depending on the order of elements in the input. This is particularly problematic for listwise ranking tasks, where the model's rankings may be influenced by the order in which items are presented.

To address this issue, the researchers propose a method called "permutation self-consistency." The key idea is to marginalize out the order biases in the prompt by repeatedly shuffling the list and passing it through the LLM while holding the instructions constant. This produces a sample of rankings, which are then aggregated to compute a central ranking that is less affected by positional biases.

Mathematically, the approach can be summarized as follows:

1. Given an input prompt containing a list, repeatedly shuffle the order of the list.
2. Pass each shuffled prompt through the black-box LLM to obtain a set of rankings.
3. Compute the central ranking that is closest in distance to all the individual rankings, effectively marginalizing out the order biases.

The researchers provide theoretical guarantees, showing that their method converges to the true ranking in the presence of random perturbations.

Empirically, the researchers evaluate their approach on five list-ranking datasets, including sorting and passage reranking tasks. They find that their permutation self-consistency method can improve ranking scores by up to 7-18% for GPT-3.5 and 8-16% for LLaMA v2 (70B), outperforming previous state-of-the-art methods.

Generating Diverse Criteria to Fly to Improve Point-Level Consistency and Beyond Self-Consistency: Ensemble Reasoning Boosts Consistency are two other techniques that aim to improve the consistency and reliability of LLM outputs.

Critical Analysis

The researchers acknowledge several caveats and limitations of their work. First, the permutation self-consistency approach relies on the LLM's ability to produce consistent rankings across multiple shuffled prompts, which may not always be the case. Additionally, the method may be computationally expensive, as it requires running the LLM multiple times for each input.

Furthermore, the researchers only evaluate their approach on a limited set of list-ranking tasks. It would be valuable to see how the method performs on a wider range of applications, particularly in domains where positional bias is known to be a significant issue.

Another potential concern is the reliance on the LLM's own outputs as the ground truth for ranking. If the LLM has inherent biases or inconsistencies, the permutation self-consistency method may not be able to fully correct for them.

Towards Logically Consistent Language Models via Probabilistic is a related paper that explores approaches for improving the logical consistency of LLM outputs, which could be a valuable complement to the work presented here.

Overall, the researchers have presented a novel and promising technique for addressing positional bias in LLMs, but further research and validation will be needed to fully assess its strengths, limitations, and broader applicability.

Conclusion

The researchers have proposed a method called "permutation self-consistency" to address the positional bias exhibited by large language models when performing listwise ranking tasks. By repeatedly shuffling the order of items in the prompt and aggregating the resulting rankings, the approach helps to neutralize the order-dependent biases of the LLM.

Empirical results on several list-ranking datasets show significant improvements over conventional inference, with gains of up to 7-18% for GPT-3.5 and 8-16% for LLaMA v2 (70B), surpassing previous state-of-the-art methods. This work represents an important step towards developing more robust and reliable language models, with potential applications in a wide range of domains that rely on ranking and ordering tasks.