Large Language Models as Optimizers
2309.03409
95
0
💬
Abstract
Optimization is ubiquitous. While derivative-based algorithms have been powerful tools for various problems, the absence of gradient imposes challenges on many real-world applications. In this work, we propose Optimization by PROmpting (OPRO), a simple and effective approach to leverage large language models (LLMs) as optimizers, where the optimization task is described in natural language. In each optimization step, the LLM generates new solutions from the prompt that contains previously generated solutions with their values, then the new solutions are evaluated and added to the prompt for the next optimization step. We first showcase OPRO on linear regression and traveling salesman problems, then move on to our main application in prompt optimization, where the goal is to find instructions that maximize the task accuracy. With a variety of LLMs, we demonstrate that the best prompts optimized by OPRO outperform human-designed prompts by up to 8% on GSM8K, and by up to 50% on Big-Bench Hard tasks. Code at https://github.com/google-deepmind/opro.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- Optimization is a common task, but traditional gradient-based methods have limitations when gradients are not available.
- The paper proposes a new approach called "Optimization by PROmpting" (OPRO) that uses large language models (LLMs) as optimizers, where the optimization task is described in natural language.
- OPRO generates new solutions iteratively, evaluates them, and adds them to the prompt for the next step.
- The authors demonstrate OPRO's effectiveness on linear regression, traveling salesman, and prompt optimization problems, showing significant improvements over human-designed prompts.
Plain English Explanation
Optimization is a fundamental problem that arises in many real-world situations, such as finding the best route for a delivery truck or selecting the most effective prompts for a language model. Traditional optimization methods that rely on calculating gradients can work well, but they struggle when gradients are not available, which is common in many practical applications.
To address this, the researchers propose a new approach called "Optimization by PROmpting" (OPRO). The key idea is to use powerful large language models (LLMs) as the optimizers, where the optimization task is described in natural language. In each optimization step, the LLM generates new candidate solutions based on the prompt, which contains information about the previously generated solutions and their values. These new solutions are then evaluated, and the best ones are added to the prompt for the next optimization step.
The researchers demonstrate OPRO's effectiveness on several problems, including linear regression and the traveling salesman problem. They also show that OPRO can be used to optimize the prompts themselves, finding instructions that significantly outperform human-designed prompts on challenging language model tasks.
Technical Explanation
The key innovation in this work is the use of large language models (LLMs) as optimization engines, where the optimization task is described in natural language. This approach, called "Optimization by PROmpting" (OPRO), iteratively generates new candidate solutions based on the current prompt, evaluates them, and adds the best ones to the prompt for the next iteration.
In each optimization step, the LLM takes the current prompt, which includes information about the previously generated solutions and their values, and generates new candidate solutions. These new solutions are then evaluated, and the best ones are added to the prompt for the next step. This process continues until a stopping criterion is met, such as a maximum number of iterations or a target objective value.
The researchers demonstrate OPRO's effectiveness on several problems, including linear regression, the traveling salesman problem, and prompt optimization for language models. In the prompt optimization task, they show that the best prompts found by OPRO can outperform human-designed prompts by up to 8% on the GSM8K benchmark and up to 50% on the more challenging Big-Bench Hard tasks.
Critical Analysis
One potential limitation of the OPRO approach is that it relies on the ability of the LLM to generate high-quality candidate solutions based on the current prompt. If the LLM struggles to understand the optimization problem or to generate promising new solutions, the optimization process may not converge to a good result. Additionally, the authors note that OPRO can be computationally expensive, as each optimization step requires running the LLM to generate new solutions.
Another concern is the need for careful prompt engineering to ensure that the LLM understands the optimization problem correctly. If the prompt is not well-designed, the LLM may generate irrelevant or suboptimal solutions, leading to poor optimization performance.
Despite these potential limitations, the OPRO approach represents an interesting and novel application of large language models, demonstrating their potential as powerful optimization tools. The authors have provided an open-source implementation of OPRO, which should encourage further research and experimentation in this area.
Conclusion
The paper presents a novel approach called "Optimization by PROmpting" (OPRO) that leverages the power of large language models (LLMs) to optimize complex problems where traditional gradient-based methods may struggle. By describing the optimization task in natural language and iteratively generating and evaluating candidate solutions, OPRO has been shown to outperform human-designed prompts on a range of tasks, including prompt optimization for language models.
While OPRO has some potential limitations, such as the need for careful prompt engineering and computational expense, the authors' work highlights the exciting potential of using LLMs as optimization tools. As language models continue to advance, the OPRO approach may become an increasingly valuable tool for tackling a wide range of optimization challenges in the real world.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Revisiting OPRO: The Limitations of Small-Scale LLMs as Optimizers
Tuo Zhang, Jinyue Yuan, Salman Avestimehr
0
0
Numerous recent works aim to enhance the efficacy of Large Language Models (LLMs) through strategic prompting. In particular, the Optimization by PROmpting (OPRO) approach provides state-of-the-art performance by leveraging LLMs as optimizers where the optimization task is to find instructions that maximize the task accuracy. In this paper, we revisit OPRO for automated prompting with relatively small-scale LLMs, such as LLaMa-2 family and Mistral 7B. Our investigation reveals that OPRO shows limited effectiveness in small-scale LLMs, with limited inference capabilities constraining optimization ability. We suggest future automatic prompting engineering to consider both model capabilities and computational costs. Additionally, for small-scale LLMs, we recommend direct instructions that clearly outline objectives and methodologies as robust prompt baselines, ensuring efficient and effective prompt engineering in ongoing research.
5/17/2024
💬
Language Models as Black-Box Optimizers for Vision-Language Models
Shihong Liu, Zhiqiu Lin, Samuel Yu, Ryan Lee, Tiffany Ling, Deepak Pathak, Deva Ramanan
0
0
Vision-language models (VLMs) pre-trained on web-scale datasets have demonstrated remarkable capabilities on downstream tasks when fine-tuned with minimal data. However, many VLMs rely on proprietary data and are not open-source, which restricts the use of white-box approaches for fine-tuning. As such, we aim to develop a black-box approach to optimize VLMs through natural language prompts, thereby avoiding the need to access model parameters, feature embeddings, or even output logits. We propose employing chat-based LLMs to search for the best text prompt for VLMs. Specifically, we adopt an automatic hill-climbing procedure that converges to an effective prompt by evaluating the performance of current prompts and asking LLMs to refine them based on textual feedback, all within a conversational process without human-in-the-loop. In a challenging 1-shot image classification setup, our simple approach surpasses the white-box continuous prompting method (CoOp) by an average of 1.5% across 11 datasets including ImageNet. Our approach also outperforms both human-engineered and LLM-generated prompts. We highlight the advantage of conversational feedback that incorporates both positive and negative prompts, suggesting that LLMs can utilize the implicit gradient direction in textual feedback for a more efficient search. In addition, we find that the text prompts generated through our strategy are not only more interpretable but also transfer well across different VLM architectures in a black-box manner. Lastly, we apply our framework to optimize the state-of-the-art black-box VLM (DALL-E 3) for text-to-image generation, prompt inversion, and personalization.
5/15/2024
When Large Language Model Meets Optimization
Sen Huang, Kaixiang Yang, Sheng Qi, Rui Wang
0
0
Optimization algorithms and large language models (LLMs) enhance decision-making in dynamic environments by integrating artificial intelligence with traditional techniques. LLMs, with extensive domain knowledge, facilitate intelligent modeling and strategic decision-making in optimization, while optimization algorithms refine LLM architectures and output quality. This synergy offers novel approaches for advancing general AI, addressing both the computational challenges of complex problems and the application of LLMs in practical scenarios. This review outlines the progress and potential of combining LLMs with optimization algorithms, providing insights for future research directions.
5/17/2024
Automatic Prompt Selection for Large Language Models
Viet-Tung Do, Van-Khanh Hoang, Duy-Hung Nguyen, Shahab Sabahi, Jeff Yang, Hajime Hotta, Minh-Tien Nguyen, Hung Le
0
0
Large Language Models (LLMs) can perform various natural language processing tasks with suitable instruction prompts. However, designing effective prompts manually is challenging and time-consuming. Existing methods for automatic prompt optimization either lack flexibility or efficiency. In this paper, we propose an effective approach to automatically select the optimal prompt for a given input from a finite set of synthetic candidate prompts. Our approach consists of three steps: (1) clustering the training data and generating candidate prompts for each cluster using an LLM-based prompt generator; (2) synthesizing a dataset of input-prompt-output tuples for training a prompt evaluator to rank the prompts based on their relevance to the input; (3) using the prompt evaluator to select the best prompt for a new input at test time. Our approach balances prompt generality-specificity and eliminates the need for resource-intensive training and inference. It demonstrates competitive performance on zero-shot question-answering datasets: GSM8K, MultiArith, and AQuA.
4/4/2024