Princeton-nlp
Models by this creator
🖼️
Sheared-LLaMA-1.3B
85
Sheared-LLaMA-1.3B is a model pruned and further pre-trained from the meta-llama/Llama-2-7b-hf model. The maintainer, princeton-nlp, dynamically loaded data from different domains in the RedPajama dataset to prune and continue pre-training the model. They used 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model is smaller-scale compared to the original LLaMA models, but shares the same vocabulary. It was derived by the maintainer with a budget of 50B tokens, leveraging existing strong large language models. Model inputs and outputs Inputs Natural language text Outputs Continued generation of natural language text Capabilities The Sheared-LLaMA-1.3B model outperforms existing large language models on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling, and knowledge-intensive tasks. What can I use it for? The Sheared-LLaMA-1.3B model can be used for a variety of natural language processing tasks, such as text generation, question answering, and language modeling. Its strong performance on downstream tasks makes it a viable option for projects that require robust language understanding and generation capabilities. Things to try Given the model's smaller size compared to the original LLaMA models, it could be an interesting option to explore for deployments with more constrained computational resources. The maintainer's approach of pruning and continued pre-training on diverse datasets also suggests that the model may have unique strengths, such as improved efficiency or specialized knowledge, that could be worth investigating further.
Updated 5/28/2024
🛠️
gemma-2-9b-it-SimPO
67
The gemma-2-9b-it-SimPO model is a large language model developed by princeton-nlp using the SimPO (Simple Preference Optimization) algorithm. It was fine-tuned on the princeton-nlp/gemma2-ultrafeedback-armorm dataset, building upon the google/gemma-2-9b-it base model. The SimPO algorithm aligns the reward function with the generation likelihood, enhancing the model's performance on preference optimization tasks. This model can be compared to the Gemma-2-9B-It-SPPO-Iter3 model, which was developed using Self-Play Preference Optimization on a similar dataset. Model inputs and outputs Inputs Text prompts or queries that the model can generate responses to. Outputs Generated text responses to the input prompts or queries. Capabilities The gemma-2-9b-it-SimPO model is capable of generating coherent and contextually appropriate text responses to a variety of prompts, including questions, descriptions, and instructions. It demonstrates strong performance in tasks such as summarization, question answering, and open-ended text generation. What can I use it for? The gemma-2-9b-it-SimPO model can be useful for a range of applications that involve natural language generation, such as: Developing conversational AI assistants or chatbots Generating creative content like stories, poems, or scripts Summarizing long-form text Answering questions or providing information on a wide range of topics By leveraging the model's capabilities, you can create innovative products and services that empower users with advanced language understanding and generation abilities. Things to try One interesting aspect of the gemma-2-9b-it-SimPO model is its ability to generate text that closely aligns with user preferences. This could be particularly useful for applications where personalization and user satisfaction are important, such as content recommendation systems or personalized writing assistants. Additionally, you could explore using the model for tasks that require a more nuanced understanding of language, such as dialogue generation, creative writing, or task-oriented conversational interactions. The model's strong performance on preference optimization may also make it a useful tool for researchers studying language model alignment and reward modeling.
Updated 9/19/2024
↗️
Llama-3-Instruct-8B-SimPO
55
Llama-3-Instruct-8B-SimPO is an AI model developed by princeton-nlp. It is a text-to-text model, which means it can generate text from text inputs. The model is based on the LLaMA architecture and has 8 billion parameters. It is designed for instructional tasks, similar to llama-3-70b-instruct-awq, Llama-3-8B-Instruct-Gradient-1048k-GGUF, and LLaMA-7B. Model inputs and outputs The Llama-3-Instruct-8B-SimPO model takes text as input and generates text as output. It can handle a variety of text-related tasks, such as language generation, question answering, and text summarization. Inputs Text prompts for the model to generate output Outputs Text generated by the model based on the input prompt Capabilities The Llama-3-Instruct-8B-SimPO model can be used for a range of text-related tasks, such as language generation, question answering, and text summarization. It can generate coherent and relevant text based on the input prompt, and can adapt to different styles and tones. What can I use it for? You can use Llama-3-Instruct-8B-SimPO for a variety of applications, such as chatbots, content generation, and language learning. For example, you could use it to generate product descriptions, write blog posts, or create personalized learning materials. The model's versatility makes it a useful tool for businesses and individuals alike. Things to try One interesting thing to try with Llama-3-Instruct-8B-SimPO is to provide it with prompts that challenge its capabilities, such as complex questions or open-ended writing tasks. This can help you understand the model's strengths and limitations, and identify potential areas for improvement or further development.
Updated 7/18/2024
🔍
Sheared-LLaMA-2.7B
54
Sheared-LLaMA-2.7B is a pruned and further pre-trained model derived from the meta-llama/Llama-2-7b-hf model. The model was developed by the princeton-nlp team and is available on the Hugging Face Hub. Like the original LLaMA model, Sheared-LLaMA-2.7B is a large language model based on the transformer architecture. However, this model was pruned and further trained on the RedPajama dataset using a budget of 50 billion tokens. Model inputs and outputs Inputs Text prompts Outputs Continuation of the input text, generating coherent and relevant text Capabilities The Sheared-LLaMA-2.7B model has demonstrated strong performance across a variety of downstream tasks, including reasoning, reading comprehension, language modeling, and knowledge-intensive tasks. The model outperforms existing large language models like OPT-2.7B and Pythia-2.8B on average performance metrics. What can I use it for? The Sheared-LLaMA-2.7B model can be used for a wide range of natural language processing tasks, such as text generation, question answering, summarization, and content creation. Developers and researchers can fine-tune the model for specific applications or use it as a strong baseline for further research and development. Things to try One interesting aspect of the Sheared-LLaMA-2.7B model is that it was trained with a budget of only 50 billion tokens, which is significantly less than the 1 trillion tokens used to train the original LLaMA models. This suggests that the model's performance can be achieved with a more efficient and cost-effective training process, making it an attractive option for those with limited computational resources.
Updated 5/28/2024