Ai21labs
Models by this creator
🎯
Jamba-v0.1
1.1K
Jamba-v0.1 is a state-of-the-art, hybrid SSM-Transformer large language model (LLM) developed by AI21 Labs. It delivers throughput gains over traditional Transformer-based models, while outperforming or matching the leading models of its size class on most common benchmarks. Jamba is the first production-scale Mamba implementation, which opens up interesting research and application opportunities. Similar models like mamba-2.8b-instruct-openhermes, mamba-2.8b-hf, and mamba-2.8b-slimpj also utilize the Mamba architecture, with varying parameter sizes and training datasets. Model Inputs and Outputs Jamba-v0.1 is a pretrained, mixture-of-experts (MoE) generative text model. It supports a 256K context length and can fit up to 140K tokens on a single 80GB GPU. Inputs Text prompts of up to 256K tokens Outputs Continuation of the input text, generating new tokens based on the provided context Capabilities Jamba-v0.1 is a powerful language model that can be used for a variety of text-generation tasks. It has demonstrated strong performance on common benchmarks, outperforming or matching leading models of similar size. The hybrid SSM-Transformer architecture allows for improved throughput compared to traditional Transformer-based models. What Can I Use It For? The capabilities of Jamba-v0.1 make it a versatile model that can be used for many text-to-text tasks, such as: Content Generation**: Write articles, stories, scripts, and other types of long-form text with high quality and coherence. Dialogue Systems**: Build chatbots and virtual assistants that can engage in natural, contextual conversations. Question Answering**: Answer questions on a wide range of topics by leveraging the model's broad knowledge base. Summarization**: Condense long passages of text into concise, informative summaries. Given its strong performance, Jamba-v0.1 can be a valuable tool for businesses, researchers, and developers looking to push the boundaries of what's possible with large language models. Things to Try One interesting aspect of Jamba-v0.1 is its hybrid SSM-Transformer architecture, which combines the strengths of structured state space models and traditional Transformers. Exploring how this architectural choice affects the model's performance, especially on tasks that require long-range dependencies or efficient processing, could yield valuable insights. Additionally, the Mamba implementation used in Jamba-v0.1 opens up new research opportunities. Investigating how this subquadratic model compares to other state-of-the-art language models, both in terms of raw performance and computational efficiency, could help advance the field of large language models.
Updated 5/28/2024
📈
AI21-Jamba-1.5-Mini
218
The AI21-Jamba-1.5-Mini model is a state-of-the-art, hybrid SSM-Transformer instruction following foundation model developed by AI21 Labs. It is part of the Jamba model family, which includes the larger AI21-Jamba-1.5-Large model. The Jamba models are designed to deliver fast inference with high-quality long-context generation, outperforming many leading models of comparable size. Model inputs and outputs Inputs Text**: The model accepts text input for tasks like question answering, summarization, and open-ended generation. Outputs Text**: The model generates relevant, coherent text in response to the input, such as answers to questions or continuations of prompts. Capabilities The AI21-Jamba-1.5-Mini demonstrates strong performance on a variety of benchmarks, including long-form tasks that require reasoning over extensive context. It also supports capabilities like function calling, structured output, and grounded generation, making it suitable for business and enterprise use cases. What can I use it for? The AI21-Jamba-1.5-Mini can be used for a wide range of natural language processing tasks, from content creation to question answering and code generation. For example, you could use it to draft marketing copy, summarize research papers, or build virtual assistants. Its efficient design and high-quality outputs make it particularly well-suited for business and enterprise applications. Things to try One interesting aspect of the Jamba models is their hybrid architecture, which combines state-space modules (Mamba) with transformer components. This allows them to maintain long-term context more effectively than traditional transformer-only models. You could experiment with prompts that require reasoning over long passages of text to see how the AI21-Jamba-1.5-Mini performs compared to other language models.
Updated 9/19/2024
📉
AI21-Jamba-1.5-Large
179
The AI21-Jamba-1.5-Large is a state-of-the-art, hybrid SSM-Transformer instruction following foundation model developed by AI21. It is part of the Jamba model family, which includes the smaller Jamba 1.5 Mini (12B/52B) and the larger Jamba 1.5 Large (94B/398B). The Jamba models are the most powerful and efficient long-context models on the market, delivering up to 2.5X faster inference than leading models of comparable sizes. They mark the first time a non-Transformer model has been successfully scaled to the quality and strength of the market's leading models. Model inputs and outputs The AI21-Jamba-1.5-Large is a text-to-text model that can handle long-form input and output. It supports a context length of up to 256K tokens, making it well-suited for tasks that require processing and generating lengthy text. Inputs Freeform text input up to 256K tokens Optional tools and documents that can be included in the input to guide the model's generation Outputs Freeform text output up to 100K tokens JSON-formatted responses for structured output Invocations of tools that are defined in the input Capabilities The Jamba 1.5 models demonstrate superior long context handling, speed, and quality. They support advanced capabilities such as function calling, structured output (JSON), and grounded generation. The models are also optimized for business use cases. What can I use it for? The AI21-Jamba-1.5-Large can be used for a variety of natural language tasks, including but not limited to: General text generation and summarization Question answering and dialogue systems Code generation and programming assistance Structured data generation (e.g., JSON, tables) Grounded generation based on provided documents The model is released under the Jamba Open Model License, which allows for full research use and commercial use under the license terms. If you need to license the model for your specific needs, you can talk to the AI21 team. Things to try One interesting capability of the Jamba 1.5 models is their ability to handle tool invocations and execute tasks in a structured way. You can include tool definitions in the input, and the model will attempt to call those tools and incorporate the results into its output. This can be useful for building AI assistants that can interact with external services or APIs. Another key feature is the models' support for grounded generation, where the model can use provided documents or snippets to generate relevant and factual responses. This can be valuable for use cases that require generating content based on a specific knowledge base or set of resources.
Updated 9/19/2024