Phi-3-mini-4k-instruct-gguf

Model overview

The Phi-3-mini-4k-instruct is a 3.8 billion-parameter, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Phi-3 family, with the Mini version available in two variants - 4K and 128K - which is the context length (in tokens) it can support. The Phi-3-mini-128k-instruct is a similar model with a 128K context length. Both models have undergone a post-training process that incorporates supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures.

Model inputs and outputs

The Phi-3-mini-4k-instruct model takes text as input and generates text as output. The model is best suited for prompts using the chat format, where the user provides a prompt starting with <|user|> and the model generates the response after <|assistant|>.

Inputs

• Text prompts, typically in a chat format like:

  <|user|>
  How to explain the Internet to a medieval knight?
  <|end|>
  

Outputs

• Generated text responses, for example:

  <|assistant|>
  To explain the Internet to a medieval knight, you could say that it is a magical network of interconnected "talking scrolls" that allow people to share information and communicate across vast distances, almost as if by magic. Just as a messenger on horseback can carry news and messages between distant keeps, the Internet allows information to travel quickly between far-flung locations. However, instead of a single messenger, the Internet has millions of these "talking scrolls" all connected together, allowing information to flow freely between them.
  
  You could liken the different websites on the Internet to the various fiefs, manors, and castles of the realm, each with their own unique content and purpose. And just as a knight might travel between these different places to gather news and resources, a user on the Internet can navigate between websites to find the information they seek.
  
  Of course, the technology behind the Internet is far more advanced than anything a medieval knight would be familiar with. But hopefully this analogy helps convey the core concept of how this new magical network functions and allows for the rapid sharing of information across vast distances.
  <|end|>
  

Capabilities

The Phi-3-mini-4k-instruct model showcases robust and state-of-the-art performance on a variety of benchmarks testing common sense, language understanding, math, code, long context, and logical reasoning, particularly among models with less than 13 billion parameters. It demonstrates strong capabilities in areas like memory/compute constrained environments, latency-bound scenarios, and applications requiring strong reasoning skills.

What can I use it for?

The Phi-3-mini-4k-instruct model is intended for commercial and research use in English. It can be used as a building block for developing generative AI-powered features and applications, especially those with requirements around memory/compute constraints, low latency, or strong reasoning abilities. Some potential use cases include:

• Language model-powered chatbots and virtual assistants
• Content generation for education, journalism, or creative writing
• Code generation and programming assistance tools
• Reasoning-intensive applications like question-answering systems or intelligent tutoring systems

Things to try

One interesting aspect of the Phi-3-mini-4k-instruct model is its ability to engage in multi-turn, chat-like conversations using the provided chat format. This allows you to explore the model's conversational capabilities and see how it responds to follow-up questions or requests. Additionally, you can experiment with prompts that require strong reasoning skills, such as math problems or logic puzzles, to assess the model's capabilities in these areas.