bge-reranker-base
Maintainer: ninehills
8
| Property | Value |
|---|---|
| Run this model | Run on Replicate |
| API spec | View on Replicate |
| Github link | View on Github |
| Paper link | No paper link provided |
Create account to get full access
Model overview
The bge-reranker-base model from BAAI (Beijing Academy of Artificial Intelligence) is a cross-encoder model that can be used to re-rank the top-k documents returned by an embedding model. It is more accurate than embedding models like BGE-M3 or LLM Embedder, but less efficient. This model can be fine-tuned on your own data to improve performance on specific tasks.
Model inputs and outputs
Inputs
- pairs_json: A JSON string containing input pairs, e.g.
[["a", "b"], ["c", "d"]]
Outputs
- scores: An array of scores for the input pairs
- use_fp16: A boolean indicating whether the model used FP16 inference
- model_name: The name of the model used
Capabilities
The bge-reranker-base model can effectively re-rank the top-k documents returned by an embedding model, making the final ranking more accurate. This can be particularly useful when you need high-precision retrieval results, such as for question answering or knowledge-intensive tasks.
What can I use it for?
You can use the bge-reranker-base model to re-rank the results of an embedding model like BGE-M3 or LLM Embedder. This can help improve the accuracy of your retrieval system, especially for critical applications where precision is important.
Things to try
You can try fine-tuning the bge-reranker-base model on your own data to further improve its performance on your specific use case. The examples provided can be a good starting point for this.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Models
🛠️
bge-reranker-v2-m3
57
bge-reranker-v2-m3 is the newest balance-striking reranker model from BAAI. It outputs rank scores for query-doc pairs and has FP16 inference enabled. This model can be compared to similar query-document ranking models like qwen1.5-110b and reliberate-v3. Model inputs and outputs The bge-reranker-v2-m3 model takes a JSON string as input, which can be a list containing one query and one passage pair, or a list of such pairs. The output is an array. Inputs Input List**: A JSON string containing one or more query-passage pairs. Outputs Output**: An array containing the output of the model. Capabilities The bge-reranker-v2-m3 model can be used to rank query-document pairs, which is useful for a variety of applications such as search, question answering, and information retrieval. What can I use it for? The bge-reranker-v2-m3 model can be used for a variety of applications that involve ranking text-based content, such as web search, recommendation systems, and content moderation. For example, you could use this model to improve the relevance of search results on your website or to automatically filter out low-quality or inappropriate content. Things to try One interesting thing to try with the bge-reranker-v2-m3 model is to experiment with different types of query-document pairs and observe how the model's ranking scores change. You could also try combining this model with other natural language processing models, such as real-esrgan or absolutereality-v1.8.1, to create more sophisticated content ranking and recommendation systems.
Updated Invalid Date
bge-large-en-v1.5
202
The bge-large-en-v1.5 is a text embedding model created by BAAI (Beijing Academy of Artificial Intelligence). It is designed to generate high-quality embeddings for text sequences in English. This model builds upon BAAI's previous work on the bge-reranker-base and multilingual-e5-large models, which have shown strong performance on various language tasks. The bge-large-en-v1.5 model offers enhanced capabilities and is well-suited for a range of natural language processing applications. Model inputs and outputs The bge-large-en-v1.5 model takes text sequences as input and generates corresponding embeddings. Users can provide the text either as a path to a file containing JSONL data with a 'text' field, or as a JSON list of strings. The model also accepts a batch size parameter to control the processing of the input data. Additionally, users can choose to normalize the output embeddings and convert the results to a NumPy format. Inputs Path**: Path to a file containing text as JSONL with a 'text' field or a valid JSON string list. Texts**: Text to be embedded, formatted as a JSON list of strings. Batch Size**: Batch size to use when processing the text data. Convert To Numpy**: Option to return the output as a NumPy file instead of JSON. Normalize Embeddings**: Option to normalize the generated embeddings. Outputs The model outputs the text embeddings, which can be returned either as a JSON array or as a NumPy file, depending on the user's preference. Capabilities The bge-large-en-v1.5 model is capable of generating high-quality text embeddings that capture the semantic and contextual meaning of the input text. These embeddings can be utilized in a wide range of natural language processing tasks, such as text classification, semantic search, and content recommendation. The model's performance has been demonstrated in various benchmarks and real-world applications. What can I use it for? The bge-large-en-v1.5 model can be a valuable tool for developers and researchers working on natural language processing projects. The text embeddings generated by the model can be used as input features for downstream machine learning models, enabling more accurate and efficient text-based applications. For example, the embeddings could be used in sentiment analysis, topic modeling, or to power personalized content recommendations. Things to try To get the most out of the bge-large-en-v1.5 model, you can experiment with different input text formats, batch sizes, and normalization options to find the configuration that works best for your specific use case. You can also explore how the model's performance compares to other similar models, such as the bge-reranker-base and multilingual-e5-large models, to determine the most suitable approach for your needs.
Updated Invalid Date
bge_1-5_query_embeddings
center-for-curriculum-redesign
5
The bge_1-5_query_embeddings model is a query embedding generator developed by the Center for Curriculum Redesign. It is built on top of BAAI's bge-large-en v1.5 embedding model, which is a powerful text encoding model for embedding text sequences. Similar models include the bge-large-en-v1.5 model, the bge-reranker-base model, and the multilingual-e5-large model. Model inputs and outputs The bge_1-5_query_embeddings model takes in a list of text queries and generates corresponding embedding vectors for retrieval and comparison purposes. The model automatically formats the input queries for retrieval, so users do not need to preprocess the text. Inputs Query Texts**: A serialized JSON array of strings to be used as text queries for generating embeddings. Normalize**: A boolean flag to control whether the output embeddings are normalized to a magnitude of 1. Precision**: The numerical precision to use for the inference computations, either "full" or "half". Batchtoken Max**: The maximum number of kibiTokens (1 kibiToken = 1024 tokens) to include in a single batch, to avoid out-of-memory errors. Outputs Query Embeddings**: An array of embedding vectors, where each vector corresponds to one of the input text queries. Extra Metrics**: Additional metrics or data associated with the embedding generation process. Capabilities The bge_1-5_query_embeddings model is capable of generating high-quality text embeddings that can be used for a variety of natural language processing tasks, such as information retrieval, text similarity comparison, and document clustering. The embeddings capture the semantic meaning of the input text, allowing for more effective downstream applications. What can I use it for? The bge_1-5_query_embeddings model can be used in a wide range of applications that require text encoding and comparison, such as search engines, recommendation systems, and content analysis tools. By generating embeddings for text queries, you can leverage the model's powerful encoding capabilities to improve the relevance and accuracy of your search or recommendation results. Things to try One interesting thing to try with the bge_1-5_query_embeddings model is to experiment with different levels of precision for the inference computations. Depending on your specific use case and hardware constraints, you may find that the "half" precision setting provides sufficient accuracy while requiring less computational resources. Additionally, you could explore how the model's performance varies when using different normalization strategies for the output embeddings.
Updated Invalid Date
🤷
bge-base-zh
51
The bge-base-zh model is part of the BAAI FlagEmbedding suite, which focuses on retrieval-augmented language models. It is a Chinese-language text embedding model trained by BAAI using contrastive learning on a large-scale dataset. The model can map any Chinese text to a low-dimensional dense vector, which can be used for tasks like retrieval, classification, clustering, or semantic search. The FlagEmbedding project also includes the LLM-Embedder model, which is a unified embedding model designed to support diverse retrieval augmentation needs for large language models (LLMs). Additionally, the project features BGE Reranker models, which are cross-encoder models that are more accurate but less efficient than the embedding models. Model inputs and outputs Inputs Chinese text**: The model takes arbitrary Chinese text as input and encodes it into a low-dimensional dense vector. Outputs Embedding vector**: The model outputs a low-dimensional (e.g. 768-dimensional) dense vector representation of the input text. Capabilities The bge-base-zh model can map Chinese text to a semantic vector space, enabling a variety of downstream tasks. It has been shown to achieve state-of-the-art performance on the Chinese Massive Text Embedding Benchmark (C-MTEB), outperforming other widely used models like multilingual-e5 and text2vec. What can I use it for? The bge-base-zh model can be used for a variety of natural language processing tasks, such as: Semantic search**: Use the embeddings to find relevant documents or passages given a query. Text classification**: Train a classifier on top of the embeddings to categorize text into different classes. Clustering**: Group similar text together based on the embedding vectors. Semantic similarity**: Compute the similarity between two text snippets using the cosine similarity of their embeddings. The model can also be fine-tuned on domain-specific data to further improve performance on specialized tasks. Things to try One interesting aspect of the bge-base-zh model is its ability to generate embeddings without the need for an instruction prefix, which can simplify the usage in some scenarios. However, for retrieval tasks involving short queries and long passages, it is recommended to add an instruction prefix to the query to improve performance. When using the model, it's also important to consider the similarity distribution of the embeddings. The current bge-base-zh model has a similarity distribution in the range of [0.6, 1], so a similarity score greater than 0.5 does not necessarily indicate that the two sentences are similar. For downstream tasks, the relative order of the scores is more important than the absolute value.
Updated Invalid Date