From 9ffde58df6707040ebbac66bbeb73b8533810f77 Mon Sep 17 00:00:00 2001 From: Nirant Date: Thu, 7 Mar 2024 17:01:11 +0530 Subject: [PATCH] Getting Started Improvements (#138) * Update FastEmbed README.md * Rewrite GettingStarted to use TextEmbedding instead of DefaultEmbedding * Improve grammar * Update Getting Started.ipynb with model information and document format --- README.md | 35 ++--- docs/Getting Started.ipynb | 278 ++++++++++++++++++------------------- 2 files changed, 156 insertions(+), 157 deletions(-) diff --git a/README.md b/README.md index 2290be0..1017e1b 100644 --- a/README.md +++ b/README.md @@ -4,16 +4,13 @@ FastEmbed is a lightweight, fast, Python library built for embedding generation. The default text embedding (`TextEmbedding`) model is Flag Embedding, the top model in the [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard. It supports "query" and "passage" prefixes for the input text. Here is an example for [Retrieval Embedding Generation](https://qdrant.github.io/fastembed/examples/Retrieval_with_FastEmbed/) and how to use [FastEmbed with Qdrant](https://qdrant.github.io/fastembed/examples/Usage_With_Qdrant/). -1. Light & Fast - - Quantized model weights - - ONNX Runtime, no PyTorch dependency - - CPU-first design - - Data-parallelism for encoding of large datasets +## ๐Ÿ“ˆ Why FastEmbed? -2. Accuracy/Recall - - Better than OpenAI Ada-002 - - Default is Flag Embedding, which is top of the [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard - - List of [supported models](https://qdrant.github.io/fastembed/examples/Supported_Models/) - including multilingual models +1. Light: FastEmbed is a lightweight library with few external dependencies. We don't require a GPU and don't download GBs of PyTorch dependencies, and instead use the ONNX Runtime. This makes it a great candidate for serverless runtimes like AWS Lambda. + +2. Fast: FastEmbed is designed for speed. We use the ONNX Runtime, which is faster than PyTorch. We also use data-parallelism for encoding large datasets. + +3. Accurate: FastEmbed is better than OpenAI Ada-002. We also [supported](https://qdrant.github.io/fastembed/examples/Supported_Models/) an ever expanding set of models, including a few multilingual models. ## ๐Ÿš€ Installation @@ -26,18 +23,24 @@ pip install fastembed ## ๐Ÿ“– Quickstart ```python +import numpy as np from fastembed import TextEmbedding from typing import List -import numpy as np +# Example list of documents documents: List[str] = [ - "passage: Hello, World!", - "query: Hello, World!", # these are two different embedding - "passage: This is an example passage.", - "fastembed is supported by and maintained by Qdrant." # You can leave out the prefix but it's recommended + "This is built to be faster and lighter than other embedding libraries e.g. Transformers, Sentence-Transformers, etc.", + "fastembed is supported by and maintained by Qdrant.", ] -embedding_model = TextEmbedding(model_name="BAAI/bge-base-en") -embeddings: List[np.ndarray] = list(embedding_model.embed(documents)) # Note the list() call - this is a generator + +# This will trigger the model download and initialization +embedding_model = TextEmbedding() +print("The model BAAI/bge-small-en-v1.5 is ready to use.") + +embeddings_generator = embedding_model.embed(documents) # reminder this is a generator +embeddings_list = list(embedding_model.embed(documents)) + # you can also convert the generator to a list, and that to a numpy array +len(embeddings_list[0]) # Vector of 384 dimensions ``` ## Usage with Qdrant diff --git a/docs/Getting Started.ipynb b/docs/Getting Started.ipynb index aee9bcb..8c73db8 100644 --- a/docs/Getting Started.ipynb +++ b/docs/Getting Started.ipynb @@ -11,7 +11,9 @@ "\n", "## Quick Start\n", "\n", - "The fastembed package is designed to be easy to use. The main class is the `Embedding` class. It takes a list of strings as input and returns a list of vectors as output. The `Embedding` class is initialized with a model file." + "The fastembed package is designed to be easy to use. We'll be using `TextEmbedding` class. It takes a list of strings as input and returns an generator of vectors. If you're seeing generators for the first time, don't worry, you can convert it to a list using `list()`.\n", + "\n", + "> ๐Ÿ’ก You can learn more about generators from [Python Wiki](https://wiki.python.org/moin/Generators)" ] }, { @@ -21,15 +23,7 @@ "metadata": {}, "outputs": [], "source": [ - "!pip install fastembed --upgrade --quiet # Install fastembed " - ] - }, - { - "cell_type": "markdown", - "id": "ed81d725", - "metadata": {}, - "source": [ - "Make the necessary imports, initialize the `Embedding` class, and embed your data into vectors:" + "!pip install -Uqq fastembed # Install fastembed" ] }, { @@ -39,43 +33,115 @@ "metadata": {}, "outputs": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 76.7M/76.7M [00:05<00:00, 15.0MiB/s]\n", - "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 3/3 [00:00<00:00, 455.37it/s]" - ] + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "890cc3b969354eec8d149d143e301a7a", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Fetching 9 files: 0%| | 0/9 [00:00 ๐Ÿ’ก **Why do we use generators?**\n", + "> \n", + "> We use them to save memory mostly. Instead of loading all the vectors into memory, we can load them one by one. This is useful when you have a large dataset and you don't want to load all the vectors at once." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "8a225cb8", + "metadata": {}, + "outputs": [ + { + "name": "stdout", "output_type": "stream", "text": [ - "\n" + "Document: This is built to be faster and lighter than other embedding libraries e.g. Transformers, Sentence-Transformers, etc.\n", + "Vector of type: with shape: (384,)\n", + "Document: fastembed is supported by and maintained by Qdrant.\n", + "Vector of type: with shape: (384,)\n" ] } ], "source": [ - "from typing import List\n", - "import numpy as np\n", - "from fastembed.embedding import DefaultEmbedding\n", + "embeddings_generator = embedding_model.embed(documents) # reminder this is a generator\n", "\n", - "# Example list of documents\n", - "documents: List[str] = [\n", - " \"Hello, World!\",\n", - " \"This is an example document.\",\n", - " \"fastembed is supported by and maintained by Qdrant.\",\n", - "]\n", - "# Initialize the DefaultEmbedding class\n", - "embedding_model = DefaultEmbedding()\n", - "embeddings: List[np.ndarray] = list(embedding_model.embed(documents))\n", - "print(embeddings[0].shape)" + "for doc, vector in zip(documents, embeddings_generator):\n", + " print(\"Document:\", doc)\n", + " print(f\"Vector of type: {type(vector)} with shape: {vector.shape}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "769a1be9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(2, 384)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "embeddings_list = np.array(\n", + " list(embedding_model.embed(documents))\n", + ") # you can also convert the generator to a list, and that to a numpy array\n", + "embeddings_list.shape" ] }, { @@ -83,142 +149,72 @@ "id": "8c49ae50", "metadata": {}, "source": [ - "## Let's think step by step" - ] - }, - { - "cell_type": "markdown", - "id": "92cf4b76", - "metadata": {}, - "source": [ - "### Setup\n", - "\n", - "Importing the required classes and modules:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "c0a6f634", - "metadata": {}, - "outputs": [], - "source": [ - "from typing import List\n", - "import numpy as np\n", - "from fastembed.embedding import DefaultEmbedding" - ] - }, - { - "cell_type": "markdown", - "id": "3fd03a71", - "metadata": {}, - "source": [ - "Notice that we are using the DefaultEmbedding -- which is a quantized, state of the Art Flag Embedding model which beats OpenAI's Embedding by a large margin. \n", - "\n", - "### Prepare your Documents\n", - "You can define a list of documents that you'd like to embed. These can be sentences, paragraphs, or even entire documents. \n", + "We're using [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) a state of the art Flag Embedding model. The model does better than OpenAI text-embedding-ada-002. We've made it even faster by converting it to ONNX format and quantizing the model for you.\n", "\n", "#### Format of the Document List\n", + "\n", "1. List of Strings: Your documents must be in a list, and each document must be a string\n", - "2. For Retrieval Tasks: If you're working with queries and passages, you can add special labels to them:\n", + "2. For Retrieval Tasks with our default: If you're working with queries and passages, you can add special labels to them:\n", "- **Queries**: Add \"query:\" at the beginning of each query string\n", - "- **Passages**: Add \"passage:\" at the beginning of each passage string" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "145a56ce", - "metadata": {}, - "outputs": [], - "source": [ - "# Example list of documents\n", - "documents: List[str] = [\n", - " \"passage: Hello, World!\",\n", - " \"query: Hello, World!\", # these are two different embedding\n", - " \"passage: This is an example passage.\",\n", - " # You can leave out the prefix but it's recommended\n", - " \"fastembed is supported by and maintained by Qdrant.\",\n", - "]" - ] - }, - { - "cell_type": "markdown", - "id": "1cb3cc87", - "metadata": {}, - "source": [ - "### Load the Embedding Model Weights\n", - "Next, initialize the Embedding class with the desired parameters. Here, \"BAAI/bge-small-en\" is the pre-trained model name, and max_length=512 is the maximum token length for each document.\n", + "- **Passages**: Add \"passage:\" at the beginning of each passage string\n", "\n", - "This will download the model weights, decompress to directory `local_cache` and load them into the Embedding class.\n", + "## Beyond the default model\n", "\n", - "#### Initialize DefaultEmbedding\n", - "\n", - "We will initialize Flag Embeddings with the model name and the maximum token length. That is the DefaultEmbedding class with the model name \"BAAI/bge-small-en\" and max_length=512." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "272c8915", - "metadata": {}, - "outputs": [], - "source": [ - "embedding_model = DefaultEmbedding()" - ] - }, - { - "cell_type": "markdown", - "id": "5549d501", - "metadata": {}, - "source": [ - "### Embed your Documents\n", - "\n", - "Use the embed method of the embedding model to transform the documents into a List of np.array. The method returns a generator, so we cast it to a list to get the embeddings." + "The default model is built for speed and efficiency. If you need a more accurate model, you can use the `TextEmbedding` class to load any model from our list of available models. You can find the list of available models using `TextEmbedding.list_supported_models()`." ] }, { "cell_type": "code", "execution_count": 5, - "id": "8013eee9", + "id": "2e9c8766", "metadata": {}, "outputs": [ { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 4/4 [00:00<00:00, 361.82it/s]\n" - ] + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "9470ec542f3c4400a42452c2489a1abc", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Fetching 8 files: 0%| | 0/8 [00:00