upstream-archive/fastembed
1
0
Fork 0
mirror of https://github.com/qdrant/fastembed.git synced 2026-07-23 19:30:50 -05:00
Code Issues Packages Projects Releases Wiki Activity
408 Commits 38 Branches 59 Tags
gpu-runtime-debug
Commit Graph

119 Commits

Author SHA1 Message Date
Anush
cc4112d859 feat: Snowflake models (#207) 
* feat: Snowflake models

* Added snowflake/snowflake-arctic-embed-m

* docs: snowflake/snowflake-arctic-embed-m
 2024-04-19 19:00:10 +05:30
Nirant
864217a7d9 Re-run of identical hardware and generate graphs (#205) 2024-04-18 11:13:18 +05:30
George
f340a73a3e refactoring: update binary quantization notebook (#180) 2024-04-02 14:45:27 +05:30
Nirant
c09909773e Add Execution Counts (#177) 
* Update notebooks

* Update FastEmbed usage across docs

* Refactor code for better readability and maintainability

* Clean outputs

* Change dataset

* Add numbers inline in output

* Remove inline outputs since I used :memory:

* Fix syntax error in Hindi_Tamil_RAG_with_Navarasa7B.ipynb
 2024-04-01 20:48:26 +05:30
Nirant
3d2254d215 Bump version to 0.2.6 in pyproject.toml (#175) 2024-04-01 17:16:07 +05:30
George
e11f66bb55 refactoring: update imports in notebooks (#173) 
* new: simplify imports

* refactoring: update import

* refactoring: update imports in notebooks

* fix: fix notebook output

* Re-run notebook with revised imports

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-04-01 17:02:13 +05:30
George
8b7b8476a5 fix: fix model sizes in supported models lists (#167) 
* fix: fix model sizes in supported models lists

* fix: remove redundant comment

* fix: fix test

* fix: update supported models notebook

* Consistentcy around quantization in supported_onnx_models

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-04-01 16:18:49 +05:30
Nirant
e3d2e1dc44 Hybrid Search Tutorial (#165) 
* Re-organize docs

* Rename notebooks

* Move nbs

* Working Sparse and Dense Search

* Add RRF

* Refactor code to improve performance and readability

* Add ESCI label for the RRF results

* Update docs/examples/Hybrid_Search.ipynb

Co-authored-by: Anush  <anushshetty90@gmail.com>

* Update docs/examples/Hybrid_Search.ipynb

Co-authored-by: Anush  <anushshetty90@gmail.com>

* Remove unnecessary code and update vector format

---------

Co-authored-by: Anush <anushshetty90@gmail.com>
 2024-03-29 21:10:14 +05:30
Nirant Kasliwal
c651b2b539 Update SPLADE notebook with new sections 2024-03-22 15:18:34 +05:30
Yuvraj Wale
bc1e23849b feat: support mixedbread-ai/mxbai-embed-large-v1 (#158) 
* add: support mixedbread-ai/mxbai-embed-large-v1

* refactor: canonical vector

* Update fastembed/text/onnx_embedding.py

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-03-22 06:23:54 +05:30
Nirant
96a2a9097a Fix model name typo + Add SPLADE notebook (#155) 
* Rename model + Add SPLADE notebook

* Update docs/examples/SPLADE_with_FastEmbed.ipynb

Co-authored-by: Anush  <anushshetty90@gmail.com>

* Update docs/examples/SPLADE_with_FastEmbed.ipynb

Co-authored-by: Anush  <anushshetty90@gmail.com>

* Update docs/examples/SPLADE_with_FastEmbed.ipynb

Co-authored-by: Anush  <anushshetty90@gmail.com>

* Update CANONICAL_COLUMN_VALUES in test_sparse_embeddings.py

---------

Co-authored-by: Anush <anushshetty90@gmail.com>
 2024-03-20 18:15:53 +05:30
Nirant
d817da2e01 Add Splade v1 (#144) 
* Add SPLADE v1

* WIP SPLADE Export errors

* add ONNX model to HF hub and use that

* Update sentences in Converting_SPLADE_to_ONNX.ipynb

* Remove unnecessary files and directories

* Rename var in TextEmbedding class to use EMBEDDING_MODEL_TYPE

* Add SPLADE to list of text embeddings

* Add SPLADE model support for text embedding

* Fix deprecation warning in embedding.py

* Add test for batch embedding with sparse embeddings

* Refactor import statement in test_sparse_embeddings.py

* Rename nbs

* Update vocab size in SPLADE model

* Fix canonical vector lookup in test_text_onnx_embeddings.py

* review refactoring

* restore list_supported_models in OnnxTextEmbedding

* Remove unused method _preprocess_onnx_input() in SpladePP class

* Update SPLADE_PP_en_v1 source in splade_pp.py

* Refactor onnx_model.py to change base model behavior

* extend tests to sparse values as well as indicies

* chore: pre-commit hooks

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Anush008 <anushshetty90@gmail.com>
 2024-03-13 18:04:44 +05:30
Artem Daineko
68635efa3f Fix link to Optimus in docs (#143) 2024-03-10 11:45:44 +05:30
Nirant
9ffde58df6 Getting Started Improvements (#138) 
* Update FastEmbed README.md

* Rewrite GettingStarted to use TextEmbedding instead of DefaultEmbedding

* Improve grammar

* Update Getting Started.ipynb with model information and document format
 2024-03-07 17:01:11 +05:30
Nirant Kasliwal
5603fbe1fb Fix naming typo 2024-03-06 14:30:54 +05:30
Nirant Kasliwal
9ed4486d9c Rename typo in notebook 2024-03-06 14:24:48 +05:30
Nirant Kasliwal
e36a39c388 Update author information in notebook 2024-03-06 14:18:48 +05:30
Nirant Kasliwal
97c359c1b9 Add Navrasa LLM download and explanation sections 2024-03-06 14:17:41 +05:30
Nirant Kasliwal
9fd51425fe Add author information and Colab link to notebook 2024-03-06 14:12:04 +05:30
Nirant Kasliwal
0dec22d02d Remove inline outputs 2024-03-06 14:06:46 +05:30
Nirant Kasliwal
8a3b746b71 Rename notebook 2024-03-06 14:05:13 +05:30
Nirant
337ad9c93f Hindi RAG with Qdrant and FastEmbed (#135) 
* Add workingnb

* Add A100 Colab

* Remove old checkpoint

* Refactor code to separate HF Token
 2024-03-06 14:02:32 +05:30
Anush
1e298a00b3 feat: Added gte-large, nomic-text 1.5, cleanup (#130) 2024-02-21 17:54:32 +05:30
Armaghan
406f432edc feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 (#129) 
* feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* test: Include sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* docs: supported models update
 2024-02-21 13:35:49 +05:30
Anush
98141cc8d3 feat: Added nomic-embed-text-v1 support + formatting changes + import fixes (#118) 
* feat: Added nomic-embed-text-v1 support

* chore: xenova/nomic-embed-text-v1 -> nomic-ai/nomic-embed-text-v1

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-02-19 14:17:33 +05:30
Nirant
81bab0cd1d Make 0.2.1 Release + Update docs (#116) 
* Update version from 0.2.0 (yanked) to 0.2.1

* Update text embedding to include prefix for passages and queries

* Update supported models to use the latest API

* * fix(text_embedding_base.py): remove unnecessary prefix from texts in embed method
* feat(text_embedding_base.py): update query_embed method to updated instruction for the v1.5 model

* Remove comparison, since the ranking is identical even with varying embedding

* Refactor text embedding query handling
 2024-02-08 09:06:00 +05:30
Nirant
973da354ae Fix query to align with Qdrant mixin usage (#115) 
* fix: query in text_embedding_base to work with both Iterable and str as users might supply both

* Fix Qdrant query to align with future usage

* * refactor(text_embedding_base.py): change query parameter type from str to Union[str, Iterable[str]] in query_embed method

* Update return type of query_embed method

* Update return type in TextEmbeddingBase
 2024-02-07 22:01:31 +05:30
Anush
96f7d83d33 feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multili… (#103) 
* feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multilingual-mpnet-base-v2

* chore: updated exclude_token_type_ids check

* docs: supported models update
 2024-02-02 15:16:35 +05:30
Anush
9b63427118 chore: pre-commit formatting (#91) 
* chore: formatting

* chore: formatting

* chore: remove other hooks

* Update poetry lock

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-01-16 15:06:54 +05:30
Nirant
55379539ef * chore(docs): update Getting Started.ipynb with progressbar + New Models 
* * feat(Supported_Models.ipynb): add support for BAAI/bge-small-zh-v1.5 model
* feat(Supported_Models.ipynb): add support for jinaai/jina-embeddings-v2-base-en model
* feat(Supported_Models.ip

* * chore(docs): update Getting Started.ipynb with progressbar
 2023-12-13 14:17:25 +05:30
NirantK
9c85a899c9 * docs(experimental): update dataset size in Binary Quantization with Qdrant.ipynb from 10K to 100K 2023-10-30 23:02:18 +05:30
NirantK
299042d592 * docs(examples): add explanation of Qdrant Client usage with FastEmbed library and Qdrant API 2023-10-30 23:01:26 +05:30
NirantK
d35ff16994 * chore(docs): rename Throughput_Across_Models.ipynb to fooling_around/Throughput_Across_Models.ipynb 2023-10-30 23:01:19 +05:30
Nirant
f8f8316fea Merge pull request #38 from qdrant/explain_cossim 
* docs(examples): update FastEmbed_vs_HF_Comparison.ipynb
 2023-10-19 22:56:04 +05:30
NirantK
4999fa17b5 * docs(examples): update FastEmbed_vs_HF_Comparison.ipynb 
with cosine similarity values for BAAI/bge-small-en and BAAI/bge-small-en-v1.5 embeddings
 2023-10-19 22:49:25 +05:30
NirantK
ad297c4f13 * chore(Usage_With_Qdrant.ipynb): remove unnecessary outputs in code cells 2023-10-18 23:03:52 +05:30
NirantK
c1fdaf3303 * chore(Supported_Models.ipynb): update supported models table 
* feat(Supported_Models.ipynb): add size_in_GB column to supported models table
 2023-10-18 23:02:50 +05:30
NirantK
b61f8a48cc Update to v1.5 model 2023-10-18 20:49:31 +05:30
NirantK
a14aab8ef4 * refactor(Getting Started.ipynb): simplify code for initializing DefaultEmbedding class 2023-10-18 20:48:51 +05:30
NirantK
c408b7e13e * chore(main.html): add utm parameters to Qdrant Cloud link 2023-10-10 19:01:49 +05:30
NirantK
d2bdfee4e0 * docs(examples): update comparison notebook with more accurate description of embeddings similarity 2023-10-10 17:56:25 +05:30
NirantK
ac4375516f Rename nbs; add Cosine similarity check 2023-10-10 17:55:37 +05:30
NirantK
ca6f9d629a Add skeleton 2023-10-05 19:20:01 +05:30
NirantK
bc402694bd Update code to handle Generator 2023-10-05 19:17:18 +05:30
NirantK
050d80ab51 * chore(docs): update index.md with FastEmbed library information and usage examples 
* feat(docs): add installation instructions for FastEmbed with Qdrant Client
 2023-09-28 10:52:38 +05:30
NirantK
49c2b3e7a9 * chore(docs): update installation command for fastembed in Getting Started.ipynb 
* fix(docs): remove unnecessary casting of generator to list in code cell
 2023-09-28 10:43:14 +05:30
NirantK
4949158eff * chore(Usage_With_Qdrant.ipynb): update notebook title and remove experimental note 
* docs(Usage_With_Qdrant.ipynb): add support for qdrant-client[fastembed] installation
 2023-09-27 17:53:05 +05:30
NirantK
dd2a9e9bcc * docs(Supported_Models.ipynb): update model descriptions 2023-09-27 17:41:39 +05:30
NirantK
71289926c0 * feat(Supported_Models.ipynb): add example notebook for supported models 2023-09-27 17:40:40 +05:30
NirantK
547130a4c3 Move throughput comparison to docs 2023-09-25 17:40:38 +05:30