Commit Graph

22 Commits

Author SHA1 Message Date
Hossam Hagag
860e2ad691 Tsk 375 add jina rerankers (#379)
* feat: Added jina reranker models

* chore: Added jina reranker canonical score values

* chore: added rounding of the output for easier reproducability

* chore: Added jina reranker models in batch test

* chore: remove redundant np.round

* chore: test only <1gb files in local

* chore: Updated docs to add rerankers

* fix: recompute canonical values with fp16

* new: extend NOTICE with jina reranker v2

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-11-13 10:48:45 +01:00
paulmartrencharpro
8413066f7b Add rerankers to the list of supported models (Supported_Models.ipynb) (#393)
* Add rerankers to the list of supported models (Supported_Models.ipynb)

* fix: update imports

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-11-13 09:59:28 +01:00
Hossam Hagag
1deb830328 Tsk 374 add jina colbert v2 (#378)
* feat: Added support for jina-colbert-v2

* chore: Generalized query marker and document marker

* nit: remove github action on dispatch

* chore: updated license

* fix: Fix attention mask to be all 1 in xlmrobertatokenizer

* feat: Added class for JinaColbertV2

* feat: Added jina colbert

* chore: Change tolerance of the test

* chore: Changed encoding of attention mask to 1 to be only in queries

* chore: Changed the replacable token to be ' @' as its considered as one token

* chore: Removed redundant functions

* chore: Updated supported models docs

* nit: Remove print statement

* nit: visual stuff

* fix: Fix dimention of jina colbert in description

* fix: canonical query and document values for jina colbert
2024-11-03 11:02:41 +02:00
Anush
0e258ab875 feat: Added jina-embeddings-v2-base-code (#301)
* feat: Added jina-embeddings-v2-base-code

* fix: test embeddings for "hello world" not "Hello"

* docs: Updated supported models
2024-07-18 18:17:46 +05:30
Anush
70566dff99 docs: Updated supported models (#302) 2024-07-17 16:44:50 +05:30
Andrey Vasnetsov
29cfcda056 add examples with supported typed of models into readme (#271)
* add examples with supported typed of models into readme

* fix link

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-06-13 10:58:48 +02:00
George
48c59dc7b3 new: update supported models (#253) 2024-06-05 17:50:07 +02:00
Anush
cc4112d859 feat: Snowflake models (#207)
* feat: Snowflake models

* Added snowflake/snowflake-arctic-embed-m

* docs: snowflake/snowflake-arctic-embed-m
2024-04-19 19:00:10 +05:30
Nirant
3d2254d215 Bump version to 0.2.6 in pyproject.toml (#175) 2024-04-01 17:16:07 +05:30
George
e11f66bb55 refactoring: update imports in notebooks (#173)
* new: simplify imports

* refactoring: update import

* refactoring: update imports in notebooks

* fix: fix notebook output

* Re-run notebook with revised imports

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
2024-04-01 17:02:13 +05:30
George
8b7b8476a5 fix: fix model sizes in supported models lists (#167)
* fix: fix model sizes in supported models lists

* fix: remove redundant comment

* fix: fix test

* fix: update supported models notebook

* Consistentcy around quantization in supported_onnx_models

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
2024-04-01 16:18:49 +05:30
Yuvraj Wale
bc1e23849b feat: support mixedbread-ai/mxbai-embed-large-v1 (#158)
* add: support mixedbread-ai/mxbai-embed-large-v1

* refactor: canonical vector

* Update fastembed/text/onnx_embedding.py

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
2024-03-22 06:23:54 +05:30
Anush
1e298a00b3 feat: Added gte-large, nomic-text 1.5, cleanup (#130) 2024-02-21 17:54:32 +05:30
Armaghan
406f432edc feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 (#129)
* feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* test: Include sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* docs: supported models update
2024-02-21 13:35:49 +05:30
Anush
98141cc8d3 feat: Added nomic-embed-text-v1 support + formatting changes + import fixes (#118)
* feat: Added nomic-embed-text-v1 support

* chore: xenova/nomic-embed-text-v1 -> nomic-ai/nomic-embed-text-v1

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
2024-02-19 14:17:33 +05:30
Nirant
81bab0cd1d Make 0.2.1 Release + Update docs (#116)
* Update version from 0.2.0 (yanked) to 0.2.1

* Update text embedding to include prefix for passages and queries

* Update supported models to use the latest API

* * fix(text_embedding_base.py): remove unnecessary prefix from texts in embed method
* feat(text_embedding_base.py): update query_embed method to updated instruction for the v1.5 model

* Remove comparison, since the ranking is identical even with varying embedding

* Refactor text embedding query handling
2024-02-08 09:06:00 +05:30
Anush
96f7d83d33 feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multili… (#103)
* feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multilingual-mpnet-base-v2

* chore: updated exclude_token_type_ids check

* docs: supported models update
2024-02-02 15:16:35 +05:30
Anush
9b63427118 chore: pre-commit formatting (#91)
* chore: formatting

* chore: formatting

* chore: remove other hooks

* Update poetry lock

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
2024-01-16 15:06:54 +05:30
Nirant
55379539ef * chore(docs): update Getting Started.ipynb with progressbar + New Models
* * feat(Supported_Models.ipynb): add support for BAAI/bge-small-zh-v1.5 model
* feat(Supported_Models.ipynb): add support for jinaai/jina-embeddings-v2-base-en model
* feat(Supported_Models.ip

* * chore(docs): update Getting Started.ipynb with progressbar
2023-12-13 14:17:25 +05:30
NirantK
c1fdaf3303 * chore(Supported_Models.ipynb): update supported models table
* feat(Supported_Models.ipynb): add size_in_GB column to supported models table
2023-10-18 23:02:50 +05:30
NirantK
dd2a9e9bcc * docs(Supported_Models.ipynb): update model descriptions 2023-09-27 17:41:39 +05:30
NirantK
71289926c0 * feat(Supported_Models.ipynb): add example notebook for supported models 2023-09-27 17:40:40 +05:30