Commit Graph

26 Commits

Author SHA1 Message Date
Anush
1e298a00b3 feat: Added gte-large, nomic-text 1.5, cleanup (#130) 2024-02-21 17:54:32 +05:30
Armaghan
406f432edc feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 (#129)
* feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* test: Include sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* docs: supported models update
2024-02-21 13:35:49 +05:30
Anush
98141cc8d3 feat: Added nomic-embed-text-v1 support + formatting changes + import fixes (#118)
* feat: Added nomic-embed-text-v1 support

* chore: xenova/nomic-embed-text-v1 -> nomic-ai/nomic-embed-text-v1

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
2024-02-19 14:17:33 +05:30
generall
fcdc5690b9 rename models 2024-02-02 15:51:19 +01:00
generall
7883fa3c41 new multilingual models 2024-02-02 15:32:42 +01:00
generall
8b800da7bc ruff 2024-02-02 15:12:57 +01:00
generall
4813b18854 refactoring
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-02-02 15:12:57 +01:00
Anush
96f7d83d33 feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multili… (#103)
* feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multilingual-mpnet-base-v2

* chore: updated exclude_token_type_ids check

* docs: supported models update
2024-02-02 15:16:35 +05:30
Anush
ede507e2cf feat: HuggingFace download support for FlagEmbedding (#94)
* feat: HF support for FlagEmbedding

* chore: update docstring embedding.py

* refactor: GCS URLs models.json

* chore: toLower() models.json

* chore: update tqdm declarative

* chore: exclude keys list_supported_models

* chore: review changes
2024-01-23 12:47:55 +05:30
Anush
9b63427118 chore: pre-commit formatting (#91)
* chore: formatting

* chore: formatting

* chore: remove other hooks

* Update poetry lock

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
2024-01-16 15:06:54 +05:30
Joan Fontanals
f222d7cd87 add JinaEmbeddings class (#67)
* add JinaEmbeddings class

* fix tests dimensions

---------

Co-authored-by: Joan Fontanals Martinez <joan.fontanals.martinez@jina.ai>
2023-11-20 16:37:23 +05:30
NirantK
f28087c71a * fix(embedding.py): change dim value from 384 to 512 for the "BAAI/bge-small-zh-v1.5" model
* fix(test_onnx_embeddings.py): add canonical vector values for the "BAAI/bge-small-zh-v1.5"
2023-10-16 18:23:02 +05:30
Nirant
f79de09ff7 Merge branch 'streaming-inference' into add_model_size 2023-10-16 17:54:33 +05:30
NirantK
e24ea64e21 * test(test_onnx_embeddings.py): skip specific model if size_in_GB is greater than 1 2023-10-16 17:52:35 +05:30
generall
35ec40b3a3 review fixes 2023-10-16 14:19:58 +02:00
Nirant
c953a083cc Merge branch 'main' into streaming-inference 2023-10-16 17:33:36 +05:30
NirantK
9e5d37846c * feat(embedding.py): add support for BAAI/bge-small-en-v1.5 and BAAI/bge-base-en-v1.5 models
* feat(embedding.py): change default model to v1.5
2023-10-16 17:08:58 +05:30
generall
c719fc696d disable large models on non-ubuntu CI 2023-10-16 13:27:28 +02:00
generall
24dc24b02d implement data-parallel inference and up version 2023-10-16 13:07:07 +02:00
generall
8c6d4d2b52 remove optimum + more test + fix batching embed + ci on other machines 2023-10-03 21:19:58 +02:00
NirantK
4d6d27cffb * test(test_onnx_embeddings.py): remove unnecessary list conversion in embeddings assignment 2023-09-25 16:50:52 +05:30
NirantK
3149694c04 Utility script version of the notebook 2023-09-25 15:33:44 +05:30
NirantK
a335c8898f Remove attention pooling 2023-09-18 07:45:12 +05:30
NirantK
03b23af0a0 * refactor(test_onnx_embeddings.py): rename test_onnx_embeddigns.py to test_onnx_embeddings.py
* chore(test_onnx_embeddings.py): remove unnecessary line breaks and whitespace
* test(test_onnx_inference): add test case for onnx inference
2023-08-25 00:37:31 +05:30
NirantK
0c7574e95a * fix(test_onnx_embeddigns.py): change method name from encode to embed 2023-08-23 09:51:44 +05:30
generall
770beb1023 refactoring 2023-08-18 01:36:36 +02:00