upstream-archive/fastembed
1
0
Fork 0
mirror of https://github.com/qdrant/fastembed.git synced 2026-07-23 11:20:51 -05:00
Code Issues Packages Projects Releases Wiki Activity
355 Commits 38 Branches 59 Tags
dynamic_quantization
Commit Graph

355 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nirant Kasliwal
2b485e6e6a Add dynamic quantization from @Xenova 2024-02-27 14:10:07 +05:30
Nirant Kasliwal
b5bce4c2a1 Add try_quantized_model.py for feature extraction 2024-02-26 15:52:45 +05:30
Anush
1e298a00b3 feat: Added gte-large, nomic-text 1.5, cleanup (#130) 2024-02-21 17:54:32 +05:30
Nathan LeRoy
38c4eb1cc5 Check for existing files in cache dir before instantiating a model (#128) 
* check for existing files

* (chore: model_management.py):  Add comment

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-02-21 13:56:14 +05:30
Armaghan
406f432edc feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 (#129) 
* feat: Support sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* test: Include sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

* docs: supported models update
 2024-02-21 13:35:49 +05:30
Nirant
defb6183c1 * feat(pyproject): updated version to '0.2.2' (#124) 
* chore(pyproject): updated dev dependencies versions
v0.2.2 		2024-02-19 14:46:01 +05:30
Anush
98141cc8d3 feat: Added nomic-embed-text-v1 support + formatting changes + import fixes (#118) 
* feat: Added nomic-embed-text-v1 support

* chore: xenova/nomic-embed-text-v1 -> nomic-ai/nomic-embed-text-v1

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-02-19 14:17:33 +05:30
Nirant
b1f5e7a989 Add import statement to the warning message (#119) 2024-02-15 12:40:11 +05:30
Kumar Shivendu
558a837531 Merge pull request #112 from qdrant/KShivendu-patch-1 
docs: Describe how to change the model and how to just create embeddings
 2024-02-13 10:33:41 +05:30
Nirant
b81e40c95d Merge branch 'main' into KShivendu-patch-1 2024-02-13 08:25:51 +05:30
Nirant
81bab0cd1d Make 0.2.1 Release + Update docs (#116) 
* Update version from 0.2.0 (yanked) to 0.2.1

* Update text embedding to include prefix for passages and queries

* Update supported models to use the latest API

* * fix(text_embedding_base.py): remove unnecessary prefix from texts in embed method
* feat(text_embedding_base.py): update query_embed method to updated instruction for the v1.5 model

* Remove comparison, since the ranking is identical even with varying embedding

* Refactor text embedding query handling
v0.2.1 		2024-02-08 09:06:00 +05:30
Nirant
973da354ae Fix query to align with Qdrant mixin usage (#115) 
* fix: query in text_embedding_base to work with both Iterable and str as users might supply both

* Fix Qdrant query to align with future usage

* * refactor(text_embedding_base.py): change query parameter type from str to Union[str, Iterable[str]] in query_embed method

* Update return type of query_embed method

* Update return type in TextEmbeddingBase
 2024-02-07 22:01:31 +05:30
Zihong Lin
46968181ad Simplify imports: #110 (#113) 
* Simplify imports: #110

* Update fastembed/__init__.py

Co-authored-by: Kumar Shivendu <kshivendu1@gmail.com>

* Remove outdated import

* Remove outdated import

---------

Co-authored-by: Kumar Shivendu <kshivendu1@gmail.com>
Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-02-07 21:12:46 +05:30
Nirant Kasliwal
3948f0db2e Update fastembed v0.2.0 v0.2.0 2024-02-07 20:36:46 +05:30
Nirant
cf66d0e5e1 Update Python and dependency versions (#111) 2024-02-05 12:37:52 +01:00
Kumar Shivendu
c11ba70fbc Update README.md 2024-02-05 07:17:10 +01:00
Kumar Shivendu
ea3ef26fa2 Improve README 2024-02-05 07:07:41 +01:00
Kumar Shivendu
4b1ffb47f0 docs: Describe how to change the model and how to just create embeddings 2024-02-05 11:24:47 +05:30
Kumar Shivendu
d3f5f29ee0 docs: Improve README (#109) 2024-02-05 10:49:45 +05:30
Kumar Shivendu
05885a36dd refactor: Introduce experiments dir (#108) 2024-02-05 10:49:12 +05:30
Andrey Vasnetsov
a3bc73c556 Merge pull request #105 from qdrant/refactoring-off-everything 
Refactoring of internal structure
 2024-02-02 16:48:04 +01:00
generall
fcdc5690b9 rename models 2024-02-02 15:51:19 +01:00
generall
7883fa3c41 new multilingual models 2024-02-02 15:32:42 +01:00
generall
5dbd0073e2 rename flag -> onnx 2024-02-02 15:12:57 +01:00
generall
8b800da7bc ruff 2024-02-02 15:12:57 +01:00
generall
3e6f69e2eb review fixes 2024-02-02 15:12:57 +01:00
generall
4813b18854 refactoring 
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
 2024-02-02 15:12:57 +01:00
Anush
96f7d83d33 feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multili… (#103) 
* feat: Support xenova/multilingual-e5-large, xenova/paraphrase-multilingual-mpnet-base-v2

* chore: updated exclude_token_type_ids check

* docs: supported models update
 2024-02-02 15:16:35 +05:30
Nirant
2e3e5508c0 Update poetry lock to latest versions (#98) 
* Update poetry lock to latest versions

* Update poetry lock to latest versions
 2024-01-30 21:39:52 +05:30
Anush
87decb0d53 chore: port to Xenova Jina source (#102) 
* chore: xenova jina

* chore: try recusive model location

* chore: updated doc string, blob pattern
 2024-01-30 21:27:16 +05:30
Anush
ede507e2cf feat: HuggingFace download support for FlagEmbedding (#94) 
* feat: HF support for FlagEmbedding

* chore: update docstring embedding.py

* refactor: GCS URLs models.json

* chore: toLower() models.json

* chore: update tqdm declarative

* chore: exclude keys list_supported_models

* chore: review changes
 2024-01-23 12:47:55 +05:30
David Janes
f87330fcd1 use "with" to open JSON files (#96) 2024-01-22 12:20:06 +05:30
Nirant
3b32619a4c Update Python version and add pre-commit dependency (#93) 
* Update Python version and add pre-commit dependency

* Remove Python 3.8.x from matrix

* Update Python version and pre-commit configuration
 2024-01-16 19:30:47 +05:30
Anush
9b63427118 chore: pre-commit formatting (#91) 
* chore: formatting

* chore: formatting

* chore: remove other hooks

* Update poetry lock

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-01-16 15:06:54 +05:30
Nirant
b01f882df7 Revert "feat: embedding progress bar (#71)" (#77) 
This reverts commit 2c7fee3b95.
v0.1.3 		2023-12-13 15:13:00 +05:30
Nirant
55379539ef * chore(docs): update Getting Started.ipynb with progressbar + New Models 
* * feat(Supported_Models.ipynb): add support for BAAI/bge-small-zh-v1.5 model
* feat(Supported_Models.ipynb): add support for jinaai/jina-embeddings-v2-base-en model
* feat(Supported_Models.ip

* * chore(docs): update Getting Started.ipynb with progressbar
 2023-12-13 14:17:25 +05:30
NirantK
ea85e7430f Bump version 2023-12-13 13:27:30 +05:30
Anush
2c7fee3b95 feat: embedding progress bar (#71) 
* feat: embedding progress

* refactor: with auto __close__

* refactor: with __exit__ tqdm
v0.1.2 		2023-12-12 19:33:12 +05:30
Anush
e274dd0fc2 chore: bump tokenizers (#75) 2023-12-12 18:50:47 +05:30
Anush
0a94425735 feat: Added support for FASTEMBED_CACHE_PATH env var (#68) 
* chore: FASTEMBED_CACHE_PATH env

* chore: temp directory fallback

* chore: tempdir fallback JinaEmbedding
 2023-11-22 10:39:18 +05:30
Joan Fontanals
f222d7cd87 add JinaEmbeddings class (#67) 
* add JinaEmbeddings class

* fix tests dimensions

---------

Co-authored-by: Joan Fontanals Martinez <joan.fontanals.martinez@jina.ai>
 2023-11-20 16:37:23 +05:30
Nirant
d64b8f42f0 * chore(pyproject.toml): add huggingface-hub dependency (#66) 
* chore(pyproject.toml): update pytest version to 7.4.2
 2023-11-20 15:10:56 +05:30
dependabot[bot]
2f95205b23 build(deps): bump urllib3 from 2.0.6 to 2.0.7 (#65) 
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.0.6...2.0.7)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-11-16 11:37:48 +05:30
Nirant
a39ee46c0b Update EmbeddingModel class to remove ABC (#57) 
inheritance
 2023-11-02 15:37:43 +05:30
Dominik Weckmüller
bb86b30707 Add typing and numpy import 
typing and numpy import were missing
 2023-11-01 20:20:47 +05:30
Andrey Vasnetsov
8c20c7c172 Merge pull request #55 from qdrant/tokenizers-upgrade 
Update tokenizers dependency version to >=0.14
 2023-11-01 15:50:27 +01:00
NirantK
5f40fc2f14 * chore(pyproject.toml): update tokenizers dependency version to be at least 0.14 2023-11-01 20:14:01 +05:30
NirantK
ab2f41ef8b * chore(pyproject.toml): update tokenizers dependency version to ^0.14.1 2023-11-01 20:12:18 +05:30
Nirant
78416dd728 Merge pull request #48 from qdrant/remove-docs-clutter 
Docs: Move cluttered notebook + Fix typos
 2023-10-30 23:07:09 +05:30
NirantK
9c85a899c9 * docs(experimental): update dataset size in Binary Quantization with Qdrant.ipynb from 10K to 100K 2023-10-30 23:02:18 +05:30