upstream-archive/fastembed
1
0
Fork 0
mirror of https://github.com/qdrant/fastembed.git synced 2026-07-31 15:11:40 -05:00
Code Issues Packages Projects Releases Wiki Activity
458 Commits 38 Branches 59 Tags
4e699cc9fa7af0899fafdbddffddd082ded78230
Commit Graph

458 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dmitrii Rudenko
4e699cc9fa Isort for image tests 2024-07-30 19:37:44 +03:00
Dmitrii Rudenko
90a430cc40 Added selfish logo test as requests input 2024-07-30 19:23:01 +03:00
Dmitrii Rudenko
f59ba2e89d Additional tests for image embeddings 2024-07-30 18:44:37 +03:00
Dmitrii Rudenko
3ddc6ae56a Initial commit for opened images support 2024-07-30 18:18:38 +03:00
Anush
0e258ab875 feat: Added jina-embeddings-v2-base-code (#301) 
* feat: Added jina-embeddings-v2-base-code

* fix: test embeddings for "hello world" not "Hello"

* docs: Updated supported models
 2024-07-18 18:17:46 +05:30
George
e49789c129 fix: update push gpu command (#300) 2024-07-18 12:22:12 +03:00
Anush
1bf72922ce docs: fixed README.md examples (#298) 2024-07-17 17:49:29 +05:30
Anush
70566dff99 docs: Updated supported models (#302) 2024-07-17 16:44:50 +05:30
George Panchuk
fd116dd507 bump version to 0.3.4 v0.3.4 2024-07-15 15:59:52 +03:00
George
0315c3b8c6 new: add modifier flag into bm models config (#299) 
* new: add modifier flag into bm models config

* refactoring: rename modifier field
 2024-07-15 15:57:21 +03:00
Dmitrii Ogn
54e0f38914 Update README.md (#295) 2024-07-11 13:39:19 +03:00
George
3a8985b35c new: add retry logic for model downloading (#293) 
* new: add retry logic for model downloading

* fix: add sleep
 2024-07-10 20:09:11 +03:00
Dmitrii Ogn
f0ff09c546 Oml zoo (#291) 
* Support of Qdrant/Unicom-ViT-B-16 and Qdrant/Unicom-ViT-B-32
 2024-07-10 16:43:54 +03:00
Dmitrii Ogn
d09af55edd Nomic-embeddings-support (#280) 
* Nomic-embeddings-support

* Jina models moved to pooled-normalized embeddings

* Canonical vector for nomic-ai/nomic-embed-text-v1.5-Q

* Moved all nomics to pooled_embeddings

---------

Co-authored-by: d.rudenko <dimitriyrudenk@gmail.com>
 2024-07-10 11:45:30 +03:00
generall
9387ca3205 bump version to v0.3.3 v0.3.3 2024-07-06 00:53:45 +02:00
Andrey Vasnetsov
9c74fb3cfb unique tokens in query (#287) 2024-07-06 00:52:38 +02:00
generall
1fe42d8d18 bump version to v0.3.2 v0.3.2 2024-07-05 13:31:04 +02:00
Andrey Vasnetsov
f820c36656 fix + test for empty from_dict (#285) 2024-07-05 13:30:14 +02:00
Anush
35c535aae3 chore: Pin numpy <2 (#278) v0.3.1 2024-06-17 23:33:09 +05:30
George
e071c84f22 fix: fix hybrid search example for pydantic v1 (#263) 2024-06-14 17:25:15 +02:00
George
e1ecfe9c2f fix: fix None cache dir in parallel mode (#277) 2024-06-14 17:19:53 +02:00
Dmitrii Ogn
331207976e MiniLM fix (#275) 
* MiniLM fix

* Added MiniLM to text embedding
Fixed MiniLM source destination
Black + isort for repo

* Fixed model all-MiniLM-L6-v2 description
Recomputed canonical vector for all-MiniLM-L6-v2 in test

---------

Co-authored-by: d.rudenko <dimitriyrudenk@gmail.com>
 2024-06-14 16:42:31 +03:00
Klaus Hueck
fd0b26f009 Add support for jinaai/jina-embeddings-v2-base-de (#270) 
* feat: add support for SOTA german embedding model with long context length jinaai/jina-embeddings-v2-base-de

* Fix jina de model weight

---------

Co-authored-by: George <panchuk.george@outlook.com>
 2024-06-14 13:39:13 +02:00
George
5461012ab1 new: add bm25, fix param propagation in parallel mode, fix bm42 parallel (#274) 
* new: add bm25, fix param propagation in parallel mode, fix bm42 parallel

* refactoring: remove redundant example

* fix: fix mp start method in bm25

* refactoring: refactor token id generation

* new: replace model repository
 2024-06-13 19:52:48 +02:00
Andrey Vasnetsov
29cfcda056 add examples with supported typed of models into readme (#271) 
* add examples with supported typed of models into readme

* fix link

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update README.md

Co-authored-by: George <george.panchuk@qdrant.tech>

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
 2024-06-13 10:58:48 +02:00
Nirant
615d6ee2b6 Replace Data Source (#206) 
* Re-run of identical hardware and generate graphs

* Re-run of identical hardware and generate graphs
Fixes https://github.com/qdrant/fastembed/issues/174

* Change dataset source

* Refactor code for better readability and maintainability

* Inline outputs

* Replace hard coded constants with dataset specific n_dim

* fix: fix binary quant from scratch notebook

* fix: fix result table, explain corner case

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
 2024-06-06 20:59:27 +02:00
George Panchuk
bf4ef9d513 bump version to v0.3.0 v0.3.0 2024-06-05 18:03:25 +02:00
George
48c59dc7b3 new: update supported models (#253) 2024-06-05 17:50:07 +02:00
George
7099dec962 fix: remove outdated widgets state (#262) 2024-06-05 17:44:57 +02:00
George
a2660f8a3a new: add image embedding example notebook (#258) 2024-06-04 20:30:54 +02:00
George
e7d9abaee1 new: add a brief colbert example (#260) 2024-06-04 20:30:35 +02:00
George
01097708fe new: add gpu example, update readme (#256) 2024-06-04 20:30:23 +02:00
George
d725974fc4 new: update docs (#257) 2024-06-04 20:30:11 +02:00
George
14c067e15e new: unlock huggingface hub and ruff (#250) 2024-05-31 17:41:44 +02:00
George
c8fff66b18 Colbert (#248) 
* new: add late interaction embedding, colbert

* new: update imports

* new: add comments

* fix: rollback mp methods

* fix: restore existing padding after embed query

* fix: fix OnnxOutputContext in onnx embed, fix preprocessing for colbert
 2024-05-31 17:06:43 +02:00
Dmitrii Ogn
85aaae4c08 Add resnet (#246) 
* Resnet support added

* Tests fixed
Shapes matching for Resnet50-onnx
Example of Resnet50 to onnx conversion (basic)

* Removed optional conversion from PIL to np.ndarray and now it it's made default
Fixed test accordingly

* Refactoring of pil2ndarray

* Partial support of convnext preprocessing
Resize logic

* normalize canonical value

* Style changes for review

* new: update resnet repo

---------

Co-authored-by: d.rudenko <dimitriyrudenk@gmail.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
 2024-05-31 16:56:13 +02:00
Andrey Vasnetsov
dfd25d41c9 Attention sparse embeddings (#235) 
* WIP: sparse embeddings using attention

* support for stopwords

* apply stopwords

* proceed implementation of sparse attention embeddings (#234)

* complete inference

* query embed + comment

* use simpler weights formula instead of sorting of words

* update tests

* fix: fix bm42 usage, add query_embed to SparseTextEmbedding, update tests

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
 2024-05-24 15:25:34 +02:00
George
316c33634b new: add docstring with preprocessor keys (#245) 2024-05-23 13:10:23 +02:00
George
490c340a76 new: update tokenizers dep (#244) 2024-05-22 13:27:30 +02:00
George
ec8f06978f new: update readme for CUDA 12.x, add warning for version conflicts (#239) 
* new: update readme for CUDA 12.x, add warning about onnxruntime-gpu and cuda compatibility

* fix: change warning type

* new: update readme
 2024-05-14 21:43:03 +02:00
George
cbe00107ec chore: update bug-report (#232) 
Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-05-14 07:35:31 +05:30
George
99164c9050 Clip (#219) 
* wip: init image embeddings

* new: add clip

* new: fix clip text embedding

* fix: fix image parallel

* fix: add test images

* fix: add PIL

* fix: fix generics

* fix: fix sparse worker

* fix: fix image test path

* fix: replace models repo

* new: follow-up for onnx providers and local_files_only option

* fix: add types, refactor a bit

* refactoring: move onnxprovider type alias to types

* fix: fix type alias import
 2024-05-09 11:12:25 +02:00
Andrey Vasnetsov
6ecab6d40d bump v0.2.7 v0.2.7 2024-05-03 16:33:06 +00:00
George
d8c592032b new: allow users to override providers (#214) 
* new: add gpu support, allow users to override providers

* fix: update poetry.lock

* fix: fix type hint for 3.8

* [readme] Remove similar work

* [README] Add GPU support for FastEmbed library

* [README]  Add device check

* fix: revert changes to pyproject and lock, update readme

* Update poetry.lock

* new: add type alias for providers, add explicit providers to embeddings

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-05-03 18:31:22 +02:00
George
da603b8b7d new: add release instructions (#231) 
* new: add release instructions

* review fixes

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2024-05-03 18:30:54 +02:00
Andrey Vasnetsov
562d604375 Merge pull request #223 from Waffleboy/main 
[Bugfix] Allow user to pick local mode only so huggingface does not do a network call and timeout
 2024-05-03 18:29:54 +02:00
Andrey Vasnetsov
8b1a98a6a3 Merge pull request #230 from qdrant/update-tokenizers 
new: update tokenizers
 2024-05-03 18:26:09 +02:00
Andrey Vasnetsov
3c9b147e0a version 2024-05-03 16:21:33 +00:00
George Panchuk
6abd415f4a fix: add local_files_only to sparse, formatting, refactor 2024-05-03 17:49:10 +02:00
George Panchuk
8184acbb39 new: update tokenizers 2024-05-03 16:57:45 +02:00