upstream-archive/fastembed
1
0
Fork 0
mirror of https://github.com/qdrant/fastembed.git synced 2026-09-21 13:37:57 -05:00
Code Issues Packages Projects Releases Wiki Activity
575 Commits 38 Branches 59 Tags
800f3887b725f1d16c93036567c8e12b3a182f6e
Commit Graph
87 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
George 685fd9b5a1 new: use cuda if available (#537) 
* new: use cuda if available

* fix: fix warning msg

* fix: add missing import
 2025-12-10 20:23:34 +07:00
George b304a2aff0 new: drop python3.9, replace optional and union with | (#574) 
* new: drop python3.9, replace optional and union with |

* new: remove python 3.9 from pyproject

* refactor: replace remaining union and optional with |

* new: remove optional and union in dataclasses

* fix: add typealias to numpy type

* new: replace union with | in token count
 2025-12-10 19:01:01 +07:00
George c715416361 fix: update colbert description (#534) 2025-12-09 19:22:54 +07:00
George 428381cb04 bump version to v0.7.4 2025-12-05 18:38:45 +07:00
George 3511b08831 fix: numpy dropped 3.10 as of 2.3.0 (#585) 
* fix: numpy dropped 3.10 as of 2.3.0

* fix: update poetry lock
 2025-12-05 18:08:20 +07:00
George b718cc6a88 new: add token count method (#583) 
* new: add token count method

* fix: fix mypy

* fix: load model in token_count

* fix: remove debug code
 2025-12-04 11:29:36 +07:00
George 2ba8990260 new: try unlocking huggingface hub and pillow (#582) 
* new: try unlocking huggingface hub and pillow

* new: adjust pillow version for 3.9

* fix: fix pillow for python3.13
 2025-12-01 18:21:28 +07:00
George dab185fd9d fix: fix onnx version for python3.13 (#580) 2025-11-25 21:42:10 +07:00
George ec0e3128ee new: expose some onnx session options (#578) 
* new: expose some onnx session options

* fix: fix extra session options is None case

* fix: fix missing params

* new: add tests
 2025-11-25 17:49:02 +07:00
George 44e332999c new: try loading models from cache before making any network calls (#577) 2025-11-25 12:07:50 +07:00
George 533b54cee5 tests: introduce model cache to tests (#573) 
* tests: introduce model cache to tests

* fix: fix not cached model deletion

* new: do not run CI tests on mac os and windows on python 3.10-3.12

* fix: lowercase cache keys, bm25 caching

* tests: do not run parallel processing on all cpus in sparse text embed

* fix: fix models to cache names, do not run parallel=0

* fix: fix sparse embedding tests

* fix: bm42 language by lower case model name
 2025-11-12 18:54:33 +07:00
George Panchuk ba1f6053bd bump version to v0.7.3 2025-08-29 14:15:25 +03:00
George 4dc76e3859 fix: fix colbert query postprocessing (#557) 
* fix: fix colbert query postprocessing

* fix: improve colbert single embedding tests
 2025-08-29 13:25:43 +03:00
George 6efe06b172 new: decouple colbert query and document tokenizer (#556) 2025-08-29 13:25:18 +03:00
George Panchuk 887239239b bump version to v0.7.2 2025-08-25 16:29:23 +03:00
George acec31277b fix: fix mypy colpali (#533) 2025-06-16 11:55:29 +03:00
George Panchuk cb902149c8 bump version to 0.7.1 2025-06-16 12:23:35 +04:00
George a260022ae6 fix: propagate local files only and specific model path into embed parallel (#524) 2025-05-20 14:46:42 +04:00
George d5da56299a new: add embedding size property (#521) 
* new: add embedding size property

* fix: format exception message

* new: replace embedding size property with get_embedding_size classmethod

* fix: fix missed parts

* chore: fix docstrings

* new: add embedding_size property
 2025-05-20 14:46:32 +04:00
George 4e5575f4a7 fix: raise exception if pooling is incorrect in custom model (#522) 2025-05-19 13:49:22 +03:00
George 4736f46548 fix: check lowercase model name for warnings (#523) 2025-05-19 13:49:12 +03:00
George 0df605fc92 fix: remove onnxruntime cap as not needed anymore (#517) 
* fix: remove onnxruntime cap as not needed anymore

* fix: fix mypy complaints in python3.13
 2025-05-13 17:54:54 +04:00
George b785640bd5 fix: fix list of supported bm25 languages (#506) 2025-04-15 14:16:09 +03:00
George Panchuk b34209dcfb bump version to 0.6.1 2025-04-10 16:23:54 +03:00
George c91d42dda7 Update setting jina v3 tasks (#503) 
* new: improve task setter in jina v3

* refactor

* new: add hf_token secret

* fix: cross platform env propagation
 2025-04-10 16:19:41 +03:00
George aa0c475a1f fix: fix splade name (#499) 2025-03-17 11:07:14 +03:00
George 1729aab1ec new: preserve embeddings in a type set by their model (#492) 
* new: preserve embeddings in a type set by their model

* fix: remove type coercion

* fix: remove redundant type

* fix: fix random data type in tests
 2025-03-03 17:46:35 +01:00
George 42fca3b467 Deprecate archive struct (#490) 
* bump version to 0.6.0

* new: deprecate gcp archive structure
 2025-03-03 17:12:47 +01:00
George Panchuk 2082108baf bump version to 0.6.0 2025-02-26 13:55:26 +01:00
George 6cda2ce7f0 fix: fix batch embedding precision and shape (#488) 2025-02-26 13:51:20 +01:00
George 5bd5c0a0f0 fix: fix colpali preprocessing, add examples to readme (#487) 2025-02-26 12:51:13 +01:00
George 58ee7cc95c fix: fix thenlper, update warnings (#486) 2025-02-21 17:40:14 +01:00
George 27eeb39473 new: add custom models (#479) 
* fix: fix onnx text embedding list supported models, do not add already registered models, add tests

* fix: autouse fixture in custom model tests

* Refactor custom models (#482)

* refactor: refactor custom models

* fix: fix types

* remove commented out code
 2025-02-20 14:20:12 +01:00
George 4e527b1c63 new: allow mmh3<6.0.0 (#484) 2025-02-20 14:18:44 +01:00
George 6f43572373 fix: fix problem appeared in poetry 2.1.0 (#478) 2025-02-15 22:27:41 +01:00
George de4ecb48c7 new: add late interaction multimodal to package init (#471) 2025-02-12 18:10:07 +01:00
George b9d605138f new: add py.typed (#472) 2025-02-12 18:09:54 +01:00
Georgeandhh-space-invader 4599b933ca wip: type hints for colpali (#469) 
* wip: type hints for colpali

* new: Add colpali type hints

* refactor: Remove redundant type ignore

* fix: address remaining mypy issues

---------

Co-authored-by: hh-space-invader <h.hagag.ali@gmail.com>
 2025-02-06 20:28:10 +01:00
Georgeandd.rudenko 2fe33c5a62 colpali v1.3 by AndrewOgn (#427) 
* wip: design draft

* Operators fix

* Fix model inputs

* Import from fastembed.late_interaction_multimodal

* Fixed method misspelling

* Tests, which do not run in CI
Docstring improvements

* Fix tests

* Bump colpali to version v1.3

* Remove colpali v1.2

* Remove colpali v1.2 from tests

* partial fix of change requests:
descriptions
docs
black

* query_max_length

* black colpali

* Added comment for EMPTY_TEXT_PLACEHOLDER

* Review fixes

* Removed redundant VISUAL_PROMPT_PREFIX

* type fix + model info

* new: add specific model path to colpali

* fix: revert accidental renaming

* fix: remove max_length from encode_batch

* refactoring: remove redundant QUERY_MAX_LENGTH variable

* refactoring: remove redundant document marker token id

* fix: fix type hints, fix tests, handle single image path embed, rename model, update description

* license: add gemma to NOTICE

* fix: do not run colpali test in ci

* fix: fix colpali test

---------

Co-authored-by: d.rudenko <dmitrii.rudenko@qdrant.com>
 2025-02-06 16:23:06 +01:00
George Panchuk fa11d0f0c7 bump version to 0.5.1 2025-01-16 11:11:43 +01:00
George 50289c62ed new: move onnx dependency to dev (#439) 
* new: move onnx dependency to dev

* update poetry install, add more groups in pyproject
 2025-01-15 17:39:06 +01:00
George 516170cbaf new: add python 3.13 support (#404) 2024-12-11 19:18:54 +00:00
George 1343e55076 new: drop python 3.8 support, update type hints, ci (#403) 2024-11-15 15:54:19 +01:00
George faed7f1320 fix: fix model cleanup in tests (#400) 
* fix: fix model cleanup in tests
 2024-11-12 19:08:39 +01:00
George 5b895420b1 Update GitHub templates (#389) 
* new: add PR template, decouple bug template with new model request

* new: add feature request, model request
 2024-11-08 11:45:21 +01:00
GeorgeandAnush e868bbaebc new: add reranker example to readme (#390) 
* new: add reranker example to readme

* Update README.md

Co-authored-by: Anush  <anushshetty90@gmail.com>

* fix: compute the scores

---------

Co-authored-by: Anush <anushshetty90@gmail.com>
 2024-11-07 17:31:13 +01:00
George Panchuk a5f3f11829 bump version to v0.4.2 2024-11-07 12:03:34 +01:00
George Panchuk aba8fb43cf bump version to v0.4.1 2024-10-21 23:56:19 +04:00
George 97f2fb278e wip: remove model dir (#350) 
* wip: remove model dir

* fix: update pytest run

* wip: disable some tests not used atm

* wip: disable some tests in ci

* wip: add debug print

* fix: fix ci, remove models after usage

* fix: fix bm25 deletion

* fix: remove redundant ci commands
 2024-09-24 12:18:44 +02:00
George Panchuk 9445f95a32 bump version to v0.3.6 2024-08-23 21:15:43 +02:00
George ab9ab73278 Fix deprecated splade model (#333) 
* fix: return prithvida model to supported models

* fix: fix deprecation warning stacklevel
 2024-08-23 21:14:51 +02:00
George Panchuk 08925c9cfc bump version to v0.3.5 2024-08-23 19:52:42 +02:00
George e49789c129 fix: update push gpu command (#300) 2024-07-18 12:22:12 +03:00
George Panchuk fd116dd507 bump version to 0.3.4 2024-07-15 15:59:52 +03:00
George 0315c3b8c6 new: add modifier flag into bm models config (#299) 
* new: add modifier flag into bm models config

* refactoring: rename modifier field
 2024-07-15 15:57:21 +03:00
George 3a8985b35c new: add retry logic for model downloading (#293) 
* new: add retry logic for model downloading

* fix: add sleep
 2024-07-10 20:09:11 +03:00
George e071c84f22 fix: fix hybrid search example for pydantic v1 (#263) 2024-06-14 17:25:15 +02:00
George e1ecfe9c2f fix: fix None cache dir in parallel mode (#277) 2024-06-14 17:19:53 +02:00
George 5461012ab1 new: add bm25, fix param propagation in parallel mode, fix bm42 parallel (#274) 
* new: add bm25, fix param propagation in parallel mode, fix bm42 parallel

* refactoring: remove redundant example

* fix: fix mp start method in bm25

* refactoring: refactor token id generation

* new: replace model repository
 2024-06-13 19:52:48 +02:00
George Panchuk bf4ef9d513 bump version to v0.3.0 2024-06-05 18:03:25 +02:00
George 48c59dc7b3 new: update supported models (#253) 2024-06-05 17:50:07 +02:00
George 7099dec962 fix: remove outdated widgets state (#262) 2024-06-05 17:44:57 +02:00
George a2660f8a3a new: add image embedding example notebook (#258) 2024-06-04 20:30:54 +02:00
George e7d9abaee1 new: add a brief colbert example (#260) 2024-06-04 20:30:35 +02:00
George 01097708fe new: add gpu example, update readme (#256) 2024-06-04 20:30:23 +02:00
George d725974fc4 new: update docs (#257) 2024-06-04 20:30:11 +02:00
George 14c067e15e new: unlock huggingface hub and ruff (#250) 2024-05-31 17:41:44 +02:00
George c8fff66b18 Colbert (#248) 
* new: add late interaction embedding, colbert

* new: update imports

* new: add comments

* fix: rollback mp methods

* fix: restore existing padding after embed query

* fix: fix OnnxOutputContext in onnx embed, fix preprocessing for colbert
 2024-05-31 17:06:43 +02:00
George 316c33634b new: add docstring with preprocessor keys (#245) 2024-05-23 13:10:23 +02:00
George 490c340a76 new: update tokenizers dep (#244) 2024-05-22 13:27:30 +02:00
George ec8f06978f new: update readme for CUDA 12.x, add warning for version conflicts (#239) 
* new: update readme for CUDA 12.x, add warning about onnxruntime-gpu and cuda compatibility

* fix: change warning type

* new: update readme
 2024-05-14 21:43:03 +02:00
GeorgeandNirant cbe00107ec chore: update bug-report (#232) 
Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-05-14 07:35:31 +05:30
George 99164c9050 Clip (#219) 
* wip: init image embeddings

* new: add clip

* new: fix clip text embedding

* fix: fix image parallel

* fix: add test images

* fix: add PIL

* fix: fix generics

* fix: fix sparse worker

* fix: fix image test path

* fix: replace models repo

* new: follow-up for onnx providers and local_files_only option

* fix: add types, refactor a bit

* refactoring: move onnxprovider type alias to types

* fix: fix type alias import
 2024-05-09 11:12:25 +02:00
GeorgeNirant KasliwalNirant
 		d8c592032b new: allow users to override providers (#214) 
* new: add gpu support, allow users to override providers

* fix: update poetry.lock

* fix: fix type hint for 3.8

* [readme] Remove similar work

* [README] Add GPU support for FastEmbed library

* [README]  Add device check

* fix: revert changes to pyproject and lock, update readme

* Update poetry.lock

* new: add type alias for providers, add explicit providers to embeddings

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-05-03 18:31:22 +02:00
GeorgeandAndrey Vasnetsov da603b8b7d new: add release instructions (#231) 
* new: add release instructions

* review fixes

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2024-05-03 18:30:54 +02:00
George Panchuk 6abd415f4a fix: add local_files_only to sparse, formatting, refactor 2024-05-03 17:49:10 +02:00
George Panchuk 8184acbb39 new: update tokenizers 2024-05-03 16:57:45 +02:00
George 47cf7f9f92 new: add gpu package into workflow (#228) 
* new: add gpu package into workflow

* remove gpu tag
 2024-05-03 16:30:23 +02:00
George 5cde2898bc new: remove slurm environment variables (#213) 2024-04-26 21:45:02 +02:00
George f340a73a3e refactoring: update binary quantization notebook (#180) 2024-04-02 14:45:27 +05:30
George 912e95e5c8 fix: remove model archive if model extraction was not finished correctly (#179) 2024-04-01 20:11:32 +02:00
GeorgeandNirant Kasliwal e11f66bb55 refactoring: update imports in notebooks (#173) 
* new: simplify imports

* refactoring: update import

* refactoring: update imports in notebooks

* fix: fix notebook output

* Re-run notebook with revised imports

---------

Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-04-01 17:02:13 +05:30
GeorgeNirantNirant Kasliwal
 		8b7b8476a5 fix: fix model sizes in supported models lists (#167) 
* fix: fix model sizes in supported models lists

* fix: remove redundant comment

* fix: fix test

* fix: update supported models notebook

* Consistentcy around quantization in supported_onnx_models

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
Co-authored-by: Nirant Kasliwal <nirant.bits@gmail.com>
 2024-04-01 16:18:49 +05:30
George ae35a96bcb new: simplify imports (#171) 
* new: simplify imports

* refactoring: update import
 2024-04-01 12:58:26 +05:30
GeorgeandNirant 25671ec349 Update ruff (#172) 
* refactoring: reduce max line-length

* new: update ruff

---------

Co-authored-by: Nirant <NirantK@users.noreply.github.com>
 2024-04-01 12:55:15 +05:30
George ce98631b9a Fix spladepp parallelism (#169) 
* fix: add get_worker_class implementation to spladepp

* fix: add tests for parallel embed for spladepp
 2024-04-01 10:22:24 +05:30
George 0a4ed42b58 fix: unify existing patterns, remove redundant (#168) 2024-03-30 14:40:19 +05:30