4 Commits

Author SHA1 Message Date
George
2fe33c5a62 colpali v1.3 by AndrewOgn (#427)
* wip: design draft

* Operators fix

* Fix model inputs

* Import from fastembed.late_interaction_multimodal

* Fixed method misspelling

* Tests, which do not run in CI
Docstring improvements

* Fix tests

* Bump colpali to version v1.3

* Remove colpali v1.2

* Remove colpali v1.2 from tests

* partial fix of change requests:
descriptions
docs
black

* query_max_length

* black colpali

* Added comment for EMPTY_TEXT_PLACEHOLDER

* Review fixes

* Removed redundant VISUAL_PROMPT_PREFIX

* type fix + model info

* new: add specific model path to colpali

* fix: revert accidental renaming

* fix: remove max_length from encode_batch

* refactoring: remove redundant QUERY_MAX_LENGTH variable

* refactoring: remove redundant document marker token id

* fix: fix type hints, fix tests, handle single image path embed, rename model, update description

* license: add gemma to NOTICE

* fix: do not run colpali test in ci

* fix: fix colpali test

---------

Co-authored-by: d.rudenko <dmitrii.rudenko@qdrant.com>
2025-02-06 16:23:06 +01:00
Hossam Hagag
b05877de93 new: Added jina embedding v3 (#428)
* new: Added jina embedding v3

* refactor: Changed dim to int value

* new: Updated notice

* new: Extended text embedding with query embed and passage embed

* fix: Fix lazy load in query and passage embed

* tests: Added test for multitask embeddings

* nit: Remove cache dir from tests

* tests: Updated tests

* improve: Improve task selection

* fix: Fix ci

* fix: Update fastembed/text/multitask_embedding.py

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update fastembed/text/multitask_embedding.py

Co-authored-by: George <george.panchuk@qdrant.tech>

* fix: Pass task id using kwargs to parallel processor

* tests: Added test for task assignment

* prefer enums over ints

* tests: Added test for parallel

* improve: Updated model description

* fix: Fix ci

* fix: Fix ci

* refactor: Refactor query_embed and passage_embed

* tests: Added task propagation to parallel

* refactor: Set default task as retrieval passage

* chore: Update default task in tests

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2025-01-27 23:27:23 +01:00
Hossam Hagag
860e2ad691 Tsk 375 add jina rerankers (#379)
* feat: Added jina reranker models

* chore: Added jina reranker canonical score values

* chore: added rounding of the output for easier reproducability

* chore: Added jina reranker models in batch test

* chore: remove redundant np.round

* chore: test only <1gb files in local

* chore: Updated docs to add rerankers

* fix: recompute canonical values with fp16

* new: extend NOTICE with jina reranker v2

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-11-13 10:48:45 +01:00
Hossam Hagag
3fc0e2b382 chore: Add notice file for jina ai models (#380)
* chore: Add notice file for jina ai models

* chore: Update notice

* Update NOTICE

Co-authored-by: George <george.panchuk@qdrant.tech>

* chore: added jina embeddings v3

* chore: removed unsupported models

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-11-05 14:22:03 +02:00