Commit Graph

3 Commits

Author SHA1 Message Date
Hossam Hagag
993dcd5f68 chore: Add missing type hints in functions (#453)
* chore: Add missing type hints in functions

* add missing import, small type refactor

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-01-29 12:08:20 +01:00
Hossam Hagag
73e1e5ecb9 chore: Add missing returns in defs (#451)
* chore: Add missing returns in defs

* remove return type from init

* remove incorrect ndarray specifier

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-01-29 08:33:49 +01:00
Hossam Hagag
eaecf7d471 Multi gpu support (#358)
* feat: Added multi gpu support for text embedding

* feat: Add support for multi-gpu for special text models

* fix: Fix lazy_load to load the model to child processes when parallel is not none

* feat: Added lazy_load and multi-gpu to colbert

* feat: Add lazy_load and multi gpu to image models

* feat: Support lazy_load and multi-gpu to sparse models (except BM25)

* fix: Fixed BM25 not working

* refactor: Remove redundant GPUParallelProcessor

* refactor: Refactor _embed_*_parallel

* feat: Add cuda argument
refactor: Refactor how worker assign device

* fix: Fix if providers and cuda are None

* fix: Fix providers and cuda are none

* WIP: Multi gpu support review (#361)

* WIP: review

* wip: review

* refactor: refactor images

* refactor: refactor sparse

* refactor: refactor late interaction

* add model loading

* add tests

* fix: uncomment models in tests

* fix: fix variable declaration order

* fix: fix device id assignment

* tests: add multi gpu tests

* fix: fix device id assignment for sparse embeddings

* tests: update multi gpu tests

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* refactor: remove redundant declarations

* fix: rollback redundant changes

* fix: remove num workers device ids dep, fix type hint

* fix: fix post process for sparse models

* fix: remove redundant model loading

* new: add lazy load and new gpu support to cross encoders

* fix: add rerankers to multi gpu tests

* fix: unlock multilingual test

* fix: fix gpu test with cross encoder

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-10-16 23:42:38 +02:00