Commit Graph

10 Commits

Author SHA1 Message Date
George
b4ae703d1b Local inference upload collection and upload records (#862)
* wip: add draft implementation of batch processing

* fix: embed dict and list of docs, remove redundant code

* new: regen async, small refactor

* refactor: add docstrings, rename methods

* Upload points local inference (#881)

* new: separate single and plural model embeddings

* fix: fix lazy embed models

* new: add inference object inspections to upload methods

* wip: local inference upload parallel

* new: add local inference to upload points and upload collection, refactor mixin

* fix: remove redundant code

* redundant import

* tests: check is query for query points batch

* refactor: refactor semi ordered map

* tests: add test for local inference with batches with docs and vectors

* tests: check the order of dict processing

* new: distinguish models by options

* fix: fix typing

* fix: fix types

* new: embed batches with different options

* tests: add tests for batch with different options

* fix: ignore ide incorrect type inspection

* tests: wait for points to be inserted

* fix: set threads to 1 in parallel inference

* new: adjust max internal batch size

* fix: fix type hints

* function to get embeddings size (#892)

* function to get embeddings size

* async client

* keep sync

* new: extend embedding size to support image and late interaction models

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* new: add local inference batch size (#894)

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-01-29 15:53:24 +01:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
Hossam Hagag
ff51482402 fix: Clean up resources when qdrant not responding while uploading to elevate deadlock (#840) 2024-11-07 12:39:36 +01:00
George
bdae392066 new: add pre-commit (#126)
* new: add pre-commit

* new: exclude auto-generated files

* new: increase black line length

* fix: fix exclude pattern

* remove default version

* apply black

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-03-01 21:19:46 +04:00
George
d35a2c21da Ci type checkers 2 (#104)
* fix: update missing return types and other mypy complaints, add mypy and pyright

* new: add mypy and pyright to ci, update type hints

* fix: add types group in pyproject.toml, add types-protobuf

* fix: fix typo

* fix: fix return types in api
2023-02-09 15:27:39 +04:00
George
3e68b3786f new: add py.typed, validate code with mypy (#91) 2023-01-21 16:32:32 +04:00
Andrey Vasnetsov
98f87f45f3 lower dependency requirements (#74) 2022-09-26 23:03:32 +02:00
Andrey Vasnetsov
8a46249dae add grpc to get collection method (#48)
* add grpc to get collection method

* fix: change process start method for grpc

* new: use forkserver instead of spawn if available #48

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2022-07-04 19:18:21 +02:00
Andrey Vasnetsov
0b499b0fd5 Upd version - 0.7.0 (#26)
* upd version

* enable CI on all PRs

* update client - json as a payload + typed index (#31)

* update client - json as a payload + typed index

* refactor

* upd docker version in tests

* New condition filters (#33)

* upd rest for new filter conditions

* update gprc client

* proper shutdown on upload error

* rm debug print
2022-04-13 09:49:31 +02:00
Andrey Vasnetsov
6934c98454 add parallel upload #1 2021-05-30 23:00:39 +02:00