Commit Graph

30 Commits

Author SHA1 Message Date
George Panchuk
698db1d33f new: drop python3.9, replace union and optional with | where possible 2025-11-14 03:20:36 +07:00
George
ff7f584d33 new: update models, remove init_from and locks (#1100)
* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-11 21:17:08 +07:00
George
cee3770f63 Update filter (#1090)
* new: add missing update_filter, implement it in local mode

* fix: fix type hint, fix update operation, fix rest uploader, add tests

* fix: fix update filter is None case

* fix: mypy was not a good boy
2025-10-31 18:07:29 +07:00
tellet-q
635457b47c new: Support for too many requests (#914)
* Update grpc client

* Update rest client and add new exception

* Ensure uploaders continue after wait

* Update

* Address coderabitai review

* Fix tests

* Re-generate clients

* Address review

* Revert attempt handling

* Add new exception

* Add tests

* Do not throw ResourceQuotaExceeded

* Add support for async grpc

* Remove iscoroutine check and more

* update models

* Address some coderabbitai comments

* Address review

* Fix test

* Fix backwards compatibility tests

* Address review

* Get rid of redundant code

* Address ai review

* refactoring: refactor tests

* refactoring: do not inherit resource exhausted from aiorpcerror

* fix: remove redundant import

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-03-24 18:05:15 +03:00
George
b4ae703d1b Local inference upload collection and upload records (#862)
* wip: add draft implementation of batch processing

* fix: embed dict and list of docs, remove redundant code

* new: regen async, small refactor

* refactor: add docstrings, rename methods

* Upload points local inference (#881)

* new: separate single and plural model embeddings

* fix: fix lazy embed models

* new: add inference object inspections to upload methods

* wip: local inference upload parallel

* new: add local inference to upload points and upload collection, refactor mixin

* fix: remove redundant code

* redundant import

* tests: check is query for query points batch

* refactor: refactor semi ordered map

* tests: add test for local inference with batches with docs and vectors

* tests: check the order of dict processing

* new: distinguish models by options

* fix: fix typing

* fix: fix types

* new: embed batches with different options

* tests: add tests for batch with different options

* fix: ignore ide incorrect type inspection

* tests: wait for points to be inserted

* fix: set threads to 1 in parallel inference

* new: adjust max internal batch size

* fix: fix type hints

* function to get embeddings size (#892)

* function to get embeddings size

* async client

* keep sync

* new: extend embedding size to support image and late interaction models

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* new: add local inference batch size (#894)

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-01-29 15:53:24 +01:00
Hossam Hagag
47ff758c56 refactor: Refactored the way of warnings (#864)
* refactor: Refactored the way of warnings

* remove unused imports

* fix: Fix stacklevel in warnings

* regenerated async

* nit

* Updated some warnings with show once

* fix: Fix stack levels

* Updated async

* Update qdrant_client/qdrant_remote.py

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update async client

* Updated warnings

* Updated warnings

* Updated warnings

* fix: fix warning level

* fix: fix warning level in async

* fix: revert append payload condition

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2025-01-06 16:44:06 +01:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
Hossam Hagag
d6d25ac33a Fix grpc upload retry timeout (#841)
* improve: Added a default timeout in grpc if the connection is dead

* improve: Added 5 seconds wait before retry

* fix: Fix grpc timeout

* nit

* fix: Fix bug in timeouts
new: Add timeouts to delete and upsert

* new: Propagate timeout to grpc default options
new: Added missing timeouts

* nit

* refactor: Changed how we define the default grpc timeout

* remove redundant import

* fix: add missing timeouts

* fix: Added missing timeout in rest

* fix: Added async

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2024-11-15 13:28:29 +01:00
Drew
6d1fcdea43 Break out of retry loop upon successful upsert (#531) 2024-03-11 15:38:07 +01:00
George
4adbabec7e fix: fix implicit ids in upload collection with paralell > 1 (#460)
* fix: fix implicit ids in upload collection with paralell > 1

* fix: fix type hints

* fix: remove redundant code, simplify type hints

* fix: remove redundant import

* fix: fix batching

* fix: replace generator with list comprehension

* fix: sorry, I was wrong

* tests: update tests

* fix: extend upload records and upload points tests
2024-01-31 16:38:07 +01:00
George
a2adc2d2ad new: deprecate upload records, update tests, prohibit migration of co… (#447)
* new: deprecate upload records, update tests, prohibit migration of collections with custom shards

* Update qdrant_client/qdrant_client.py

* Update qdrant_client/async_qdrant_client.py

* new: replace autogenerated int ids with uuids (#448)

* fix: remove redundant kwargs

* fix: regen async

* fix: remove redundant import

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2024-01-19 20:11:31 +01:00
George
3e7c743e06 new: add shard keys to upload_collection and upload_records (#396)
* new: add shard keys to upload_collection and upload_records

* fix: update async client, update type hint

* new: remove split by batch (#422)

* new: remove split by batch

* fix: fix type hints, remove redundant arg

* fix: fix mypy

* new: ignore shard keys from records

* new: add warning if shard key is set in records in upload_records
2024-01-05 20:55:26 +01:00
Andrey Vasnetsov
aed7ecaefb sync API (#280)
* sync API

* fix tests

* backward compatibility version up
2023-09-07 11:29:10 +02:00
Nirant
536b0eeb22 Add fastembed integration (#210)
* Add skeleton for fastvector registration

* * fix(qdrant_client.py): fix import statement for QdrantClientMixin from fastembed instead of fastvector
* fix(qdrant_client.py): remove unnecessary print statement

* * chore(README.md): add bullet point for extensive test coverage
* feat(README.md): add section for Fast Embeddings + Simpler API
* fix(README.md): fix formatting of code block and update code example for Qdrant Client usage
*

* * fix(qdrant_client.py): remove unused imports and code
* feat(qdrant_client.py): add support for adding and querying documents with fastembed installed
* test(qdrant_client.py): add tests for adding and querying documents with and without fastembed installed

* * feat(qdrant_client.py): add support for fastembed.qdrant_mixin.QdrantAPIExtensions for upsert_docs and search_docs methods

* * fix(qdrant_client.py): remove unused import of SearchParams
* feat(qdrant_client.py): add return type hint to QdrantClient.search_docs method

* * fix(qdrant_client.py): fix import statement for SearchParams class in qdrant_client.py

* * feat(models.py): add QueryResponse model for handling query responses

* * feat(qdrant_client.py): add batch_iterable method to QdrantClient class

* * chore(qdrant_client.py): reformat code for better readability

* * chore(test_qdrant_client.py): add type hinting to fastembed_setup fixture
* test(test_qdrant_client.py): add test for client_close function

* * chore(.gitignore): add pattern for ignoring .tar.gz files

* * refactor(test_qdrant_client.py): remove unused imports and variables
* chore(test_qdrant_client.py): reformat import statements for better readability

* * chore(test_qdrant_client.py): remove unused fixtures and test cases

* * chore(qdrant_client.py): remove unused imports

* Add test skeleton

* * chore(.gitignore): add local_cache/*/* to ignore local cache files

* * fix(qdrant_client.py): change import statement for fastembed to use DefaultEmbedding instead of FlagEmbedding
* refactor(qdrant_client.py): refactor code to remove unnecessary loop

* * fix(qdrant_client.py): import missing models in qdrant_client.py
* feat(qdrant_client.py): add support for search parameters in search method
* refactor(qdrant_client.py): refactor indexing logic to handle embeddings correctly

* * test(test_fast_embed.py): add assertion to verify count of documents in collection
* test(test_fast_embed.py): remove unused code
* test(test_fast_embed.py): add TODO comment for future assertions

* * chore(pyproject.toml): add optional fastembed group to poetry dependencies
* feat(pyproject.toml): add fastembed dependency to fastembed group

* * chore(qdrant_client.py): add check for fastembed installation in QdrantClient constructor

* * fix(test_fast_embed.py): skip test if FastEmbed is not installed
* fix(test_fast_embed.py): add default values for test_no_install parameters
* fix(test_fast_embed.py): skip test if FastEmbed is installed

* Delete unused usage file

* * chore(models.py): remove unused QueryResponse class
* feat(qdrant_client.py): add QueryResponse class

* * refactor(qdrant_client.py): organize imports and fix type hinting for QueryResponse class

* review fixes

* make mypy happy

* review fixes

* fix mypy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2023-08-18 15:17:46 +02:00
George
ea412c4959 Fix congruence upload (#183)
* fix: fix congruence upload collection list of np.array, fix type hints

* fix: update type hints, remove broken code

* tests: delete collection after test just in case

* fix: return Dict[str, NumpyArray], support it in local mode

* fix: update type
2023-07-08 15:40:36 +04:00
George
f8c7b30282 fix: fix retry mechanism (#184)
* fix: fix retry mechanism

* fix: fix mypy
2023-06-16 15:11:23 +04:00
Andrey Vasnetsov
329b4150cf v1.2.0 (#173)
* wip: interface for new APIs

* upd docker version

* remove duplicated comments + extent interface

* start implementing remote client + remove duplicated comments there

* implement remote methods

* quantization converter

* extend coverege

* wip: saving and loading of optional vectors

* update and delete vectors test

* rm unused imports

* nested filters

* fix mypy

* fix pyright

* wip: add search groups and recommend groups (#174)

* wip: add search groups and recommend groups

* fix: fix signature

* fix: fix mypy

* simplify group-by and condition checks

* fix tests

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>

* add match except condition + improve group-by tests

* fix: lock typing exchanges due to broken release

* tests: add maybe and maybe_null checks to group tests

* new: add new methods to type stubs

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2023-05-24 11:06:10 +02:00
George
bdae392066 new: add pre-commit (#126)
* new: add pre-commit

* new: exclude auto-generated files

* new: increase black line length

* fix: fix exclude pattern

* remove default version

* apply black

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2023-03-01 21:19:46 +04:00
M. Yusuf Sarıgöz
017fa6ad23 Enable named vectors uploading in .upload_collection() and retry logic (#121)
* Enable named vectors uploading in `.upload_collection()`

* Implement retry logic in initial upload

* Fix: Fix return type

* Fix: fix mypy complaints
2023-03-01 13:36:22 +03:00
George
d35a2c21da Ci type checkers 2 (#104)
* fix: update missing return types and other mypy complaints, add mypy and pyright

* new: add mypy and pyright to ci, update type hints

* fix: add types group in pyproject.toml, add types-protobuf

* fix: fix typo

* fix: fix return types in api
2023-02-09 15:27:39 +04:00
George
3e68b3786f new: add py.typed, validate code with mypy (#91) 2023-01-21 16:32:32 +04:00
Arnaud Gourlay
ddd24214b8 v0.10.0 (#71)
* generate gRPC client

* generate REST client

* draft

* wip

* fix conversion test

* old api compatible

* clean deprecated

* batch search + reco tests

* multi-vector test

* full-text filtering

* Fast protobuf (#67)

* generate grpc client

use new grpc client in qdrant methods

fix grpc parallel issue

remove betterproto

remove betterproto

remove betterproto type annotation

extend test coverage

upd protobuf to master

* rm eventloop

* upd version

* shortcut for models

* v0.10.1 compatibility hotfix: make deprecated ram_data_size optional

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-09-19 12:59:29 +02:00
Andrey Vasnetsov
bb4d97a0b4 record-based collection uploader 2022-08-25 12:51:37 +02:00
Arnaud Gourlay
248756613c [WIP] propagate client kwargs to batchers (#64)
* propagate client kwargs to batchers

* pass connection params to batch upload + fix nested asyncio for jupyter

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2022-08-25 00:07:29 +02:00
Andrey Vasnetsov
7d78688695 allow iterable vectors on upload + fix async loop warning 2022-08-22 15:16:33 +02:00
Andrey Vasnetsov
84d1114751 support https in collection uploader 2022-07-10 13:18:14 +02:00
Andrey Vasnetsov
7204275931 gRPC search (#41)
* allow using grpc for search

* add docs

* assert + docs fix
2022-05-23 14:12:17 +02:00
Andrey Vasnetsov
0b04f9f3b1 automatic validation scaffold (#38)
* automatic validation scaffold

* grpc -> http for filter

* More conversions

* More conversions

* add all fixtures (#39)

* add rest-to-grpc convertions (#40)

* More conversions

* full coverage for conversions

* fix imports

* add exec permissions

Co-authored-by: M. Yusuf Sarıgöz <yusufsarigoz@gmail.com>
2022-05-22 11:20:55 +02:00
Andrey Vasnetsov
0b499b0fd5 Upd version - 0.7.0 (#26)
* upd version

* enable CI on all PRs

* update client - json as a payload + typed index (#31)

* update client - json as a payload + typed index

* refactor

* upd docker version in tests

* New condition filters (#33)

* upd rest for new filter conditions

* update gprc client

* proper shutdown on upload error

* rm debug print
2022-04-13 09:49:31 +02:00
Andrey Vasnetsov
621e885968 upload collection with gPRC (#28)
* grpc client generator

* grpc: upload data with grpc

* integration test

* allow-prerelease

* upd poetry

* fix review
2022-03-09 14:10:32 +01:00