Commit Graph

272 Commits

Author SHA1 Message Date
George
fddf1185ae new: expose the remaining cluster methods (#1123)
* new: expose the remaining cluster methods

* fix: fix type hints

* new: add deadline info to the first hw

* fix: fix review comments
2025-11-24 20:02:44 +07:00
George
9ce7346e05 new: convert cluster operations to grpc (#1116)
* new: convert cluster operations to grpc

* fix: rollback coverage html

* fix: address ai comments

* fix: address review comments
2025-11-20 13:38:17 +07:00
George
f2e0e22a17 new: expose cluster operations (#1114)
* new: expose cluster operations

* fix: fix cluster collection update return result, add tests

* fix test

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-14 20:20:22 +01:00
George
27acfd0d75 new: remove vectors_count, update http and grpc models (#1069)
* new: remove vectors_count, update http and grpc models

* fix: update inspection cache

* new: add conversions and update interface

* fix: fix some conversions

* fix: fix typo

* fix: fix isinstance

* fix: regen async

* fix: fix update_filter usage, fix isinstance

* tests: collection metadata test

* fix: address backward compatibility in test

* new: update models, add max payload index count and copy vectors

* fix; update _inspection_cache

* new: add read consistency to count points

* Allow uuids in interface (#1085)

* new: direct uuid support

* tests: add uuid tests

* fix: update inspection cache

* new: add collection metadata and tests to local mode (#1089)

* new: add collection metadata and tests to local mode

* fix: regen async client

* new: implement parametrized rrf in local mode (#1087)

* new: implement parametrized rrf in local mode

* refactoring: use a variable for a magic value

* fix: adjust conversion according to AI

* Update filter (#1090)

* new: add missing update_filter, implement it in local mode

* fix: fix type hint, fix update operation, fix rest uploader, add tests

* fix: fix update filter is None case

* fix: mypy was not a good boy

* Text any filter (#1091)

* new: add match text any local mode

* tests: add match text any tests

* new: update models, remove init_from and locks (#1100)

* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: yet another update

* new: add initial_state to create shard key (#1109)

* chore: remove obsolete imports

* fix: add metadata parameter to recreate collection in local

* fix: fix metadata handling in local more

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-14 17:01:17 +07:00
George
6b031b3262 fix: fix overriding prefetch limit in local mode (#1081)
* fix: fix overriding prefetch limit in local mode

* fix: replace recursion with a while loop in prefetch
2025-10-27 18:14:55 +07:00
George
784b27ab1c new: queryless score 1 (#1088)
* new: queryless score 1

* Reduce hf requests (#1092)

* tests: remove redundant test

* new: improve caching, reduce number of hf requests

* fix: fix possible cache_dir key error
2025-10-24 18:48:13 +07:00
George
200020f19a wip: builtin bm25 support (#1060)
* wip: builtin bm25 support

* remove debug print

* new: add builtin embedder, add server version check for inference

* fix: regen async

* fix: fix fastembed import

* fix: regen async

* tests: add tests for builtin inference

* fix: fix fastembed checks, improve exceptions
2025-09-15 16:18:41 +08:00
George
3d29466823 tests: extend remote inference tests (#1055) 2025-07-31 21:00:32 +03:00
George
e4c7bfaf4f fix: fix grpc document, image, inference object without options conversion (#1052) 2025-07-31 20:41:32 +03:00
Luis Cossío
1a63b86e51 better FieldCondition conversion (#1051) 2025-07-29 11:17:56 -04:00
Luis Cossío
7b17db5a7e MMR integration and local mode (#1044)
* generate rest client

* gen grpc client

* conversions

* inpection cache

* wip implement mmr

* upd openapi models

* tests: add mmr tests

* wip: implement mmr

* new: use postprocessed scores

* fix maxsim asymmetry and empty sparse intersection

* fix: fix conversions

* fix: fix type hints

* chore: update inspection cache

* fix: revert changes to add points and update named vectors

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-07-18 12:06:50 +03:00
George
dbf7f14949 fix uploading snapshot in ci (#1043)
* debug

* debug

* omg mypy

* fix: remove trap around pytest

* wip

* remove debug code
2025-07-15 18:43:53 +03:00
George
e823402cf5 new: add b64 encoding function (#1031)
* new: add b64 encoding function

* tests: remove garbage

* refactor base64 function name and params

* manual test for remove iamge inference with base64

* use tempfile

* fix: rename remove_inference->remote_inference

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-15 18:41:58 +03:00
George
4da06b08d6 new: update models for v1.15, add conversions (#1038)
* new: update models for v1.15, add conversions

* rename SnowballParameters to SnowballParams

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-07-15 15:39:48 +02:00
George
c1df3a7c28 tests: relax test comparison for distance matrix test (#1042) 2025-07-15 15:39:37 +02:00
George
fbf72f4a31 new: relax test comparison (#1039)
* new: relax test comparison

* rollback: rollback is context search addition

* rollback: return redundant parentheses

* Sparse no overlap distance computation (#1040)

* fix: fix sparse distance computation for recommend, discovery and context queries

* fix: use empty_is_zero

* add zero case to score cmp

* add snapshot saving on failure

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-15 13:30:23 +02:00
George
bb7016fd4b fix: fix edge case in upsert, fix multivec update, add tests (#1030)
* fix: fix edge case in upsert, fix multivec update, add tests

* fix: fix tests
2025-07-10 13:22:52 +02:00
Tim Visée
bf515ae2d7 Change user agent to python-client (#1028) 2025-06-24 14:07:07 +02:00
George
808b0abb2d Fix urljoin (#996)
* fix: fix urljoin usage

* fix: do not modify rest_uri, add tests
2025-05-21 14:46:10 +04:00
George
42856424a6 new: move fastembed imports and constants into a class to support cus… (#994)
* new: move fastembed imports and constants into a class to support custom models and improve exceptions

* new: add is_supported model to FastEmbedMisc

* fix: fix types
2025-05-20 14:46:18 +04:00
George
8632844582 fix: init_options should not crush with unpickleable objects (#993)
* fix: init_options should not crush with unpickleable objects

* fix: regen async

* fix: do not deepcopy kwargs

* Update tests/test_qdrant_client.py

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-05-20 14:45:54 +04:00
George
e32daed967 fix: fix set payload in batch update points, fix operation conversion… (#977)
* fix: fix set payload in batch update points, fix operation conversions, add tests

* new: add some checks

* add comments to test cases

* fix: return comment

* tests: uncomment test due to a fix in core
2025-05-15 19:09:13 +03:00
George
7bce5f0b75 tests: catch math domain error, catch abrupted error message in remote (#989) 2025-05-14 20:08:25 +03:00
George
4cac634cb8 fix: fix local inference on list of prefetches (#965) 2025-04-24 17:28:17 +03:00
George
351b17ee63 fix: set default limit in query batch points in local mode (#967) 2025-04-24 17:16:55 +03:00
Luis Cossío
337c70173e dedup order_by by value and id (#959)
* dedup order_by by value and id

* fix: fill order_value when order_by is used

* tests: add tests for duplicated values, fix payload value comparison

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-24 16:53:02 +03:00
George
5afdab1c3c fix: fix bm25 search on empty collection (#966) 2025-04-24 16:52:42 +03:00
George
ee2f5f451f Rethink offset prefetch (#955)
* fix: fix local mode offset in sample random

* fix: fix random sampling, do not propagate offset into prefetches

* fix: ignore offset in random sampling
2025-04-22 16:53:58 +03:00
George
6544484a0c refactor: refactor test local inference (#952) 2025-04-17 11:43:42 +03:00
George
a0bbadaa08 new: support multimodal embeddings (#908)
* new: support multimodal embeddings

* fix: fix embed when model name is present in both text and image supported models

* fix: fix type checkers

* refactor: split multimodal embed into embed text and embed image

* chore: comment batch separation

* chore: rollback formatting
2025-04-15 18:11:33 +03:00
George
2afadf4cb5 fix: non-list conditions in filter clauses (#949)
* fix: non-list conditions in filter clauses

* r u happy mypy?
2025-04-15 16:43:22 +03:00
George
5c2f3e883c Remove multitask models list (#945)
* new: remove multitask embedding list

* new: update fastembed
2025-04-15 16:43:06 +03:00
Ed Jinks
162a199a52 Fix/add geo polygon support for grpc (#950)
* add geo point to grpc conversion, add rpc recovery, add test

* rm whitespace

* add whitespace

* run pre-commit

* update polygon in test, add missing interiors conversion

* fix: nit fix in geo polygon conversoin, update conversion tests

---------

Co-authored-by: ed <ed@extrac.ai>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-15 11:07:25 +03:00
George
5a454a7402 Recommend sum score (#940)
* new: add sum_scores strategy, fix lookup from in query_batch_points, fix euclid between multivectors local mode

* new: recommend average multivectors, fix tests, add new tests for multivectors

* fix: address review comments

* fix: regen async

* tests: uncomment tests

* new: update poetry lock, require pydantic >=2.11 for populate inspection cache

* fix: tmp disable euclidean multivec search tests, fix multivec with nan test

* fix: align euclid and manhattan with core, add tests

* fix: fix type hints

* tests: use atol in comparison

* tests: remove manhattan tests

* tests: reduce number of multivectors

* remove float comparison relaxation

* explain why it is different for euclid and manhattan

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-04-07 11:45:33 +03:00
Luis Cossío
e704e4fe02 [score boosting] Support datetime expressions (#930)
* gen grpc

* generate rest client

* use float for formula evaluation, match multivalue behavior

* support datetime expressions

* populate inspection cache

* review remarks

* support datetime_key

* fix: address AI comments

* fix: regen inspection cache

* regen inspection cache with pydantic 2.10

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-03 14:51:14 +03:00
Luis Cossío
983fce8a1b [score boosting] support decay functions (#929)
* update grpc client

* generate rest client

* add decay expressions evaluation and fixtures

* add conversions

* validate in formula and fix test

* add fixtures

* new: update inspection cache

* Review remarks

* review nit

* fix rebase

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-03-24 15:38:22 -03:00
Luis Cossío
d938ca3f27 Score boosting local mode (#915)
* add local mode of formula scoring

* grpc<->rest conversions

* fixes for formula evaluation

* add mixed payload type to fixture payload

* update tests

* ignore math warnings (invalid operations raise an error anyway)

* fix conversions

* use same haversine distance as in core

* upd tests

* add fixtures for expressions

* fix import

* temporary hack for tests

* evaluation fixes

* satisfy mypy

* gen async client

* reduce flakiness

..by nesting less equations, so that floating point differences affect
less

* get_args_subscribed

* retriever match multivalue behavior

* review remarks

* more review remarks

* review nit

* no type ignore

* fix imports
2025-03-24 15:24:55 -03:00
tellet-q
635457b47c new: Support for too many requests (#914)
* Update grpc client

* Update rest client and add new exception

* Ensure uploaders continue after wait

* Update

* Address coderabitai review

* Fix tests

* Re-generate clients

* Address review

* Revert attempt handling

* Add new exception

* Add tests

* Do not throw ResourceQuotaExceeded

* Add support for async grpc

* Remove iscoroutine check and more

* update models

* Address some coderabbitai comments

* Address review

* Fix test

* Fix backwards compatibility tests

* Address review

* Get rid of redundant code

* Address ai review

* refactoring: refactor tests

* refactoring: do not inherit resource exhausted from aiorpcerror

* fix: remove redundant import

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-03-24 18:05:15 +03:00
Arnaud Gourlay
553b004065 strict mode tests
* Fix backward compatibility strict mode

* Basic test for strict mode

* remove backward compat patch test from ci

* tests: update strict mode test

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-03-17 13:10:19 +03:00
George
6d8509afee optimize schema parser recursion calls, update models (#924)
* fix: optimize schema parser recursive calls

* fix: fix mypy

* new: update models, regenerate cache (#925)

* new: update models, regenerate cache

* new: update conversions

* fix: fix strict mode config in migrate

* new: update field condition conversion with is_null, is_empty

* fix: apply ruff
2025-03-16 19:59:55 +03:00
Andrey Vasnetsov
d8951c5dae add tests and fix field schema conversion in case grpc is used (#910)
* add tests and fix field schema conversion in case grpc is used

* regen async client
2025-03-06 12:34:52 -03:00
George
ce4c51dd3c Default inference batch size (#907)
* fix: fix inference batch size propagation

* fix: propagate batch size to embed worker

* fix: fix type ignore comment

* fix: replace mocking class method with object method

* fix: add missing inference batch size

* fix: fix batch size propagation to worker
2025-02-21 12:47:57 +01:00
George
3391aa6145 Optional vectors config (#905)
* new: make vectors_config optional in create collection

* remove redundant import

* fix: allow to create a collection without vectors
2025-02-20 14:23:13 +01:00
Alex Ermolaev
3b4a657d74 fix: use the same name for check_compatibility flag in args, warnings and tests (#898)
* Replace `check_version` with `check_compatibility` in warnings and tests to fit flag name

* refactoring: refactor warnings

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-02-17 22:38:59 +01:00
Arnaud Gourlay
321d2a8073 Regen REST and gRPC clients (#903)
* Regen REST and gRPC clients

* update inspection cache

* update dev doc

* add conversions for new strict mode fields

* handle empty strict mode fields + test

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-02-17 18:32:17 +01:00
Dmitrii Ogn
cfe43ed820 Delete non-existant points test (#895)
* Added warnings
Added tests
Added checks

* Removed warning

* Fixed non existent idx

* Fixed text straightforwardness

* Rebuild test
2025-01-30 12:20:14 +01:00
George
b4ae703d1b Local inference upload collection and upload records (#862)
* wip: add draft implementation of batch processing

* fix: embed dict and list of docs, remove redundant code

* new: regen async, small refactor

* refactor: add docstrings, rename methods

* Upload points local inference (#881)

* new: separate single and plural model embeddings

* fix: fix lazy embed models

* new: add inference object inspections to upload methods

* wip: local inference upload parallel

* new: add local inference to upload points and upload collection, refactor mixin

* fix: remove redundant code

* redundant import

* tests: check is query for query points batch

* refactor: refactor semi ordered map

* tests: add test for local inference with batches with docs and vectors

* tests: check the order of dict processing

* new: distinguish models by options

* fix: fix typing

* fix: fix types

* new: embed batches with different options

* tests: add tests for batch with different options

* fix: ignore ide incorrect type inspection

* tests: wait for points to be inserted

* fix: set threads to 1 in parallel inference

* new: adjust max internal batch size

* fix: fix type hints

* function to get embeddings size (#892)

* function to get embeddings size

* async client

* keep sync

* new: extend embedding size to support image and late interaction models

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* new: add local inference batch size (#894)

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-01-29 15:53:24 +01:00
Dmitrii Ogn
ecbaf73220 Migrate cleat fix (#888)
* Migrate cleat fix

* removed redundant import
2025-01-20 12:41:18 +01:00
George
dffab2e208 update clients (#880)
* update clients

* new: update conversions

* fix: update inspection cache

* update grpc

* bump fastembed to 0.5.1

* update poetry lock

* update poetry lock
2025-01-16 13:01:27 +01:00
Andrew Rove (Rover)
349b35bb44 Fix for if a path is given for a remote client URL it is stored as the prefix and not discarded. (#876)
* Fix for if a path is given for a remote client URL it is stored as the prefix.

Though there is an error thrown if the prefix argument and url path are both present.

* refactoring

* refactoring

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-01-07 14:59:20 +01:00