Commit Graph

39 Commits

Author SHA1 Message Date
George
9f9c500bf4 Merge branch 'dev' into drop-python3.9 2025-12-04 11:58:23 +07:00
George Panchuk
64da41deba fix: replace remaining union, optional, etc, address review comments 2025-12-04 11:49:15 +07:00
George
458646cb95 tests: speed up tests (#1130)
* fix: do not try to check compatibility in test_client_init

* tests: remove sparse-code vectors (10_000 dim), remove euclid recommend methods as non-applicable for sparse

* tests: add payload indexes to query group test

* debug: add durations=0 to pytest

* debug: test only test query group

* debug: try adding payload indexes to query group test

* rollback test launch
2025-12-04 11:22:46 +07:00
George
27acfd0d75 new: remove vectors_count, update http and grpc models (#1069)
* new: remove vectors_count, update http and grpc models

* fix: update inspection cache

* new: add conversions and update interface

* fix: fix some conversions

* fix: fix typo

* fix: fix isinstance

* fix: regen async

* fix: fix update_filter usage, fix isinstance

* tests: collection metadata test

* fix: address backward compatibility in test

* new: update models, add max payload index count and copy vectors

* fix; update _inspection_cache

* new: add read consistency to count points

* Allow uuids in interface (#1085)

* new: direct uuid support

* tests: add uuid tests

* fix: update inspection cache

* new: add collection metadata and tests to local mode (#1089)

* new: add collection metadata and tests to local mode

* fix: regen async client

* new: implement parametrized rrf in local mode (#1087)

* new: implement parametrized rrf in local mode

* refactoring: use a variable for a magic value

* fix: adjust conversion according to AI

* Update filter (#1090)

* new: add missing update_filter, implement it in local mode

* fix: fix type hint, fix update operation, fix rest uploader, add tests

* fix: fix update filter is None case

* fix: mypy was not a good boy

* Text any filter (#1091)

* new: add match text any local mode

* tests: add match text any tests

* new: update models, remove init_from and locks (#1100)

* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: yet another update

* new: add initial_state to create shard key (#1109)

* chore: remove obsolete imports

* fix: add metadata parameter to recreate collection in local

* fix: fix metadata handling in local more

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-14 17:01:17 +07:00
George Panchuk
698db1d33f new: drop python3.9, replace union and optional with | where possible 2025-11-14 03:20:36 +07:00
George
ce1d6fee13 new: implement parametrized rrf in local mode (#1087)
* new: implement parametrized rrf in local mode

* refactoring: use a variable for a magic value

* fix: adjust conversion according to AI
2025-10-31 18:07:05 +07:00
George
6b031b3262 fix: fix overriding prefetch limit in local mode (#1081)
* fix: fix overriding prefetch limit in local mode

* fix: replace recursion with a while loop in prefetch
2025-10-27 18:14:55 +07:00
Luis Cossío
7b17db5a7e MMR integration and local mode (#1044)
* generate rest client

* gen grpc client

* conversions

* inpection cache

* wip implement mmr

* upd openapi models

* tests: add mmr tests

* wip: implement mmr

* new: use postprocessed scores

* fix maxsim asymmetry and empty sparse intersection

* fix: fix conversions

* fix: fix type hints

* chore: update inspection cache

* fix: revert changes to add points and update named vectors

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-07-18 12:06:50 +03:00
George
7bce5f0b75 tests: catch math domain error, catch abrupted error message in remote (#989) 2025-05-14 20:08:25 +03:00
George
5afdab1c3c fix: fix bm25 search on empty collection (#966) 2025-04-24 16:52:42 +03:00
George
ee2f5f451f Rethink offset prefetch (#955)
* fix: fix local mode offset in sample random

* fix: fix random sampling, do not propagate offset into prefetches

* fix: ignore offset in random sampling
2025-04-22 16:53:58 +03:00
George
5a454a7402 Recommend sum score (#940)
* new: add sum_scores strategy, fix lookup from in query_batch_points, fix euclid between multivectors local mode

* new: recommend average multivectors, fix tests, add new tests for multivectors

* fix: address review comments

* fix: regen async

* tests: uncomment tests

* new: update poetry lock, require pydantic >=2.11 for populate inspection cache

* fix: tmp disable euclidean multivec search tests, fix multivec with nan test

* fix: align euclid and manhattan with core, add tests

* fix: fix type hints

* tests: use atol in comparison

* tests: remove manhattan tests

* tests: reduce number of multivectors

* remove float comparison relaxation

* explain why it is different for euclid and manhattan

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-04-07 11:45:33 +03:00
Luis Cossío
e704e4fe02 [score boosting] Support datetime expressions (#930)
* gen grpc

* generate rest client

* use float for formula evaluation, match multivalue behavior

* support datetime expressions

* populate inspection cache

* review remarks

* support datetime_key

* fix: address AI comments

* fix: regen inspection cache

* regen inspection cache with pydantic 2.10

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-03 14:51:14 +03:00
Luis Cossío
d938ca3f27 Score boosting local mode (#915)
* add local mode of formula scoring

* grpc<->rest conversions

* fixes for formula evaluation

* add mixed payload type to fixture payload

* update tests

* ignore math warnings (invalid operations raise an error anyway)

* fix conversions

* use same haversine distance as in core

* upd tests

* add fixtures for expressions

* fix import

* temporary hack for tests

* evaluation fixes

* satisfy mypy

* gen async client

* reduce flakiness

..by nesting less equations, so that floating point differences affect
less

* get_args_subscribed

* retriever match multivalue behavior

* review remarks

* more review remarks

* review nit

* no type ignore

* fix imports
2025-03-24 15:24:55 -03:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
George
b2add2dd72 fix: fix resolve prefetch input (#816)
* fix: fix resolve prefetch input

* fix: regen async
2024-10-25 18:19:03 +02:00
paul2302
19b80b6bae Fix nested prefetches in qdrant_local.py (#772)
* Fix nested prefetches in qdrant_local.py

Local qdrant client with nested prefetches raised the exception: none type has no attribute query. This little change fixes it

* chore: Added the check for prefetch to async

* fix: Fix nested prefetch in async local mode by generating the async client

* chore: Added a test case for query using nested prefetch in congruence tests

* remove redundant version check

---------

Co-authored-by: hh-space-invader <h.hagag.ali@gmail.com>
Co-authored-by: George <panchuk.george@outlook.com>
2024-09-13 20:31:51 +02:00
Hossam Hagag
97e317e1f1 improved semver comparison (#736)
* refactor: Improved semver comparision in tests

refs: #723

* refactor: Improved semver comparision in tests

* refactor: Refactored semver comparision in recommend
2024-08-23 19:35:34 +02:00
Hossam Hagag
c92a0432f3 Refactor recreate collection (#719)
* deprecated recreate_collection from tests

* checked if collection exists before deletion

* reverted deletion of recreate_collection

* added missing collection_exsits before deletion

* nit if condition

* fix type hint in local_collection

* removed run mypy on async client generator

* added checks for create_collection if exists

* nit change collection name

* add else to create_collection

* nit

* nit

* refactoring: remove redundant checks

* fix: do not test collection_exists through itself

* refactoring: remove redundant collection exists checks

* fix: remove redundant collection exists checks

* refactoring: remove redundant collection checks

* refactoring: remove redundant collection existence check in test sparse search

* refactoring: remove redundant checks in test_updates

* refactoring: remove redundant checks

* fix: remove redundant checks

* refactoring: remove redundant checks

* refactoring: remove redundant imports

* new: add recreate collection test

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-08-09 23:33:31 +02:00
Arnaud Gourlay
44bbded8f2 universal-search: Query Group API and local mode (#690)
* universal-search: Query Group API and local mode

* requires different logic for gRPC error assertion

* you come to me at runtime for a compile time issue

* suddenly throwing a different error

* test more group key types

* extend limit of prefetches during group_by

* rescoring is the issue

* one problem at a time please

* code review

* regen clients

* code review

* add lookup_from to query_points_groups

* test with_lookup

* test and fix gRPC

* drop dedicated conversion

* Update qdrant_client/qdrant_client.py

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* regen async

* Distribution-based score fusion in local mode (#703)

* pre-implement dbsf

* add dbsf congruence tests

* mypy lints

* add conversions

* tests: add test for dbsf conversion

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* Random sample in local mode (#705)

* pre-implement random sampling

* generate models

* add conversions and tests

* fix mypy lints

* tests: add test for sample random conversion

* use camelcase Sample.Random

* review fixes

* fix mypy

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>

* fix: add type ignore for mypy

* fix: fix type hints for 3.8

* fix: do not run mypy on async client generator in CI, simplify condition

* Grpc comparison in tests (#726)

* add parametrized fixture for using grpc too

* compare grpc and http without running each setup twice

* fix: fix exception types in invalid types test

* fix: remove random seed which led to a erroneous sequence

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-08-09 17:03:28 +02:00
George
5c082756c7 fix: do not modify input structs in-place (#715)
* fix: do not modify input structs in-place

* fix: regen async
2024-08-08 16:54:53 +02:00
George
fb7e396e67 fix: fix grpc conversion bugs for sparse and multi vectors (#682)
* fix: fix grpc conversion bugs for sparse and multi vectors

* fix: fix mypy
2024-07-08 20:12:53 +03:00
George
a77a2271ea fix: skip no prefetch query test if version is low (#681) 2024-07-08 18:24:43 +03:00
Luis Cossío
93b3e7929b Add a test for scrolling query with offset (#676) 2024-07-08 12:40:30 +03:00
George Panchuk
4785cd7973 fix: fix lookup test, refactor search params test 2024-06-28 20:00:11 +03:00
Luis Cossío
74345cd7f8 danse dense revolution 🕺 2024-06-28 12:30:43 -04:00
Luis Cossío
ecc344ee7d fix return types 2024-06-28 12:29:30 -04:00
Arnaud Gourlay
0de0124dde do not query with empty sources 2024-06-28 16:31:22 +02:00
tellet-q
f4b080c405 Revert "tests: Add lookup_from test"
This reverts commit 9ee4a756f557773ea3867cecda2abc203cf570f9.
2024-06-28 16:31:22 +02:00
tellet-q
680a5aecfd tests: Add lookup_from test 2024-06-28 16:31:22 +02:00
tellet-q
bc2d47276b tests: Add more test cases 2024-06-28 16:31:22 +02:00
tellet-q
2a4a6bdb9a tests: Add 2 cases (no query, no prefetch and on filtered prefetch). 2024-06-28 16:31:18 +02:00
tellet-q
5f02bad5a1 Raise ValueError if vector name not found 2024-06-28 15:50:23 +02:00
tellet-q
f2330ed228 Fix make negative test fail 2024-06-28 15:44:10 +02:00
tellet-q
d6211808bc Fix positive flow and add negative tests 2024-06-28 15:42:00 +02:00
tellet-q
2514a4aa32 tests: Add lookup_from test 2024-06-28 14:27:46 +02:00
George
6400c8316b fix: increase context search limit (#666)
* fix: increase context search limit

* fix: reduce collection size for context test
2024-06-28 10:10:44 +03:00
Arnaud Gourlay
37e167d2e2 add init tests for sparse & multi 2024-06-27 19:59:08 +02:00
Luis Cossío
c3e8ec00e5 Local mode for Query 2024-06-27 19:46:25 +02:00