Commit Graph

142 Commits

Author SHA1 Message Date
George Panchuk
05380b1527 new: remove optional and union from qdrant local 2025-12-02 12:30:47 +07:00
George Panchuk
baf2d0e303 new: replace optional and union with | 2025-12-02 12:08:55 +07:00
George
fc6ce2f587 Merge branch 'dev' into drop-python3.9 2025-12-02 11:51:15 +07:00
George
fddf1185ae new: expose the remaining cluster methods (#1123)
* new: expose the remaining cluster methods

* fix: fix type hints

* new: add deadline info to the first hw

* fix: fix review comments
2025-11-24 20:02:44 +07:00
George
5284c02238 fix: fix local mode backward compatibility (#1124)
* fix: fix local mode backward compatibility

* chore: fix comment

* fix: fix fix
2025-11-24 19:43:17 +07:00
George Panchuk
94dded22e4 fix: update type hints from merge 2025-11-18 02:04:27 +07:00
George Panchuk
47b9a83580 Merge branch 'dev' into drop-python3.9 2025-11-18 01:54:26 +07:00
George Panchuk
deb986a9fc fix: remove redundant optional 2025-11-17 17:43:41 +07:00
George
f2e0e22a17 new: expose cluster operations (#1114)
* new: expose cluster operations

* fix: fix cluster collection update return result, add tests

* fix test

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-14 20:20:22 +01:00
George
ac5dca82a5 Merge branch 'dev' into drop-python3.9 2025-11-14 17:37:06 +07:00
George
27acfd0d75 new: remove vectors_count, update http and grpc models (#1069)
* new: remove vectors_count, update http and grpc models

* fix: update inspection cache

* new: add conversions and update interface

* fix: fix some conversions

* fix: fix typo

* fix: fix isinstance

* fix: regen async

* fix: fix update_filter usage, fix isinstance

* tests: collection metadata test

* fix: address backward compatibility in test

* new: update models, add max payload index count and copy vectors

* fix; update _inspection_cache

* new: add read consistency to count points

* Allow uuids in interface (#1085)

* new: direct uuid support

* tests: add uuid tests

* fix: update inspection cache

* new: add collection metadata and tests to local mode (#1089)

* new: add collection metadata and tests to local mode

* fix: regen async client

* new: implement parametrized rrf in local mode (#1087)

* new: implement parametrized rrf in local mode

* refactoring: use a variable for a magic value

* fix: adjust conversion according to AI

* Update filter (#1090)

* new: add missing update_filter, implement it in local mode

* fix: fix type hint, fix update operation, fix rest uploader, add tests

* fix: fix update filter is None case

* fix: mypy was not a good boy

* Text any filter (#1091)

* new: add match text any local mode

* tests: add match text any tests

* new: update models, remove init_from and locks (#1100)

* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: yet another update

* new: add initial_state to create shard key (#1109)

* chore: remove obsolete imports

* fix: add metadata parameter to recreate collection in local

* fix: fix metadata handling in local more

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-14 17:01:17 +07:00
George Panchuk
5c1682678d fix: fix missing type hints, regen async 2025-11-14 03:24:03 +07:00
George Panchuk
698db1d33f new: drop python3.9, replace union and optional with | where possible 2025-11-14 03:20:36 +07:00
George
ff7f584d33 new: update models, remove init_from and locks (#1100)
* new: update models, remove init_from and locks

* deprecate: remove init from tests

* deprecate: remove lock tests

* new: convert ascii_folding

* fix: fix type stub

* new: convert acorn

* new: convert shard key with fallback

* new: update grpcio and grpcio tools in generator (#1106)

* new: update grpcio and grpcio tools in generator

* fix: bind grpcio and tools versions to 1.62.0 in generator

* Remove deprecated methods (#1103)

* deprecate: remove old api methods

* deprecate: remove type stub for removed methods

* deprecate: remove old api methods from test_qdrant_client

* deprecate: replace search with query points in test_in_memory

* deprecate: replace search methods in fastembed mixin with query points

* deprecate: replace old api methods in test async qdrant client

* deprecate: replace search with query points in test delete points

* deprecate: replace discover and context with query points in test_discovery

* deprecate: replace recommend_groups with query_points_groups in test_group_recommend

* deprecate: replace search_groups in test_group_search

* deprecate: replace recommend with query points in test_recommendation

* deprecate: replace search with query points in test search

* deprecate: replace context and discover with query points in test sparse discovery

* deprecate: replace search with query points in test sparse idf search

* deprecate: replace recommend with query points in test sparse recommend

* deprecate: replace search with query points in test sparse search

* deprecate: replace missing search request with query request in qdrant_fastembed

* deprecate: replace search with query points in test multivector search queries

* deprecate: replace upload records with upload points in test_updates

* deprecate: remove redundant structs (#1104)

* deprecate: remove redundant structs

* fix: do not use removed conversions in local mode

* fix: remove redundant conversions, simplify types.QueryRequest

* deprecate: replace old style grpc vector conversion to a new one (#1105)

* deprecate: replace old style grpc vector conversion to a new one

* fix: ignore union attr in conversion

* review fixes

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>

* new: deprecate add, query, query_batch in fastembed mixin (#1102)

* new: deprecate add, query, query_batch in fastembed mixin

* 1.16 -> 1.17

---------

Co-authored-by: generall <andrey@vasnetsov.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-11-11 21:17:08 +07:00
George
c1678f3ebd Text any filter (#1091)
* new: add match text any local mode

* tests: add match text any tests
2025-11-02 17:29:18 +07:00
George
cee3770f63 Update filter (#1090)
* new: add missing update_filter, implement it in local mode

* fix: fix type hint, fix update operation, fix rest uploader, add tests

* fix: fix update filter is None case

* fix: mypy was not a good boy
2025-10-31 18:07:29 +07:00
George
ce1d6fee13 new: implement parametrized rrf in local mode (#1087)
* new: implement parametrized rrf in local mode

* refactoring: use a variable for a magic value

* fix: adjust conversion according to AI
2025-10-31 18:07:05 +07:00
George
a0b5a11aba new: add collection metadata and tests to local mode (#1089)
* new: add collection metadata and tests to local mode

* fix: regen async client
2025-10-31 18:06:55 +07:00
George
c89f615b48 Allow uuids in interface (#1085)
* new: direct uuid support

* tests: add uuid tests

* fix: update inspection cache
2025-10-31 18:06:32 +07:00
George Panchuk
39eead0dff fix: regen async 2025-10-29 16:39:07 +07:00
George
6b031b3262 fix: fix overriding prefetch limit in local mode (#1081)
* fix: fix overriding prefetch limit in local mode

* fix: replace recursion with a while loop in prefetch
2025-10-27 18:14:55 +07:00
George
784b27ab1c new: queryless score 1 (#1088)
* new: queryless score 1

* Reduce hf requests (#1092)

* tests: remove redundant test

* new: improve caching, reduce number of hf requests

* fix: fix possible cache_dir key error
2025-10-24 18:48:13 +07:00
Luis Cossío
7b17db5a7e MMR integration and local mode (#1044)
* generate rest client

* gen grpc client

* conversions

* inpection cache

* wip implement mmr

* upd openapi models

* tests: add mmr tests

* wip: implement mmr

* new: use postprocessed scores

* fix maxsim asymmetry and empty sparse intersection

* fix: fix conversions

* fix: fix type hints

* chore: update inspection cache

* fix: revert changes to add points and update named vectors

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-07-18 12:06:50 +03:00
George
fbf72f4a31 new: relax test comparison (#1039)
* new: relax test comparison

* rollback: rollback is context search addition

* rollback: return redundant parentheses

* Sparse no overlap distance computation (#1040)

* fix: fix sparse distance computation for recommend, discovery and context queries

* fix: use empty_is_zero

* add zero case to score cmp

* add snapshot saving on failure

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-15 13:30:23 +02:00
George
bb7016fd4b fix: fix edge case in upsert, fix multivec update, add tests (#1030)
* fix: fix edge case in upsert, fix multivec update, add tests

* fix: fix tests
2025-07-10 13:22:52 +02:00
George
e32daed967 fix: fix set payload in batch update points, fix operation conversion… (#977)
* fix: fix set payload in batch update points, fix operation conversions, add tests

* new: add some checks

* add comments to test cases

* fix: return comment

* tests: uncomment test due to a fix in core
2025-05-15 19:09:13 +03:00
George
351b17ee63 fix: set default limit in query batch points in local mode (#967) 2025-04-24 17:16:55 +03:00
Luis Cossío
337c70173e dedup order_by by value and id (#959)
* dedup order_by by value and id

* fix: fill order_value when order_by is used

* tests: add tests for duplicated values, fix payload value comparison

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-24 16:53:02 +03:00
George
5afdab1c3c fix: fix bm25 search on empty collection (#966) 2025-04-24 16:52:42 +03:00
George
ee2f5f451f Rethink offset prefetch (#955)
* fix: fix local mode offset in sample random

* fix: fix random sampling, do not propagate offset into prefetches

* fix: ignore offset in random sampling
2025-04-22 16:53:58 +03:00
George
2afadf4cb5 fix: non-list conditions in filter clauses (#949)
* fix: non-list conditions in filter clauses

* r u happy mypy?
2025-04-15 16:43:22 +03:00
Hossam Hagag
5f69b4f28b Unlock numpy ci (#946)
* ci: Unlcok numpy

* fix: Fix type hints

* chore: Remove comment

* fix: Fix type hints
2025-04-15 10:31:48 +03:00
George
5a454a7402 Recommend sum score (#940)
* new: add sum_scores strategy, fix lookup from in query_batch_points, fix euclid between multivectors local mode

* new: recommend average multivectors, fix tests, add new tests for multivectors

* fix: address review comments

* fix: regen async

* tests: uncomment tests

* new: update poetry lock, require pydantic >=2.11 for populate inspection cache

* fix: tmp disable euclidean multivec search tests, fix multivec with nan test

* fix: align euclid and manhattan with core, add tests

* fix: fix type hints

* tests: use atol in comparison

* tests: remove manhattan tests

* tests: reduce number of multivectors

* remove float comparison relaxation

* explain why it is different for euclid and manhattan

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-04-07 11:45:33 +03:00
Luis Cossío
e704e4fe02 [score boosting] Support datetime expressions (#930)
* gen grpc

* generate rest client

* use float for formula evaluation, match multivalue behavior

* support datetime expressions

* populate inspection cache

* review remarks

* support datetime_key

* fix: address AI comments

* fix: regen inspection cache

* regen inspection cache with pydantic 2.10

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2025-04-03 14:51:14 +03:00
Luis Cossío
d938ca3f27 Score boosting local mode (#915)
* add local mode of formula scoring

* grpc<->rest conversions

* fixes for formula evaluation

* add mixed payload type to fixture payload

* update tests

* ignore math warnings (invalid operations raise an error anyway)

* fix conversions

* use same haversine distance as in core

* upd tests

* add fixtures for expressions

* fix import

* temporary hack for tests

* evaluation fixes

* satisfy mypy

* gen async client

* reduce flakiness

..by nesting less equations, so that floating point differences affect
less

* get_args_subscribed

* retriever match multivalue behavior

* review remarks

* more review remarks

* review nit

* no type ignore

* fix imports
2025-03-24 15:24:55 -03:00
George
3391aa6145 Optional vectors config (#905)
* new: make vectors_config optional in create collection

* remove redundant import

* fix: allow to create a collection without vectors
2025-02-20 14:23:13 +01:00
Dmitrii Ogn
cfe43ed820 Delete non-existant points test (#895)
* Added warnings
Added tests
Added checks

* Removed warning

* Fixed non existent idx

* Fixed text straightforwardness

* Rebuild test
2025-01-30 12:20:14 +01:00
Hossam Hagag
47ff758c56 refactor: Refactored the way of warnings (#864)
* refactor: Refactored the way of warnings

* remove unused imports

* fix: Fix stacklevel in warnings

* regenerated async

* nit

* Updated some warnings with show once

* fix: Fix stack levels

* Updated async

* Update qdrant_client/qdrant_remote.py

Co-authored-by: George <george.panchuk@qdrant.tech>

* Update async client

* Updated warnings

* Updated warnings

* Updated warnings

* fix: fix warning level

* fix: fix warning level in async

* fix: revert append payload condition

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2025-01-06 16:44:06 +01:00
Andrey Vasnetsov
3f56da7d02 Update API to v1.13 (#874)
* update openapi and grpc + conversions

* add strict mode to conversions + coverage

* add strict mode to collection creation

* add local mode for has_vectors condition

* regen async with python 3.10

* tests for has-vector

* fix filters in tests

* fix conversion

* fix test

* Has vector tests (#878)

* new: add has vector fixture

* tests: add has vector multivector test

* fix: add missing strict mode config usage and conversion (#877)

* fix: add missing strict mode config usage and conversion

* fix: pass strict mode config in update collection

* fix: add strict mode config to recreate collection

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
2025-01-06 11:44:25 +00:00
Hossam Hagag
88543cb852 User warnings large upload (#852)
* chore: Added warning messages when working with large points in local mode

* chore: Added deprecation message

* chore: Add default to idx

* Fix: Updated local mode user warning for large volume of points

* chore: Added cloud

* chore: Updated async client

* improve: Improved how we handle warning messages

* chore: Changed warning stack level
refactor: Refactored user warnings in local mode when large upload

* fix: Fix ci

* refactor: update warning messages

---------

Co-authored-by: George Panchuk <george.panchuk@qdrant.tech>
2024-12-05 13:01:05 +01:00
George
baa3e264c8 Drop python 3.8 (#848)
* new: drop python 3.8 support, update type hints

* fix: remove 3.8 from ci

* fix: update type hints

* fix: make netlify use python3.10

* fix: try python3.9 for sphinx

* debug: try updating sphinx

* new: bump ffastembed to 0.4.2

* fix: install numpy<2 for mypy

* fix: install numpy via poetry
2024-11-15 15:54:32 +01:00
George
6d29cc6c1c new: extend signatures with images and inference objects (#842)
* new: extend signatures with images and inference objects

* fix: remove draft code
2024-11-11 17:52:55 +01:00
George
b2add2dd72 fix: fix resolve prefetch input (#816)
* fix: fix resolve prefetch input

* fix: regen async
2024-10-25 18:19:03 +02:00
Hossam Hagag
b1bd522e41 fix: Raise if named vector not found in local mode while upsert (#813)
* fix: Raise if named vector not found in local mode

* fix: Raise if named vector not found in local mode while upsert

* chore: Added test case
2024-10-18 11:24:12 +04:00
George
589772635b fix: fix prefetch conversion, fix local mode query batch points offset, add tests (#812) 2024-10-15 11:25:06 +02:00
George
f2494cacdc fix: fix default vector distance matrix (#810) 2024-10-11 20:43:54 +02:00
Andrey Vasnetsov
2cd8214deb make api names consistent with grpc + fix a BUG (#805) 2024-10-07 23:35:55 +02:00
Luis Cossío
a6e2324884 chore: Update clients (#758)
* update rest client

* update grpc client with grpcio 1.65.4

* update grpc client with grpcio==1.49.1

* generate grpc client with grpcio==1.48.2

* generate grpc client with grpcio==1.66.1 and mypy-protobuf extension

* add mypy-protobuf to pyproject

* relax version for grpcio, but keep grpcio-tools at 1.48.2

* improve generation script

* fix bash script comment

Co-authored-by: George <george.panchuk@qdrant.tech>

* use old grpcio to generate client

* rollbakc mypy-protobuf

* regenerate with distance matrix

* new: update clients (#774)

* new: update clients

* rollback grpcio version

* feature: Facets (#759)

* facet in local_collection.py

* replace usages of calculate_payload_mask

* - qdrant remote
- qdrant base
- qdrant client
- conversions

* congruence tests + local mode fixes

* generate async client

* add type stubs and misc fixes

* fix mypy in Python 3.8

* generate async client

* review remarks

* gen async client

* update for bool facets

* Feature: Distance Matrix API (#769)

* add remote impls

* add client impl

* regen async client

* start local

* regen async

* local mode

* start congruence

* Fix local mode

* regen async of course

* test filtering

* fix min samples count

* simplify comparaison loop

* simplify samples loop

* add rest/gRPC conversion tests

* fix conversions + tests

---------

Co-authored-by: George <george.panchuk@qdrant.tech>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2024-10-05 23:03:13 +02:00
Ruben
52d96974fd Remove-np-ninf (#782)
* change np.NINF to -np.inf

* Update qdrant_client/local/local_collection.py

---------

Co-authored-by: George <panchuk.george@outlook.com>
2024-09-24 21:07:25 +02:00
George
36995d41e5 fix: fix update_vectors in local mode, fix record comparison in tests (#787) 2024-09-24 16:15:33 +02:00