* feat: add search_max_batchsize to strict mode config
* added test case for search_max_batchsize
* Changes for fixing CI issue dure openapi
* Modify check_strict_mode_batch
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Don't account for deferred points in some places
# Conflicts:
# lib/collection/src/shards/local_shard/scroll.rs
* Add in QueryContext
* Cover more places
* Coderabbit review remarks
* Properly count amount of deleted deferred points (#8386)
* Properly count amount of deleted deferred points
* Prevent double-counting of the same point
* Remove hints to estimations
* Properly handle counts in ProxySegment
* Add tests and fix deleted point count issue
* Adjusts tests + fix issues
* Separte fields for deferred points in telemetry (SegmentInfo)
* Remove deferred_points_count()
* openapi
* Fix test by manually calculating visible points
* Adjust ProxySegment test to revertion of SegmentInfo
* Throw error if collection was not found in telemetry (e2e Test)
* Update lib/segment/src/segment/segment_ops.rs
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Make new fields in API optional
* Don't take range if no deferred point exist
* Review remarks
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* restrict recovery from snapshot folder
* fmt
* clippy
* Extend test, path must be inside snapshots directory
* In release builds, hide detailed tar error message to not leak contents
* Change from bad request to forbidden
* fix(auth_tests): place snapshot in peer snapshots dir for recover_collection_snapshot
The recover from snapshot API now requires file:// paths to be inside the
configured snapshots directory. Write the test snapshot into the peer's
snapshots directory instead of a temp file so the request is allowed.
Made-with: Cursor
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Cursor Agent <agent@cursor.com>
* [manual] make untagged enum for consistent API for FeedbackStrategy
* add #[validate(nested)]
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Add integration test to assert all queued updates are also transferred
* Plunge update queue in stream records transfers
* Migrate existing plunger usages to new plunge helper
* Skip test if not compiled with staging flag
* Only send delay operation when staging feature is enabled
* Reformat
* Fix review remarks
* add rest and grpc interfaces
Also handle inference for this query
* follow refactor from base branch
* renaming from Ms Cooper
* get started on validation testing
* more validations
* test equivalence with query when less than 2 feedback elements
* rename feedback query to relevance_feedback query
* fix extraction of context pairs
* rename feedback vector to example
* make coderabbit happier
* upd test
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* handle `score_threshold` in Formula queries
* AI: add openapi test
prompt: Upload 4 points with a numeric payload, use the payload as the
score, and set a score threshold. We can assert which ids should be in
the result and which shouldn't
* weighted rrf implementation
* test
* fmt
* fix edge
* validate number of sources and number of weights
* do not partial match
* upd schema
* review fixes
* update formula
* remove calcualtions from tests
* update comment, because AI have OCD
* fmt
* Update queue info into CollectionInfo
* basic test
* fix
* cleaner
* Update lib/collection/src/shards/local_shard/mod.rs
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* fix the fix
* rename to op_num
* skip_serializing_if just in case
* first step
* test queue length with staging feature
* gate integration test based on binary feature
---------
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert
* add test
* upd dockstring
* require resharding once all peers have updated version
* use service error
* fix clippy again
* wait for same version before resharding in tests
* feat: Add enable_hnsw option for payload field indexes
Add optional enable_hnsw parameter to all payload index types to control
whether additional HNSW graph links are built for each indexed field.
- Add enable_hnsw field to all 8 payload index param types
- Update gRPC proto definitions and conversions
- Update OpenAPI schema
- Modify HNSW graph builder to respect enable_hnsw flag
- Add enable_hnsw() helper methods to PayloadSchemaParams and PayloadFieldSchema
- Update all tests to include new field (default: None)
When enable_hnsw is true and payload_M > 0, additional HNSW links will
be built for the payload field. Default value is true for backward compatibility.
* Fix Some format problems
* fix: address comment problem
---------
Co-authored-by: EC2 Default User <ec2-user@ip-10-78-171-148.ec2.internal>
* Add progress_tracker.rs
* Pass progress tracker around
* Populate progress tracker with actual data
* Expose progress on `/collections/{name}/optimizations` endpoint
* Add vector count per vector-name to metrics API
* Add to metrics API
* Improve TinyMap::get_or_insert_default and add tests
* Minor improvements
* Update openapi
* Review remarks
* Remove `collection_vectors` since it can be calculated manually
* [strict-mode] Add max number of payload index count
* improve docs and validation for StrictModeConfig
* fix from coderabbit
* test blocking access to payload indexes
* polish test
* internal implementation for filter any
* implement api for match `text_any`
* allow naive implementation of text match any
* dedup after kmerge + renames
* improve `expected_should_estimation`
* congruence test
* openapi test
* setup index in each test
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Add test to catch bug
Bug: <https://github.com/qdrant/qdrant/issues/6880>
* Fix loading null index from wrong path
* Simplify test
* Also bump total point count when removing a point
* Add debug assertion, ensure we don't create null index in selector
* Update bug link
* Remove unused import
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* expose mmr in rest and grpc
* fix referenced vectors in mmr queries
* rename to MmrInput in grpc for consistency
* mmr is now an optional parameter of nearest query
* fix score_threshold at local shard
* fix conversion of rest -> collection
* fix and refactor planned query
* nits
* rename `lambda` to `diversity` in interface
* handle score_threshold within mmr calculation
* add openapi test
* fix candidate limit at collection level
* candidate_limit -> candidates_limit
* finish rename to candidates_limit
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* add `"match": { "phrase": ... }` condition
* gen grpc and openapi
* [phrase matching] expose `phrase_matching` flag in rest and grpc (#6620)
* expose setting in rest and grpc
* phrase matching openapi test
* regen openapi
* [phrase matching] Text index fixes (#6730)
* allow rocksdb-based immutable text index
* fix repeated-token phrases
* fmt
* Update OpenAPI spec
---------
Co-authored-by: timvisee <tim@visee.me>
* add repeated word case in openapi test
* [phrase match | strict mode] Allow phrase condition when enabled in index (#6749)
* allow phrase filter when index is present
* prettier error message
* clippppppy
---------
Co-authored-by: timvisee <tim@visee.me>
* Strict mode allows fullscan for multitenant payload index
* different error for missing payload index in multitenant case
* handle payload_m
* add simple test
* handle hnsw.m not set
* add another simple test
* fix test
* fallback to global HSNW config
* new error status code
* do not block on global HNSW and improver error reporting
* clearer error reporting
* review fixes
* fix test error messages
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Fix UnindexedField infra to handle lookup&range index requirements
* more generic approach
* better test
* restore
* small unit test for sanity
* review nits
* Do not recommend parametrized index
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
