* feat: slice filtering condition for sliced scroll and deterministic sampling
Add a `slice` filter condition selecting points where
`stable_hash(point_id) % total == index`. The hash is SipHash-2-4 with a
zero key over canonical id bytes (8 LE bytes for numeric ids, 16 RFC 4122
bytes for UUIDs) — a frozen public contract, independent of the internal
resharding ring hash, reproducible by clients to predict membership.
For a fixed `total`, slices are disjoint and cover all points, enabling
parallel scroll streams (ES sliced-scroll style) and reproducible sampling
that composes with any other filter condition.
- REST: `{"slice": {"total": N, "index": R}}`; gRPC: `SliceCondition` in
the condition oneof (tag 8)
- Evaluated per point via id_tracker external-id lookup; no payload index
needed; cardinality estimated as `points / total` with no primary clause
- `total >= 1` enforced by NonZeroU32 at parse time, `index < total` by
validation in both REST and gRPC paths
- Hash contract locked by test vectors independently reproduced with a
reference SipHash-2-4 implementation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tests: minimal OpenAPI test for slice filter condition
Scrolls all slices of a fixed total over numeric + UUID ids asserting
disjointness and full coverage, checks must_not inversion, and pins the
two rejection paths (422 for index >= total, 400 for total = 0). Requests
and responses are validated against the regenerated OpenAPI spec by the
test harness.
Note: the spec cannot itself reject total = 0 client-side — the Condition
anyOf falls through to the permissive Filter schema, as with any invalid
condition — so rejection is asserted via the server response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add per-query IDF corpus for sparse vector search
Let the caller choose, per query, which population sparse IDF statistics
are computed over. `params.idf` is either `"global"` (default, unchanged
behavior) or `{"corpus": <filter>}`, where the corpus filter is
independent of - and usually broader than - the retrieval filter.
Decoupling the two keeps the score scale stable when the retrieval
filter tightens: term importance is measured against a population the
user names, not against whatever subset the filter happens to select.
Design decisions:
- Corpus grammar is restricted to a conjunction (`must`) of `match`
conditions on payload fields; loosening later is backward compatible.
- Strict mode validates the corpus filter like a read filter
(unindexed fields rejected).
- `idf` on a vector without the IDF modifier is a validation error,
never silently ignored.
- An empty corpus yields degenerate but corpus-scoped scores (smoothed
IDF over N=0), never a fallback to global statistics - in multi-tenant
collections a fallback would leak term statistics across tenants.
Implementation:
- QueryContext IDF stats are keyed by corpus, so one batch can mix
requests with different corpora.
- Statistics come from the sparse index: df(term) is counted over the
query terms' posting lists only, never by scanning stored vectors.
Small corpora (under ~1/32 of the segment, by cardinality estimate)
are kept as a sorted id list galloping through posting lists via
skip_to; large ones as a dense membership mask filled streaming from
the filtered-points iterator. A misestimated small corpus degrades
into the mask.
- Exposed uniformly: REST (`params.idf`), gRPC (`IdfParams` message),
edge python bindings; OpenAPI schema regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Apply rustfmt
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix clippy manual_is_multiple_of in sparse IDF corpus test.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Allow any filter as IDF corpus
Drop the must+match grammar restriction on the corpus filter. A
restriction enforced only as a validation step over the full Filter
type buys nothing; if a narrower corpus syntax is ever wanted, it
should be a dedicated API-level type instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix build: add memory field to SparseIndexConfig in idf corpus test
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add unified `memory: cold|cached|pinned` placement parameter for collection components
Introduce a single `memory` parameter that controls how each collection
component's data is held in RAM, replacing the inconsistent zoo of
`on_disk` / `always_ram` / `on_disk_payload` flags:
- `cold`: not pre-loaded from disk, cached with usage
- `cached`: pre-populated into page cache on load, evictable under pressure
- `pinned`: materialized on heap, never evicted by cache pressure
The parameter is available on dense vectors, HNSW config, all quantization
configs, the sparse index, all payload field index types, and payload
storage (as a new `payload: { memory }` sub-object on collection params).
When set, it overrides the deprecated legacy flag; when unset, behavior is
unchanged. Legacy flags are marked deprecated (Rust + proto) but keep
working; conflicts are resolved in favor of `memory` with a warning.
New capabilities enabled by the tri-state model:
- HNSW graph links can be pinned (first production caller of the existing
`GraphLinksResidency::Pinned`)
- sparse mmap index, quantized vectors and on-disk payload field indexes
gain a `cached` tier (mmap + populate on open)
`pinned` is rejected by API validation for components without a heap
variant (dense vector storage, payload storage). Low-memory mode degrades
placements at load time via `Memory::clamp_to_low_memory`, matching the
existing `prefer_disk`/`skip_populate` behavior. Effective-placement
comparison in the config-mismatch optimizer avoids spurious rebuilds when
the same placement is expressed through the new parameter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix gpu-gated tests for the new `memory` field
CI clippy runs with --all-features, which compiles the gpu-gated tests
that were missed locally: add the `memory` field to config literals and
allow deprecated placement params, same as in the rest of the tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add OpenAPI tests for memory placement, keep sparse config downgrade-clean
- OpenAPI tests: create/update collections with `memory` on every component,
assert the parameters are echoed in collection info, assert legacy-only
collections expose no new fields, and assert `pinned` is rejected (422)
for dense vector storage and payload storage on both create and update.
- Persist only the explicitly requested `memory` parameter in
`sparse_index_config.json` instead of the legacy-resolved placement, so
configurations using only the deprecated `on_disk` flag keep byte-identical
files that older Qdrant versions load without unknown fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Validate collection meta ops at construction, not only in the API layer
The `memory: pinned` rejection for dense vectors and payload storage
lived in `Validate` impls on the internal request types, which only ran
through the REST actix extractor. gRPC validates just the proto message,
so a gRPC client could persist `pinned` where it is not supported and
have it silently treated as `cached`.
Run the derived validation in `CreateCollectionOperation::new` and
`UpdateCollectionOperation::new` instead: the constructors are the
common chokepoint for all API paths, before the operation is proposed
to consensus. This covers every validator on these types, not just the
`memory` checks, and keeps consensus-apply unaffected so mixed-version
clusters never reject already-committed operations.
`UpdateCollectionOperation::new` becomes fallible; `remove_replica` now
uses `new_empty` since it carries no user config. Regression tests drive
the gRPC conversion path and assert `InvalidArgument` for `pinned` on
create and update, with `cold`/`cached` accepted as a control.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add prefix matching option to keyword index
Introduce an opt-in `prefix` option for the keyword payload index and a
new `match: { "prefix": ... }` filter condition, enabling efficient
byte-wise prefix filtering over keyword values (e.g. URL prefixes,
web-ui value autocompletion via facet + prefix filter).
Index side: a new `prefix_index.bin` file stores a sorted, front-coded
key dictionary with a resident block index (cumulative counts per
block); it is an ordered view over the keys of `values_to_points.bin`
and stores no postings. Presence of the file signals prefix support at
load time, so legacy segments load unchanged and enabling the option
goes through the standard incompatible-schema rebuild. The mutable
variant keeps an in-RAM ordered key set (not persisted), the immutable
variant builds a sorted key vector at load, and the on-disk variant
reads the dictionary lazily (block index resident, 1-2 block reads per
prefix lookup; reader is generic over UniversalRead).
Query side: prefix conditions are served from the dictionary when
available (filter + cardinality estimation from per-block aggregates),
from the forward index as per-point checks, and degrade to the payload
full-scan fallback otherwise - same execution model as other match
conditions. Strict mode (`unindexed_filtering_*`) rejects prefix
queries on fields without a prefix-enabled keyword index via a new
KeywordPrefix capability.
HNSW payload blocks: prefix-enabled indexes additionally emit prefix
blocks for heavy branching trie nodes (single-child chains collapsed to
their longest common prefix, one block per distinct point set, emitted
largest-first) so filtered search with prefix conditions gets navigable
subgraphs without rebuilding the same subset repeatedly.
API: `prefix` flag on KeywordIndexParams (REST bool, gRPC empty
message for extensibility), `prefix` variant in the Match oneof, edge
python bindings, regenerated OpenAPI spec.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Split prefix index into a dedicated module, fix clippy in tests
Reorganize the flat prefix_index.rs / prefix_read.rs into a
map_index/prefix_index/ module: format.rs (on-disk layout primitives),
writer.rs, reader.rs (PrefixIndex), map_read.rs (StrMapIndexPrefixRead
with per-variant impls) and tests.rs, with a file-format diagram and a
read-path walkthrough in the module docs. No logic changes.
Also fix clippy --all-targets complaints in test code: replace a
wildcard Match arm with an exhaustive list and a field-reassign-with-
default with a struct literal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add OpenAPI test for prefix match and snapshot file-tracking test
- tests/openapi/test_prefix_match.py: index-less fallback, prefix index
creation with schema echo, scroll/count parity against ground truth,
facet + prefix filter (the autocompletion flow), strict-mode rejection
without the prefix capability.
- test_prefix_index_file_tracking: `prefix_index.bin` is listed in
`files()` / `immutable_files()` exactly when built with the option.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Replace hand-rolled varint parsing with bytemuck Pod records
Per review: the prefix index format now uses fixed-size little-endian
Pod records (BlockEntry 24 B, KeyEntry 12 B, Header 40 B) written with
bytemuck::bytes_of and read back by copy via pod_read_unaligned — no
manual varint encode/decode, no alignment requirement, one shared
read_record helper. Costs ~9 bytes per key on disk versus LEB128; the
raw key bytes dominate dictionary size, so the simplification wins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fetch the whole candidate block range with a single storage read
Candidate key blocks of a prefix lookup are contiguous in the file, so
enumerate them from one ranged read instead of one read per block; the
over-read versus the exact key range is bounded by the two boundary
blocks. Block decoding is split into a storage-free helper reused by
the per-block path of stats estimation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Align prefix payload blocks with the geo index granularity principle
Geo's large_hashes emits only the smallest geohash regions above the
threshold — a disjoint antichain, never a parent nested with its
children. Prefix payload blocks now follow the same rule: a heavy
collapsed trie node is emitted only if nothing heavy is nested inside
it, counting both deeper qualifying prefixes and single heavy values
(which already get their own exact-match blocks). Emitted blocks are
therefore mutually disjoint and disjoint from exact-value blocks; no
near-collection-sized ancestor subgraphs, no reliance on the HNSW
connectivity check to skip nested duplicates.
Implemented as a `covered` flag propagated through the existing
LCP-interval scan, still one O(total key bytes) pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Document block wire format and unaligned-read rationale in decode_block
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Covers turbo4 as vector storage (config persistence, search across
Cosine/Dot/Euclid) and binary / turbo quantization layered on top of
turbo4 storage (quantized search with rescore/oversampling on an
optimized segment).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test: add openapi regression test for values_count on missing fields
Reproduces https://github.com/qdrant/qdrant/issues/9586 where points with
a missing payload field are not matched by values_count filters that should
treat the count as 0.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: use module fixture for values_count missing field test
Follow the standard openapi test structure with setup/teardown fixture
instead of explicit drop_collection calls in the test body.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: treat missing field as count 0 for values_count filter (#9607)
`FieldCondition::check_empty()` ignored the `values_count` condition, so a
point with a missing payload field was never matched by a `values_count`
filter. The desired semantics (and the existing `ValuesCount::check_empty`)
treat a missing field as having a value count of 0.
Forward the empty check to `values_count.check_empty()` so bounds like
`lt: 1`, `gte: 0` and `lte: 0` correctly match points whose field is absent.
Fixes#9586
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: reproduce MatchAny(any=[]) strict-mode rejection on integer index
Adds an openapi test that demonstrates the strict-mode bug where
`match: {"any": []}` on an integer-indexed payload field is rejected with:
Bad request: Index required but not found for "<field>" of one of
the following types: [keyword, uuid]
even though the field is indexed as integer. Root cause is that an empty
`any` list deserializes as `AnyVariants::Strings(empty)` (untagged enum;
Strings variant listed first), and strict mode infers a keyword/uuid
index requirement from the variant tag — ignoring that the list is
empty (i.e. a no-op condition).
The test covers:
- Baseline no-op semantics with and without indexes (passes today).
- Reproduction under strict mode for `must`, `must_not`, and a
FormulaQuery FieldCondition (currently failing; will pass once
empty `any`/`except` no longer requires an index).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: treat empty MatchAny/MatchExcept as no-op in strict-mode index check
An empty `match: {"any": []}` (or `{"except": []}`) is a no-op: `any: []`
matches nothing and `except: []` excludes nothing, regardless of the
field's data type. Because an empty list cannot carry type information it
deserializes as the keyword `AnyVariants::Strings(empty)` variant, which
previously caused strict mode to demand a keyword/uuid index and reject
the request with:
Index required but not found for "<field>" of one of the following
types: [keyword, uuid]
even on a field indexed as integer.
Fix:
- `infer_index_from_any_variants` returns no required index for an empty
variant set.
- `Extractor::update_from_condition` skips a condition whose required
index set is empty, so a no-op condition is never reported as needing
an index (this also covers the FormulaQuery condition path).
This makes the openapi reproduction in test_match_any_empty.py pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* [ai] TQDT in the API
* [ai] unify new sparse error for TQDT
* Rename to `turbo4`
* Add `Turbo4` to comments and doc strings.
* Also validate named sparse vector creation
* fix: validate vector dimensions before WAL write for async upserts
When upserting points with wait=false (the default), dimension
mismatches were silently discarded during background processing.
The API returned 200 "acknowledged" but the points were never stored,
causing silent data loss with no error feedback to the user.
This adds an early dimension validation check in do_upsert_points()
that runs before the operation is written to WAL. This ensures that
dimension errors are returned to the client regardless of the wait
parameter, matching the behavior of wait=true.
The validation handles all vector types:
- Dense single vectors
- Multi-dense vectors
- Named vectors (dense, multi-dense, sparse)
- Sparse vectors are skipped (no fixed dimension)
Closes#9039
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: move vector dimension validation into dedicated module
Extract validate_vector_dimensions and helper functions from update.rs
into src/common/validate_vectors.rs for better code organization.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: update shard update test for early dimension validation
The test expected a shard-level error message, but now dimension
mismatches are caught before reaching the shards. Update the assertion
to accept either the early validation error or the shard-level error.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: assert actual dimension error message in shard update test
Check for the descriptive error ("Vector dimension error: expected dim: 4, got 3")
rather than the generic shard failure wrapper.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add integration test for `match: {except: []}` with integer index
Regression test for https://github.com/qdrant/qdrant/issues/9050
An empty `except` list (NOT IN []) should always match all points that
have the field, both with and without a payload index. Currently the
integer (and keyword) indexed path incorrectly returns zero results
because:
1. serde deserializes `except: []` as `AnyVariants::Strings([])` (first
variant of the untagged enum)
2. The map index filter_impl returns `iter::empty()` for empty
cross-type variant, instead of matching everything
The test covers:
- except: [] without any index (baseline, currently works)
- except: [] with an integer index (currently broken)
- except: [] with a keyword index (currently broken)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix `match: {except: []}` returning zero results with payload index
Fixes https://github.com/qdrant/qdrant/issues/9050
Root cause: `except: []` deserializes as `AnyVariants::Strings([])`
due to the untagged serde enum trying `Strings` first. On an integer
index, the filter_impl matched `Except + Strings(empty)` and returned
`iter::empty()` (zero results). The same issue existed symmetrically
on keyword indexes with `Integers(empty)` and on UUID indexes with
`Integers(empty)`.
The fix: when the cross-type variant reaches the Except branch (e.g.
Strings on an integer index, Integers on a keyword/UUID index), return
`None` unconditionally — regardless of whether the set is empty. This
delegates to the fallback condition checker, which already handles
`except: []` correctly by matching all values.
The `estimate_cardinality_impl` functions had the same bug (returning
`CardinalityEstimation::exact(0)` for the empty cross-type case) and
are fixed the same way.
Affected index types: integer, keyword (str), UUID.
Bool index is NOT affected — it already returns `None` for all
Any/Except conditions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Expand match-except test to cover all index types and cross-type filters
Extends the regression test for #9050 to comprehensively cover:
- Integer, keyword, and UUID field indexes
- Empty except list (the original bug) for each index type
- Non-empty except list with matching types (normal filtering)
- Cross-type except values (e.g. strings on integer index, integers on
keyword/UUID index) — type mismatch should exclude nothing → all match
- Exclude-all: listing every value should return zero results
- All scenarios tested both with and without a payload index
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Raw requests already use QDRANT_HOST, but bypass request_with_validation
and missed QDRANT_HOST_HEADERS. Without these headers, tests cannot run
behind host-based reverse proxies or mocks.
These tests rely on the collection size stats cache being refreshed to
detect that a size limit has been exceeded. Without wait=true, upsert
operations are written to WAL and acknowledged immediately without being
applied to segments. When the cache refreshes, it reads segment data
which may not yet reflect the pending WAL operations, causing the size
check to see stale values and not reject the request.
Adding wait=true ensures operations are applied to segments before the
response returns, so the cache refresh sees the correct sizes.
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The documented format `2023-02-08T10:49` was not accepted because the
parser only had `%Y-%m-%d %H:%M` (space separator) but was missing
`%Y-%m-%dT%H:%M` (T separator). Add the missing format variant and
tests.
Closes#8718
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
* Add empty placeholder vector storage types for named vector CRUD
Introduce EmptyDenseVectorStorage and EmptySparseVectorStorage as
placeholder storages for newly created named vectors on immutable
segments. These report all vectors as deleted, consume no disk space,
and are reconstructed from segment config on load via the new
VectorStorageType::Empty and SparseVectorStorageType::Empty variants.
Key design decisions:
- is_on_disk is derived from original user config, not hardcoded
- MultiVectorConfig is preserved for multi-vector support
- Config mismatch optimizer skips Empty storage to avoid false rebuilds
- Quantization delegates normally (handles 0 vectors gracefully)
- get_vector includes debug_assert to catch unexpected access
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AI] segment-level operations for creating and deleting anmed vectors
* [AI] implement named vector creation and deleting in proxy segment
* [AI] Step 3: Proxy Segment Handling for Named Vector Operations
* [AI] implement for Edge
* [AI] implement consensus operations for named vector operations
* [AI] refactor VectorNameConfig, remove VectorNameConfigInternal
* [AI] handle vector schema inconsistency in raft snapshot recovery
* [AI] rest + grpc API
* [AI] clippy
* [AI] generate openAPI schema
* fmt
* ci fixes
* [AI] fix jwt access test
* [AI] nop operation for awaiting of consensus-commited update ops
* [AI] move vector name operations into points service
* [AI] implement internal api for vector name operations
* [AI] change collection-level config along with segment level operation
* [AI] vector schema reconceliation instead of error
* fmt
* missing compile-time option
* [AI] integration test
* [AI] fix missing JWT tests
* [AI] remove NOP
* [AI] openapi test
* [AI] fix initialization of mutable segment
* [AI] more simple integration tests
* fmt
* [AI] make cluster test a bit harder
* [AI] make test less flacky
* [AI] rabbit comments
* [AI] check params compatibility before writing vector config
* [AI] make sure to register vector storages in structure payload index
* [AI] vector name validation
* lower vector length validation to 200 chars to account for prefix in filename
* [AI] proxy segment: prevent stale data leak through optimization
* fmt
* [AI] filter out removed vectors from proxy response
* [AI] handle vector name in proxy
* fmt
* adjust proxy info based on dropped vectors
* [AI] proxy segment: update filters to correct has_vector condition
* fmt
* clippy
* Fix consensus snapshot applicaiton for vector schema
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix empty sparse vector name validation
* Reorder validation to check empty sparse name before duplicate name
---------
Co-authored-by: leohenon <77656081+lhenon999@users.noreply.github.com>
* feat: add search_max_batchsize to strict mode config
* added test case for search_max_batchsize
* Changes for fixing CI issue dure openapi
* Modify check_strict_mode_batch
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Don't account for deferred points in some places
# Conflicts:
# lib/collection/src/shards/local_shard/scroll.rs
* Add in QueryContext
* Cover more places
* Coderabbit review remarks
* Properly count amount of deleted deferred points (#8386)
* Properly count amount of deleted deferred points
* Prevent double-counting of the same point
* Remove hints to estimations
* Properly handle counts in ProxySegment
* Add tests and fix deleted point count issue
* Adjusts tests + fix issues
* Separte fields for deferred points in telemetry (SegmentInfo)
* Remove deferred_points_count()
* openapi
* Fix test by manually calculating visible points
* Adjust ProxySegment test to revertion of SegmentInfo
* Throw error if collection was not found in telemetry (e2e Test)
* Update lib/segment/src/segment/segment_ops.rs
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Make new fields in API optional
* Don't take range if no deferred point exist
* Review remarks
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* restrict recovery from snapshot folder
* fmt
* clippy
* Extend test, path must be inside snapshots directory
* In release builds, hide detailed tar error message to not leak contents
* Change from bad request to forbidden
* fix(auth_tests): place snapshot in peer snapshots dir for recover_collection_snapshot
The recover from snapshot API now requires file:// paths to be inside the
configured snapshots directory. Write the test snapshot into the peer's
snapshots directory instead of a temp file so the request is allowed.
Made-with: Cursor
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Cursor Agent <agent@cursor.com>
* [manual] make untagged enum for consistent API for FeedbackStrategy
* add #[validate(nested)]
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Add integration test to assert all queued updates are also transferred
* Plunge update queue in stream records transfers
* Migrate existing plunger usages to new plunge helper
* Skip test if not compiled with staging flag
* Only send delay operation when staging feature is enabled
* Reformat
* Fix review remarks
* add rest and grpc interfaces
Also handle inference for this query
* follow refactor from base branch
* renaming from Ms Cooper
* get started on validation testing
* more validations
* test equivalence with query when less than 2 feedback elements
* rename feedback query to relevance_feedback query
* fix extraction of context pairs
* rename feedback vector to example
* make coderabbit happier
* upd test
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* handle `score_threshold` in Formula queries
* AI: add openapi test
prompt: Upload 4 points with a numeric payload, use the payload as the
score, and set a score threshold. We can assert which ids should be in
the result and which shouldn't
* weighted rrf implementation
* test
* fmt
* fix edge
* validate number of sources and number of weights
* do not partial match
* upd schema
* review fixes
* update formula
* remove calcualtions from tests
* update comment, because AI have OCD
* fmt
* Update queue info into CollectionInfo
* basic test
* fix
* cleaner
* Update lib/collection/src/shards/local_shard/mod.rs
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* fix the fix
* rename to op_num
* skip_serializing_if just in case
* first step
* test queue length with staging feature
* gate integration test based on binary feature
---------
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
* introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert
* add test
* upd dockstring
* require resharding once all peers have updated version
* use service error
* fix clippy again
* wait for same version before resharding in tests
