* [TQDT] Align TQ vector storage layout with the reference dense storage
Make the TurboQuant vector storage structurally mirror the reference dense
(and multi_dense) storages, down to file names and their contents:
- Rename files + structs to the dense convention:
- immutable.rs -> turbo_vector_storage.rs (ImmutableTurboVectorStorage ->
TurboVectorStorageImpl)
- appendable.rs -> appendable_turbo_vector_storage.rs
(AppendableTurboVectorStorage -> AppendableMmapTurboVectorStorage)
- multi.rs -> multi_turbo/appendable_mmap_multi_turbo_vector_storage.rs
(TurboMultiVectorStorage -> AppendableMmapMultiTurboVectorStorage)
- ReadOnlyTurboMultiVectorStorage -> ReadOnlyChunkedMultiTurboVectorStorage
- Thin out turbo/mod.rs to module declarations + re-exports: open_* fns move
into their storage files, consts + turbo_storage_roundtrip into shared.rs,
and TurboScoring / TurboMultiScoring join the other TQ traits in
vector_storage_base.rs.
- Split read_only/ into the chunked storage (read_only/) and the single-file
storage (read_only/immutable/), each with the mod/lifecycle/live_reload/
read_ops 4-file layout, mirroring dense/read_only/.
- Introduce multi_turbo/ mirroring multi_dense/, with its own read_only/
submodule holding ReadOnlyChunkedMultiTurboVectorStorage.
- Relocate the storage test suites to
tests/test_appendable_turbo_vector_storage.rs and
tests/test_appendable_multi_turbo_vector_storage.rs, paralleling the
dense/multi_dense integration test files (tests moved verbatim, no new
tests added).
- Fix a gpu-gated VectorStorageEnum match that referenced stale DenseTurbo /
DenseTurboAppendable variant names.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* are you happy fmt
* fix after rebase
* [TQDT] Address review feedback on TQ vector storage split
- gpu tests: use the real `VectorStorageEnum::DenseTurboAppendableMemmap`
variant (the old `DenseTurboAppendable` name never existed post-rename, so
the gpu-feature test failed to compile — missed because `cargo build
--features gpu` does not compile the `#[cfg(test)]` code).
- memory_reporter: report `DenseTurboUring` files as `FileStorageIntent::OnDisk`
like the other io_uring variants; io_uring never mmap-caches, so delegating
to `is_on_disk()` could wrongly report `Cached` for a populated backend.
- turbo_vector_storage: fix the misleading `insert_tq_bytes` doc comment — the
single-file backend rejects the upsert via `?`, so `set_deleted` is never
reached.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [TQDT] Fix clippy::wildcard_enum_match_arm in read-only routing test
Spell out the non-routing `VectorStorageType` variants instead of `_`, so a
future added variant fails the match rather than silently mapping to `false`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [TQDT] Fix stale Turbo4 storage-variant assertion in quantization test
The segment is built with the default (appendable/chunked) storage type, so a
Turbo4 datatype now lands in `DenseTurboAppendableMemmap`, not the single-file
`DenseTurboMemmap`. The assertion was left on the pre-split variant; align it
with the non-turbo branch, which already expects the appendable variants.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* are you happy fmt
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Support memory placement in service-level storage config defaults
Follow-up to #9684: `storage.payload.memory` and
`storage.collection.vectors.memory` set service-wide placement defaults
for newly created collections, deprecating `storage.on_disk_payload` and
`storage.collection.vectors.on_disk`.
Defaults resolve as: request `memory` > request legacy flag > service
`memory` > service legacy flag; exactly one level is filled to avoid
spurious memory-vs-legacy mismatch warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Mark hnsw_index.on_disk deprecated in config.yaml, document memory option
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add shared qdrant-edge-ffi crate with UniFFI bindings
Introduce a shared FFI crate that wraps Qdrant Edge's core types with
UniFFI attributes so the same Rust source can power both the Swift and
Kotlin bindings. The crate lives at lib/edge/ffi/ and exposes ~60 public
types (EdgeShard, Point, Query, Filter, UpdateOperation, …) plus their
enum / record / sealed-union variants.
- lib/edge/ffi/src/ UniFFI-wrapped domain types (config, types,
filter, query, update, error, lib)
- lib/edge/ffi/bindgen/ Separate crate housing the uniffi-bindgen CLI
(needed so consumers of the `uniffi` runtime
don't have to depend on its CLI feature)
- lib/edge/ffi/uniffi.toml Sets the generated Kotlin package to
tech.qdrant.edge.ffi (Swift uses the crate
name verbatim)
Every public type carries Rust doc comments that UniFFI propagates to
Swift Quick Help and Kotlin KDoc, so the generated bindings ship with
first-class IDE documentation.
Workspace changes:
- Cargo.toml Adds the new crates as workspace members and
introduces a `release-mobile` profile
(thin LTO, codegen-units=1, strip=symbols,
panic=abort) for size-conscious mobile builds
- Cargo.lock Pins uniffi 0.31 and its transitive dependencies
Made-with: Cursor
* fix(edge-ffi): harden FFI boundary, add tests, quantization parity, optimize/HNSW
Builds on @ivan-afanasiev's qdrant-edge-ffi foundation (preceding commit) — takes
it to a tested, safe, reviewable state. Split out of #9359 per maintainer request
so the FFI crate can be reviewed in isolation; Swift/Android SDK PRs stack on top.
Boundary safety (host input → catchable error, never a process abort):
- release-mobile profile switched to `panic = "unwind"` so UniFFI's catch_unwind
turns a panic into a catchable error (abort would risk WAL/segment consistency
on an on-device DB).
- Fallible boundary conversions reject bad input (UUID, geo, JSON path, payload
JSON, contradictory match filters) with InvalidArgument instead of panicking.
- Host-supplied counts bounded: limit/offset (bounded_limit, 1 Mi cap), vector
size (1..=65536), HNSW params (m/payload_m ≤ 2048, ef_construct 4..=100000,
max_indexing_threads ≤ 1024) — these drive eager allocation / thread spawning
at optimize(), so unbounded values would abort uncatchably.
API:
- Quantization parity with the Python Edge SDK: all four strategies
(Scalar/Product/Binary/Turbo) accepted; HnswIndexConfig + optimize() exposed
(without optimize() search is brute-force).
- config() is an honest "as-requested" read-back (HNSW + quantization round-trip).
- EdgeError stays branchable (ShardClosed / InvalidArgument / OperationError;
field is `reason`, not `message`, to avoid the Kotlin Throwable collision).
edge core (required by the boundary):
- EdgeShard::flush is fallible (OperationResult) instead of panicking on lock
contention; Drop logs a flush error instead of aborting; python flush()? updated.
- scroll.rs drops a with_capacity(limit) pre-alloc a huge limit could turn into an
allocator abort (defense-in-depth alongside bounded_limit).
Tests (CI: cargo +nightly test -p qdrant-edge-ffi): 4 unit + 22 conversion +
18 integration — persistence, crash-recovery, payload round-trip, concurrency,
delete-reload, search ranking, scroll pagination, quantization accept + config
round-trip, HNSW optimize, boundary rejection.
* fix(edge-ffi): validate geo radius/rings and reject empty field conditions
Three boundary-validation gaps surfaced in review of #9374, all rejected
now with EdgeError::InvalidArgument instead of producing wrong results or
reaching a panic in the geo index:
- GeoRadius: a negative or non-finite radius passed straight through to
the geo index. Reject !is_finite() || < 0.0.
- GeoLineString rings (exterior + interiors of a GeoPolygon): the segment
type was built by direct struct literal, bypassing the engine's
validate_line_string (which only runs on the serde path). A malformed
ring (<4 points or unclosed) could panic in the geo index on indexed
payloads. Mirror validate_line_string at the single GeoLineString
conversion chokepoint, covering both exterior and interior rings.
- FieldCondition: a condition with no predicate set is a silent no-op
(matches every point). Reject it, mirroring the engine's
validate_field_condition. This is the engine/gRPC/REST/Python contract
of "at least one" predicate -- NOT "exactly one"; multiple predicates
remain valid and AND together. The doc comment is corrected accordingly.
Adds 9 conversion tests (geo radius negative/NaN/infinite/valid, ring
too-few/unclosed/bad-interior, field-condition no-predicate/multiple).
cargo +nightly test -p qdrant-edge-ffi: 53 green.
* fix(edge-ffi): adapt to memory/idf API and fallible info after rebase
Keep FlushMode::Sync from recent segment-holder changes while preserving
fallible flush. Fill newly required memory/idf fields (matching the Python
edge bindings) and propagate EdgeShard::info()'s OperationResult.
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci(edge): disable checkout credential persistence in edge-test
actions/checkout persists GITHUB_TOKEN into .git/config by default. This
workflow runs on pull_request from any branch and then builds and runs
repository-controlled code (Rust/Python examples), which could read or
exfiltrate that token. The job never pushes, so drop the persisted
credentials with persist-credentials: false.
Addresses a CodeRabbit security finding on the edge-ffi PR.
* fix(edge): block on lock in flush instead of failing on contention
flush() used try_lock()/try_read() and returned a 'lock busy' OperationError
when a concurrent update/optimize held the WAL or segment lock. That branch is
the wrong trade-off: callers of flush() expect their data persisted, and the
one spot it would fire on the direct Rust path is exactly when an in-flight
update is holding the WAL lock across its whole operation — i.e. when there is
unflushed data most worth persisting. At the FFI boundary it is moreover dead
code, since the outer Mutex<Option<EdgeShard>> already serializes every call.
Switch to blocking lock()/read(), matching the semantics update() and optimize()
already use on these same locks. flush() stays fallible so a genuine WAL/segment
I/O error is still surfaced rather than panicking. Drop cannot contend (it needs
&mut self, so no &self borrow can hold the locks), so it will not hang.
Addresses a CodeRabbit review nitpick on the edge-ffi PR.
* fix(edge-ffi): harden vector/query boundary from multi-agent review
Addresses findings from a multi-agent review of the FFI boundary:
- Multivector conversions were infallible: an empty outer Vec panicked in
release (MultiDenseVectorInternal::new_unchecked only debug_asserts) and a
ragged matrix was silently reshaped against row[0].len(), storing data the
host never sent. Make NamedVector/Vector -> persisted conversions TryFrom and
validate the matrix (non-empty, uniform non-zero row width) with the same
rules as the engine's try_from_matrix, returning InvalidArgument.
- Reject non-finite (NaN/inf) vector components at ingest, mirroring the geo
is_finite guard. The engine validates only dimensionality, so a poisoned
component would be stored and later serialized back as JSON null silently.
- unload() now returns Result: it flushes explicitly and surfaces a final fsync
failure instead of only reaching Drop's log line (no default log sink exists).
On error the shard stays loaded so the host can retry.
- Cap filter/prefetch nesting depth (MAX_QUERY_NESTING_DEPTH). Condition::Filter
and nested Prefetch are self-recursive; an unbounded host tree would overflow
the stack — a SIGABRT that panic=unwind cannot catch. Reject deeper trees as
InvalidArgument via depth-threaded conversion helpers.
- Replace the '""'-on-serialization-failure fallback in payload/vector JSON
encoding with .expect (serialization is infallible; fail loud, not silent
invalid JSON).
- Docs: correct the flush() # Errors (can return OperationError), the edge-core
flush() caller enumeration, upsert_points/update_vectors # Errors (vector
validation), and reword the fictional lib/edge/VERSION / version-sync comment
in ffi/Cargo.toml to reflect that no automated check exists yet.
* test(edge-ffi): add behavior coverage for vectors, filters, query, updates
Adds 18 integration tests closing gaps a multi-agent review flagged (the suite
proved type conversion but not behavior):
Safety (back the new boundary validation):
- multi_vector_invalid_matrices_rejected — empty/ragged/zero-dim multi-vectors
- non_finite_vector_components_rejected — NaN/inf across single/named/multi/sparse
- finite_vectors_accepted_by_upsert_constructor — over-rejection guard
- deeply_nested_filter_rejected_shallow_accepted, deeply_nested_prefetch_rejected
— depth cap rejects >64, accepts shallow
Behavior:
- filter_restricts_count_scroll_and_search — a filter actually narrows the result
set across count/scroll/search (not just that conversion succeeds)
- flush_under_concurrent_upserts — flush() blocks under a concurrent update loop,
no panic, final count == successful upserts
- vector_content_round_trips_through_retrieve_and_search
- cosine_distance_ranks_by_direction, euclid_and_manhattan_rank_nearest_first
- delete_points_by_filter / update_vectors / delete_vectors / delete_payload /
clear_payload — the five previously-untested update ops
- multivector_round_trips, sparse_vector_round_trips
- rrf_fusion_over_prefetches_returns_fused_set
Suite: 4 unit + 32 conversion + 36 integration = 72, all green.
Not covered (blocked by FFI surface, tracked for follow-up): facet() and OrderBy
scroll both need a payload index, and UpdateOperation exposes no create-index
constructor.
* fix(edge): avoid lost-update TOCTOU in set_vector_hnsw_config
set_vector_hnsw_config read().clone()'d the config, mutated the clone, then
write(|c| *c = cfg) overwrote the whole config. The read lock is released before
the write, so a concurrent config update between the two is silently discarded
(lost-update TOCTOU). Use SaveOnDisk::write_optional to run the fallible mutation
on a clone inside the held lock, returning None on failure to abort persist+swap
without overwriting — the atomic pattern the repo's SaveOnDisk learning
prescribes for fallible config mutations.
Addresses a CodeRabbit critical finding on the edge-ffi PR.
* fix(edge-ffi): adapt read requests to edge::* structs after #9901 rebase
#9901 (Edge: request-specific structures for EdgeShardRead) moved the read API
off the shard/core request types onto edge-owned request structs. Retarget the
FFI conversions accordingly:
- QueryRequest/SearchRequest/CountRequest/ScrollRequest/FacetRequest/Prefetch
now convert into edge::{QueryRequest,SearchRequest,CountRequest,ScrollRequest,
FacetRequest,Prefetch} (were ShardQueryRequest/CoreSearchRequest/*Internal).
Field shapes are identical except score_threshold, which is a plain ScoreType
(f32) on the edge structs, not OrderedFloat — drop the wrap. Nesting-depth
guard and bounded_limit validation preserved.
- retrieve() builds an edge::RetrieveRequest and calls the new single-argument
EdgeShardRead::retrieve (was a 3-arg call).
- Cargo.lock reconciled onto dev's lockfile + the FFI/uniffi deps (dev advanced
23 commits incl. dependency bumps).
cargo +nightly test --locked -p qdrant-edge-ffi: 4 unit + 32 conversion + 18
integration all green.
* feat(edge-ffi): full engine coverage — restructure, all update ops, all scoring queries, missing shard methods
Restructure the crate to mirror the edge crate's layout: shard.rs owns the
EdgeShard object and lifecycle, ops/ has one file per read operation
(request/response records + conversions + exported method together), and
update construction stays in update.rs. Multiple #[uniffi::export] impl
blocks merge cleanly in the generated bindings.
Interface modernization:
- retrieve() takes a RetrieveRequest record mirroring edge::RetrieveRequest
- config surface moves off the deprecated always_ram/on_disk booleans to a
Memory placement enum (cold/cached/pinned); read-back resolves legacy
flags via memory_placement(), None-preserving for HNSW
- EdgeShard.inner is RwLock<Option<...>>: operations take the read half and
run in parallel; unload/update_from_snapshot take the write half and
drain in-flight requests
- request/config records carry #[uniffi(default = ...)], so generated
Swift/Kotlin constructors get default arguments (count exact=true, facet
limit=10, HNSW 16/100/10000/0, everything optional defaults to nil/null)
- all conversions destructure their source exhaustively; intentionally
unexposed internal fields are named `field: _` with a why-comment
Full update-operation coverage (Python SDK parity):
- upsert_points gains condition/update_mode (conditional upsert),
update_vectors gains condition, set_payload gains a JSON-path key
- new: delete_vectors_by_filter, set/overwrite/delete/clear payload
by-filter forms, overwrite_payload, create/delete_field_index (with a
PayloadSchemaType enum), create_dense_vector, create_sparse_vector,
delete_vector_name
Full scoring-query coverage:
- Query::Nearest takes a NamedVector (dense/sparse/multi-vector search)
- new Query variants: Recommend (BestScore/SumScores), Discover, Context,
Feedback; new ScoringQuery variants: Formula (recursive Expression
object with validating, depth-capped constructors) and Mmr;
Fusion::Rrf gains weights
Missing shard methods: query_groups, search_matrix, create, path,
snapshot_manifest, update_from_snapshot (full + partial recovery),
set_hnsw_config, set_vector_hnsw_config, set_optimizers_config.
Two compile-time coverage maps (update.rs, ops/query.rs) exhaustively
match the engine's operation/query enum trees with no wildcard arms, so a
new engine variant fails compilation in this crate until the FFI surface
decision is recorded. A staging passthrough feature keeps them exhaustive
when shard/staging is enabled. The scoring-query map immediately caught
the otherwise-missed Mmr variant.
Tests: 82 total (4 unit + 32 conversion + 46 integration), including new
end-to-end coverage for field-index-enabled facet, conditional upsert,
overwrite/clear-by-filter, recommend/discover/context/MMR, formula
re-scoring, grouped queries, search matrix, vector-name ops, and the
lifecycle additions. Kotlin+Swift binding generation verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(edge-ffi): cover the remaining enum boundaries — filter tree, formula, schema, selections
Add coverage maps for the four construction-only enum families that had no
exhaustiveness guard, and expose everything they revealed as missing:
Filter surface (the big one — filter.rs map over Condition/Match/
RangeInterface/AnyVariants/ValueVariants):
- Condition::Slice — deterministic id-space slice filter, with the
serde-path total/index validation re-applied at the boundary
- Condition::Nested — nested-object array filters, depth-counted against
the recursion cap
- Match::TextAny / Match::Phrase / Match::Prefix — the three previously
unreachable text-match modes
- FieldCondition.datetime_range — RFC 3339 datetime ranges (mutually
exclusive with the float range, matching the engine's single
RangeInterface slot)
- Filter.min_should
- WithPayload::Exclude — exclude-style payload selection (types.rs map
over WithPayloadInterface/PayloadSelector/WithVector)
- OrderBy.start_from — Integer/Float/Datetime cursor (mapped in the
scoring-query coverage map)
- CustomIdChecker recorded as intentionally unexposed (serde-skipped,
runtime-internal)
Formula (formula.rs map over ExpressionInternal + DecayKind): all 17
expression variants were already constructible; the map now forces a
decision when the engine grows a new one.
Field-index schema and vector-name config (update.rs map extended):
PayloadFieldSchema::FieldType covered per schema type; the per-type
FieldParams forms recorded as not exposed yet; VectorNameConfig
Dense/Sparse tied to their constructors.
Tests: 91 total (+7 conversion, +2 integration: slice partitioning and
payload-exclude retrieval). Kotlin binding generation verified for the
new types.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(edge-ffi): resolve clippy warnings (inline format arg, large enum variant)
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(edge-ffi): address multi-agent review of the full-coverage expansion
Fixes findings from a multi-agent review (each independently re-verified),
targeting the new surface added in the full-engine-coverage expansion. All
verdicts CONFIRMED except query_groups (refuted → no change). 100 tests green.
- formula: cap Expression node count (MAX_FORMULA_NODES = 10_000), not just
depth. Expression is a host-held Arc; combinators eagerly deep-clone their
children's inner tree, so reusing one handle as several children
(sum([e, e]) repeated) grows node count as 2^depth while depth stays under
the 64 cap — an eager multi-GB clone that aborts the process (uncatchable
under panic=unwind). Track a saturating size in node() and decay(); the
depth guard stays (it protects the stack for narrow-but-deep chains).
- sparse: validate host sparse vectors at the boundary via
validate_sparse_vector_impl (indices.len == values.len, unique indices).
Without it a length mismatch reached an out-of-bounds panic at insert/scoring
(opaque caught-panic, not a typed error) and duplicate indices silently
double-counted. Route both sparse arms through a fallible helper; the
infallible From<SparseVector> is removed so no path can bypass validation.
- order_value: surface it on ScoredPoint/Record (new OrderValue { Int, Float },
matching the Python edge SDK). It was dropped behind a comment claiming "no
order-by-scored surface" — false: order-by query/scroll and OrderBy.start_from
are exposed, and ordered results carry a constant score and no next_offset, so
order_value is the only way to resume ordered pagination when payload is off.
- search_matrix: drop from v1. It is an O(n^2) analytics op the reference Python
edge SDK does not expose; its flat 1-Mi bounds are the wrong shape (one call
hangs/OOM-crashes the device on a normal shard). Removing pre-publish is free;
re-adding later with proper caps is non-breaking, the reverse is not.
- tests: node-count reject, sparse len/dup reject, order_value populated +
start_from resume, and snapshot negative tests (bad path / corrupt archive /
post-unload surface OperationError, not a panic, and the shard survives).
* fix(edge-ffi): reject oversized formula before the eager clone; close test gaps
Follow-up from a re-review of the fix commit.
- formula: the node-count check ran AFTER the eager `inner` deep-clone, because
the clone was an argument to `node()` (evaluated before the function body). A
host could build one accepted handle and reuse it as many children in one call
(sum(vec![e; N])), materializing N x e.size nodes before the size check could
reject it — the same uncatchable OOM abort the cap was meant to prevent, in two
calls. `node()` now takes the children handles plus a build closure and runs
both caps BEFORE invoking it, so a rejected tree never clones (decay() already
did this). All combinators route through it.
- tests: decay node-count reject + happy path (decay has its own guard, was
untested); wide-fanout reject (regression for the ordering fix); query-side
ScoringQuery::OrderBy -> ScoredPoint.order_value (only the scroll/Record side
was covered); real multi-page order-by continuation via StartFrom (was a
single-page degenerate case); sparse values-longer reject; float OrderValue;
with_payload=false omits payload. Suite: 107 green (4 + 39 + 64).
* feat(edge-ffi): keep search_matrix behind an off-by-default `matrix` feature
Reconsidered the outright removal: the FFI crate is a general UniFFI boundary,
not mobile-only, so non-mobile consumers (desktop/server/Rust) may want the
distance-matrix op. Instead of deleting it, gate the whole `ops/matrix.rs`
module behind a new off-by-default `matrix` Cargo feature.
- The mobile Swift/Kotlin bindings build without the feature, so search_matrix
stays off the mobile surface (verified: default `cargo test` excludes it and
its test; `--features matrix` includes both).
- The O(n^2) DoS is documented, not capped here: the op is opt-in and off the
constrained mobile surface, so bounding sample_size/limit_per_sample for a
given deployment is the enabling consumer's / SDK layer's responsibility. The
module doc spells this out; the per-field bounded_limit only stops a lone
u64::MAX value, not the quadratic compute.
- CI: add an `--all-features` test leg so the feature-gated surface (matrix +
the pre-existing staging passthrough) can't bit-rot — this also closes the
build-publish review note that `staging` had no CI coverage.
* docs+test(edge-ffi): tighten matrix doc wording; pin formula node-count boundary
Non-blocking polish from a final all-reviewer pass (6/6 APPROVE):
- matrix.rs / Cargo.toml docs: (1) the per-field bounded_limit caps at
MAX_RESULT_COUNT (1,048,576), not merely a lone u64::MAX — say so, since 1 Mi
is itself catastrophic for an O(n^2) op; (2) the DoS bound is owned by the
opting-in consumer, not an "SDK/wrapper layer" that need not exist for a raw
UniFFI consumer; (3) the mobile bindgen (a follow-up PR) must build with
default features to keep the op off the mobile surface — stated as intent, not
present-tense fact (no swift/android dirs exist yet).
- formula_node_count_exact_boundary: pins the exact MAX_FORMULA_NODES threshold
(10_000 accepted, 10_001 rejected) — the existing tests jumped to ~16k, so the
precise cap edge was unverified.
Suite: 108 default / 109 --all-features, all green.
* feat(edge-ffi): make the matrix feature on by default
Flip `matrix` to on-by-default (`default = ["matrix"]`). General (desktop /
server-side / Rust) UniFFI consumers now get `search_matrix` without opting in;
the mobile Swift/Kotlin bindgen builds with `--no-default-features` to drop the
O(n^2) analytics op from the mobile binding surface.
CI now covers all three shapes: default (matrix on), `--no-default-features`
(mobile surface, matrix excluded — guards the crate still compiles/passes
without it), and `--all-features` (matrix + staging). Verified: default 66 /
no-default 65 / all-features 66 integration tests, all green; no Cargo.lock
drift.
* fix(edge-ffi): rustfmt the sparse validator; correct stale matrix-feature docs
From a full all-reviewer pass of the review fixes:
- types.rs: rustfmt the `to_internal_sparse` `.map_err` chain. It failed
`cargo +nightly fmt --all -- --check` (the rust-lint.yml CI gate) — the
edge-test legs only run `cargo test`, so it slipped through locally.
- ops/mod.rs + integration.rs: fix two doc comments that still said the `matrix`
feature is 'off-by-default' after it was flipped on-by-default. Left uncorrected
they'd mislead the follow-up mobile-bindgen author into skipping
`--no-default-features` and shipping the O(n^2) op to phones.
Design decisions (recorded): matrix stays on-by-default; the O(n^2) DoS is
documented, not capped — a static cap can't know the target device (compute cost
is device-dependent; the caller owns it via async/timeout). No cap added.
Reviewers: 3 APPROVE, 3 CHANGES-REQUESTED, all CR items were these two doc/fmt
misses plus the recorded design calls. All 3 feature configs green (66/65/66).
* fix(edge-ffi): resolve clippy errors in FFI tests
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(edge): reject conflicting vector-name re-create instead of desyncing config
The segment-level `create_vector_name` is idempotent: re-creating an existing
name is a no-op that leaves storage untouched. `EdgeShard::update` treated any
`Ok` from that no-op as success and unconditionally re-applied the requested
params to the shard config, so a second `create_dense_vector("v", 8, Cosine)`
after `("v", 4, Dot)` left `config()` advertising 8/Cosine while storage kept
4/Dot — a shape the shard then rejects on upsert. The empty name "" (the
default vector) hit the same path against a single-vector shard's primary field.
Reject a conflicting re-create up front (before the WAL, so it can't brick
replay) with a clear error, accept an identical re-create as idempotent, and
only re-apply config when storage actually changed.
* fix(edge-ffi): harden FFI boundary from an adversarial per-search-type review
Boundary-validation and doc fixes surfaced by fuzzing each query type with
executed repros:
- Reject non-finite floats that JSON cannot represent but the raw-f64 FFI can:
order-by `StartFrom::Float` (a NaN panicked on a float-indexed field and
silently truncated an integer scan), and formula `decay` midpoint/scale +
`div` by_zero_default (a NaN evaded the engine's comparison-based range
checks → debug panic across the boundary / silent all-zero rescore).
- Drop the `key` parameter from `overwrite_payload`/`overwrite_payload_by_filter`:
the engine has no payload selector on the overwrite path (the server discards
it too), so a keyed overwrite silently replaced the whole payload.
- Reject a `FieldCondition` carrying more than one predicate: the engine has no
defined semantics for it (it evaluates one, index-dependent), so passing
several through diverged silently from a Qdrant server. Callers AND predicates
via separate `must` conditions.
- Doc fixes: FeedbackCoefficients b/c (exponent/multiplier, not weight/margin),
RecommendStrategy::BestScore default, div-by-zero behavior, retrieve
duplicate-ID collapsing, non-atomic partial-snapshot recovery.
Also fix the clippy `--all-targets -D warnings` lints in the test files that
reddened CI's `lint` job (uninlined_format_args, err_expect, disallowed
std::fs::write) and add regression tests for the validations above.
* fix(edge): compare only vector identity fields when detecting re-create conflicts
The conflicting-re-create guard added in the previous commit compared the full
`EdgeVectorParams`/`EdgeSparseVectorParams` via `PartialEq`, but a vector
declared in the initial `EdgeConfig` is stored with `on_disk: Some(false)` (from
`from_vector_data_config`) while a `CreateVectorName` op leaves it `None`. So an
identical re-create of a construction-defined vector — or of any vector after
`set_vector_hnsw_config` — was falsely rejected as a conflict, a regression on
the idempotent path.
Compare only the identity fields the op actually defines (dense: size, distance,
multivector_config, datatype; sparse: modifier, datatype); the storage/tuning
fields it cannot express are set from the config, the optimizer, or
`set_*_config` and must not trigger a false conflict. Adds a regression test
re-creating the config-defined "vec".
* style: apply cargo fmt
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(edge-ffi): expose payload_schema in info() and parameterized index creation
Close the last interface gap in the FFI surface: `info()` previously
elided the engine's `payload_schema`, and `create_field_index` only
accepted a bare type.
- New `payload_index` module mirroring the full `PayloadSchemaParams`
family (all 8 index types incl. tokenizer/stopwords/stemmer options)
with bidirectional conversions.
- `ShardInfo.payload_schema` reports each index's type, creation params,
and indexed point count.
- New `UpdateOperation::create_field_index_with_params` constructor;
coverage map updated accordingly.
- Boundary normalizations, following the quantization-config precedent:
deprecated `on_disk` folds into the reported `memory` placement, and
contradictory integer params (lookup+range both off) reject with
`InvalidArgument`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Ivan Afanasiev <ivan.afanasiev@yahoo.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Add doc comments to `StartResharding` fields so the generated OpenAPI
spec explains what a user has to pass, and regenerate the spec.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(segment): support Turbo4 in the read-only vector storage path
Read-only segments (UniversalRead-backed: mmap / cache / remote) could not
open a `Turbo4`-typed vector storage — every dispatch arm returned "not yet
supported". This adds the missing read-only TurboQuant storages so read-only
segments can retrieve and score TQ-typed vectors.
- Reuse the existing, correct TQ scoring: extract `TurboScoring` /
`TurboMultiScoring` traits and generalize `TurboQueryScorer` /
`TurboCustomQueryScorer` (+ multi) and `raw_turbo_*_scorer_impl` from the
concrete `TurboVectorStorage` to `&impl TurboScoring`. Scoring logic is
untouched.
- Split `DenseTQVectorStorage` / `MultiTQVectorStorage` into read-only
(`*Read`) + write supertraits, mirroring the dense storages, so the
read-only storage need not implement the ingest path.
- Add `ReadOnlyTurboVectorStorage<S>` (single-file + chunked backends) and
`ReadOnlyTurboMultiVectorStorage<S>` over `QuantizedStorage<S>` /
`QuantizedChunkedStorageRead<S>` + `InMemoryBitvecFlags`; the quantizer is
rebuilt from `(dim, distance)`, so nothing beyond the encoded bytes and
flags is read from disk.
- Wire `DenseTurbo` / `MultiDenseTurbo` variants into `VectorStorageReadEnum`
(dispatch, scorer, live-reload) and replace the `Turbo4` stubs in the
read-only `open` / `preopen`.
- Round-trip tests: writable TQ -> reopen read-only, asserting byte-exact raw
TQ bytes, identical dequantization, deletion flags and per-point scores
matching the writable scorer (dense single-file + chunked, and multi).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* review remarks
* Split turbo read_only.rs into a module
Mirror the layout of the other read-only vector storages: split the
oversized read_only.rs into mod.rs (struct definitions + shared encoded
backend enum), lifecycle.rs (preopen/open), read_ops.rs (retrieval +
scoring trait impls) and live_reload.rs.
Co-authored-by: Cursor <cursoragent@cursor.com>
* review remarks
* fix after rebase
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
The turbo model tests, HNSW graph properties tests, and id-tracker
mapping tests generate their datasets with StdRng (ChaCha12 in rand
0.10). Switching to SmallRng (Xoshiro256++) makes the turbo model tests
~10-14% faster and the 400k-mapping id-tracker test ~30% faster.
All affected tests pass with the changed seeded sequences, including
the tolerance-carrying turbo model comparisons.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* make `EncodedStorage::for_each_batch` mandatory
* make `DenseVectorStorageRead::for_each_in_dense_batch` mandatory
* make `DenseTQVectorStorage::for_each_in_dense_batch` mandatory
* make `DenseTQVectorStorage::read_dense_tq_bytes` mandatory
* make `QueryScorer::score_stored_batch` mandatory
...and implement for tq multivectors
* [AI] make `IdTrackerRead::internal_versions_batch` mandatory
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [AI] make `IdTrackerRead::external_ids_batch` mandatory
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [AI] make `DiskMappingsSource::resolve_internal_batch` mandatory
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`load_from_wal` splits WAL recovery in two: it replays
`[first_index, applied_seq + APPLIED_SEQ_SAVE_INTERVAL + 1)` synchronously and
hands the remaining tail to the update worker, which applies it in the
background *after* `LocalShard::load` has returned and the shard has started
serving reads.
That split was introduced by #8008 and applies to every collection, so up to
`update_queue_size - 1` operations already acknowledged to a client with
`wait=true` can be missing from reads right after a restart, reappearing one by
one as the worker catches up.
Only `prevent_unoptimized` needs that routing: the update worker signals the
optimizer per operation, and optimization is the only thing that makes deferred
points visible. Everywhere else the synchronous replay is sufficient, so gate
the use of `applied_seq` on the flag -- the same condition that already gates
the worker's deferred-points wait -- and replay the whole WAL before load
returns, as it did before #8008.
Found by the crasher: after a crash-restart cycle it reported 72 missing points
out of a confirmed 3202, with the shard counting 3202 points while 3930 had been
acknowledged and 332 WAL entries were still queued across two shards.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ConditionChecker::check_batched: trait + ConditionCheckerEnum
* check_batched for OptimizedFilter
* OnDiskPointToValues::values_iter_batch: improve performance
* OnDiskPointToValues::values_iter_batch: update interface
Accept bitvec, call on every point, pass UserData.
* check_batched for geo index
* geo_index tests: add same_geo_index_between_points_with_dups_test
Catches broken load_from_on_disk/for_all_points_values.
* check_batched for map index
* check_batched for numeric index
* check_batched for full-text index
* tests for check_batched
* Review fixups
- default_check_batched: avoid running pred twice at boundary
- condition_checker: use explicit match over matches!/if-else
- Partitioner: use assert! over debug_assert!
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* ConditionChecker::check_batched: &mut self -> &self
* ConditionChecker::check_batched: make mandatory
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
A transfer source that misses the transfer abort (e.g. while partitioned
or paused) keeps its local shard wrapped in a proxy. When such a peer can
only catch up via consensus snapshot, snapshot application re-creates
payload indexes with an update operation that the stale forward proxy
forwards to a transfer target which may no longer have the shard. The
resulting precondition error fails snapshot application and stops the
consensus thread ("No target shard N found for update"), leaving the
peer unable to ever catch up.
Snapshot application now explicitly cleans up transfers that are no
longer registered in consensus: the transfer task is stopped and the
proxy is reverted via the new `ShardReplicaSet::discard_proxy_local`,
which is infallible, never contacts the remote, and forgets queued
updates (replica states in the same snapshot already reflect the
transfer outcome).
The consensus test reproduces the incident: pause the transfer source
mid-transfer, restart the other peers so the aborted transfer can only
be learned via snapshot, and verify the source recovers.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add batch id-tracker lookups, pipelined in the disk-resident trackers
Introduce batch counterparts for the per-point IdTracker read operations
(external->internal resolution, internal->external lookup, version lookup,
deleted checks) and implement them with pipelined reads in the disk-resident
trackers, where the per-point path costs one storage round-trip per lookup:
- StoredBitSlice::get_bits_batch: dedupe the containing u64 elements and
fetch them through one read_batch pass.
- DiskMappingReader::lookup_batch: group e2i keys by run block, read each
unique block once, binary-search all its keys.
- DiskMappingReader::external_ids_batch: schedule all i2e data slots plus
the deduplicated is_uuid bitmap bytes in one pass.
- DiskMappingsSource::{points_deleted_batch,resolve_internal_batch,
resolve_external_batch}: shared batched resolution for both disk trackers.
- IdTrackerRead::{external_ids_batch,internal_versions_batch}: trait-level
batch methods with loop defaults for in-RAM trackers; overridden (along
with resolve_external_ids) in DiskIdTracker and ReadOnlyDiskIdTracker,
and forwarded through both tracker enums so the overrides dispatch.
Wire the batch operations into the read paths: search result processing
(external ids + versions), scroll filtered_read_by_index (chunked), and
HasId condition conversion/cardinality estimation. retrieve() already goes
through resolve_external_ids and picks up the batched dispatch.
Batch read-by-id keeps the read-only tracker's laziness: no full
deleted-set materialization (covered by test).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rework batch id-tracker interfaces to iterator inputs and callback output
Review follow-up: the slice-in/Vecs-out shapes forced intermediate collects
at every call site, costing the in-RAM trackers (whose defaults are plain
loops) allocations the pre-batch code never had.
- resolve_external_ids now takes `impl IntoIterator<Item = PointIdType>`
and delivers `(id, offset)` pairs through a callback, in input order:
has_id conditions stream the id set straight into the offsets set/vec,
retrieve fills its parallel vectors directly.
- external_ids_batch / internal_versions_batch take iterator inputs but
keep `Vec<Option<_>>` outputs (search result processing needs one aligned
slot per input); search drops its two offset collects, scroll passes the
itertools chunk directly.
- In-RAM trait defaults stream with no intermediate allocation; the disk
overrides buffer the input once internally, where the block-grouped
reads need the whole batch anyway.
Also document on the writable DiskIdTracker why points_deleted_batch and
internal_versions_batch are deliberately not overridden there: deleted and
versions are RAM-resident by design, so the default loop has no IO to
pipeline; only the read-only tracker keeps them on disk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove accidentally committed local example
inspect_mutable_id_tracker.rs is a local debugging harness that was swept
into the previous commit by a directory-level git add; it is not part of
the batch-interface change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add PointIdBatch and thread it through resolve_external_ids
Replace the `impl IntoIterator` / `&[PointIdType]` inputs on the
id-tracker external->internal resolve path with a small `PointIdBatch`
trait (Copy + num_ids + iter_ids), implemented for `&[PointIdType]` and
`&AHashSet<PointIdType>`. Being re-iterable and length-known lets the
disk trackers batch the reads without first collecting the input.
On the disk path this drops several intermediate allocations:
- no input `Vec<PointIdType>` collect in resolve_external_ids_batch;
- lookup_batch returns compact `(id, offset)` pairs (id rebuilt from
is_uuid + key) instead of a size-N `Option` reorder buffer;
- lookup_batch no longer groups keys by block up front: a block is one
~16 KiB DiskCache block, so same-block reads are deduplicated by the
cache (piggybacked in flight, a plain hit once resident);
- resolve_internal_batch filters deleted pairs in place and feeds
points_deleted_batch an offset iterator, dropping the offsets Vec.
Delivery is no longer input-ordered; the id travels with the offset into
the callback, so callers that need positions still have them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream disk id resolve through a callback; prefetch deleted flags whole
Follow-up to the PointIdBatch rework, removing the last buffers from the
disk external->internal resolve path and dropping the redundant deleted
batching.
- `lookup_batch` and `resolve_internal_batch` now hand each resolved
`(id, offset)` to a callback as its block completes instead of
returning a `Vec`; `resolve_external_ids_batch` just forwards the
caller's callback and logs on error. No buffer is built on the path.
- Drop `points_deleted_batch`: the deleted set is resident (writable
tracker) or prefetched whole (read-only), so a per-point `point_deleted`
check is as cheap as a pipelined read would be. `resolve_internal_batch`
captures the first check error out of band and surfaces it after the
pass.
- Prefetch the deleted flags whole (`Populate::PreferBackground`) in both
`try_preopen` and `try_open`, via a shared `deleted_open_options`, so
the per-point checks stay off remote storage.
- `PointIdBatch` loses the now-unused `num_ids`; only `iter_ids` remains.
Delivery is fully streamed, so on a mid-pass storage error the callbacks
for already-resolved live points have fired before the error surfaces —
acceptable at the best-effort `IdTrackerRead` boundary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Surface id resolve errors to the trait; drop redundant batch wrapper
`IdTrackerRead::resolve_external_ids` now returns `OperationResult<()>`
instead of swallowing storage errors. The disk path previously logged
and dropped a `resolve_internal_batch` error at this boundary; now it
propagates so callers see failed id resolution.
- The in-RAM default returns `Ok(())`; both enum forwarders and both disk
overrides propagate. The three call sites (retrieve, HasId cardinality,
HasId checker) are already in `OperationResult` functions, so they just
gain a `?`.
- Drop the `resolve_external_ids_batch` free function: with its
log-and-swallow gone it only forwarded to `resolve_internal_batch`, so
the two overrides call that directly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream internal_versions_batch through a callback
Give internal_versions_batch the same treatment as resolve_external_ids:
it takes a callback and returns OperationResult<()> instead of a
Vec<Option<..>>, so the disk override no longer allocates. It walks the
input lazily (no collect), tags each pipelined read with its internal_id,
and streams (internal_id, version) to the callback as reads complete;
out-of-range offsets are skipped and a storage error propagates instead
of being logged and swallowed.
The in-RAM default and both enum forwarders match. process_search_result
now builds a small internal_id -> version map from the callback and looks
up by scored offset (handling duplicate offsets naturally), keeping the
same missing-version-is-an-error behaviour.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream external_ids_batch through a callback; drop chunked scroll batching
Align external_ids_batch with internal_versions_batch: iterator input,
(offset, id) callback output, no internal buffering, storage errors
propagated. The deleted filter runs inside the lazy range iterator, so
each tracker implements the override directly against its own deleted
source (resident bitvec vs prefetched on-disk file) and the redundant
DiskMappingsSource::resolve_external_batch pass-through is removed,
along with the now-unused log_lookup_err_batch.
filtered_read_by_index feeds the whole candidate iterator into one
batch pass and lets the IO layer pipeline the reads; the limit case
keeps the top-smallest ids in a bounded priority queue. This retires
ID_TRACKER_BATCH_SIZE. Search result processing pairs external ids by
internal id, mirroring the versions map.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove accidentally committed local example, again
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop unused StoredBitSlice::get_bits_batch
Its only caller, the read-only tracker's points_deleted_batch override,
was removed when the deleted flags became whole-file prefetched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Write deleted_open_options fields explicitly, comment the why
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify read-only external_ids_batch error handling via try_filter
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add infallible point_deleted helper to the writable disk tracker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop PointIdBatch in favor of impl IntoIterator<Item = PointIdType>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test(model_testing): add slice matcher to generated scroll filter
Extend ScrollFilter with a Slice variant so paginated scroll exercises
Condition::Slice. The generator draws small totals (1/2/3/4/5/8) and a
valid index; the model verifier mirrors membership via Slice::check —
the same hash contract the engine uses — so the existing paged-scroll
id-set assertion covers sliced scroll under soak (optimizer, WAL reload,
multi-shard, mixed UUID/numeric ids).
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(model_testing): add CountBySlice verification op
Exercise Condition::Slice through the exact count API under soak.
Shares the slice generator with ScrollPaged; the model oracle uses
Slice::check so engine and in-memory counts must agree.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(model_testing): compose slice with num on scroll and delete-by-filter
- ScrollFilter::NumAndSlice: indexed num drives candidates; slice is a
per-candidate check via Filter::merge.
- DeleteByFilter { num, slice: Option<Slice> }: half the deletes also
restrict by slice so submit-time filter resolution and WAL-replayed
id lists exercise Condition::Slice.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Setting metadata to an empty object does not clear it: the update merges
key by key, so an empty object is a no-op (and over gRPC an empty map is
indistinguishable from an absent one). Per-key removal via null values is
the mechanism that was actually implemented and tested in #7123.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: slice filtering condition for sliced scroll and deterministic sampling
Add a `slice` filter condition selecting points where
`stable_hash(point_id) % total == index`. The hash is SipHash-2-4 with a
zero key over canonical id bytes (8 LE bytes for numeric ids, 16 RFC 4122
bytes for UUIDs) — a frozen public contract, independent of the internal
resharding ring hash, reproducible by clients to predict membership.
For a fixed `total`, slices are disjoint and cover all points, enabling
parallel scroll streams (ES sliced-scroll style) and reproducible sampling
that composes with any other filter condition.
- REST: `{"slice": {"total": N, "index": R}}`; gRPC: `SliceCondition` in
the condition oneof (tag 8)
- Evaluated per point via id_tracker external-id lookup; no payload index
needed; cardinality estimated as `points / total` with no primary clause
- `total >= 1` enforced by NonZeroU32 at parse time, `index < total` by
validation in both REST and gRPC paths
- Hash contract locked by test vectors independently reproduced with a
reference SipHash-2-4 implementation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tests: minimal OpenAPI test for slice filter condition
Scrolls all slices of a fixed total over numeric + UUID ids asserting
disjointness and full coverage, checks must_not inversion, and pins the
two rejection paths (422 for index >= total, 400 for total = 0). Requests
and responses are validated against the regenerated OpenAPI spec by the
test harness.
Note: the spec cannot itself reject total = 0 client-side — the Condition
anyOf falls through to the permissive Filter schema, as with any invalid
condition — so rejection is asserted via the server response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Replace the mixed read interface (internal types, explicit parameter
enumeration, ad-hoc custom types) with edge-owned request structs, one
file per request in src/requests/. Each has a new() constructor taking
only the required parameters, a no-macro fluent builder in src/builders/,
and a From conversion into the internal request type in
requests/conversions/, grouped by request type. Conversions construct
and destructure with full field lists, so a parameter added on either
side fails compilation instead of being silently dropped.
The old reexport aliases (ScrollRequest = ScrollRequestInternal, etc.)
are replaced by the edge types under the same names; retrieve() takes a
RetrieveRequest instead of a parameter triple. Python bindings wrap the
edge types, and the published Rust examples use the builders.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Non-generic TransformInto::transform
I think dyn is fine here because it is executed once per query, not in a
hot loop.
* Query::score_by impls: manual loops
Iterator chains (`.iter().map(..).sum()`) produce a lot work for the
compiler to do.
* QuantizedCustomQueryScorer: move generics from struct to `new` method
* refactor: group edge crate internals into edge_shard and read_view modules
Restructure lib/edge/src so the top level only contains the modules that
form the public crate surface. Implementation files move under the type
they implement:
- edge_shard/: the EdgeShard struct with its load/config-resolution
helpers (previously inlined in lib.rs), plus optimize, shard_read,
snapshots, and update
- read_view/: the EdgeShardRead/ReadSegmentHandle traits and
EdgeReadView (previously read_view.rs), plus the per-operation impl
files count, facet, grouping, info, matrix, query, retrieve, scroll,
and search; build_search_pool (previously pool.rs) is folded into
read_view/mod.rs next to par_map_segments, the seam it powers
lib.rs is now a thin facade of module declarations and re-exports. The
public API is unchanged: all previously exported names resolve exactly
as before, verified against the edge tests, the python bindings,
edge-shard-query, and the regenerated publish amalgamation with its
examples.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: trim EdgeShardRead surface and split read_view/read_only modules
Trait surface:
- Drop query_scroll and rescore_with_formula from EdgeShardRead and the
EdgeShard inherent wrappers; only the internal query pipeline used
them, via the pub(crate) EdgeReadView methods that remain.
- Hide the snapshot plumbing (read_segments, search_pool, plus
config_snapshot/path providers) in a crate-private ReadViewProvider
trait. EdgeShardRead now declares only user-facing methods and is
implemented for every provider through a blanket impl, so the
plumbing is not callable from user code (verified with a negative
compile test; a private supertrait alone leaves supertrait methods
callable through generic bounds). An empty sealed marker supertrait
keeps the trait unimplementable downstream.
- config_snapshot and path stay public: used by edge-shard-query and
the python bindings.
Module layout:
- read_view/: mod.rs keeps EdgeReadView and build_search_pool;
ReadSegmentHandle moves to handle.rs, the trait machinery to
shard_read.rs, and the nine per-operation impl files into ops/.
- read_only/: the follower's ReadViewProvider impl moves out of mod.rs
into shard_read.rs, mirroring edge_shard/shard_read.rs.
Verified: edge tests, python bindings, edge-shard-query, regenerated
publish amalgamation with all examples compiling and running.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: segment manifest optimizing state with lease
* fix: clippy
* fix: exhaustive manifest state matching
* fix: merge manifest rebuilds under the write lock
* refactor: named state predicates, drop redundant enumerator test
Review follow-up: move the enumerator's filter into
SegmentManifestState::is_usable, name the preserving predicate
is_optimizer_mark, delete the enumerator test that re-tested serde.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The persisted_hashmap and disk_cache tests generate their datasets with
StdRng (ChaCha12 in rand 0.10). String keys draw one random_range call
per character, so the crypto generator dominated the test runtime:
test_k_str_* drop from ~3.3s to ~1.4s each with SmallRng (Xoshiro256++).
Assertions are self-consistency checks (write then read back), so the
changed sequences carry no retuning risk.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
copy_to_path returned Ok(()) without flushing, so the copy was not guaranteed on disk. Flush after the copy, matching create() and flush() in the same file. Also add a round-trip test for copy_to_path, which had none.
* Benches: use SmallRng instead of ChaCha12-based generators
All benchmarks used StdRng or rand::rng() (ThreadRng), both backed by the
ChaCha12 block cipher in rand 0.10. Benchmarks do not need crypto-strength
randomness, and several draw random values inside the timed closure, so
cipher work was included in the measurement itself.
Switch every bench target to SmallRng (Xoshiro256++), and key the HNSW
graph cache and sparse index cache by RNG algorithm so stale caches built
from the old generator are not reused against newly generated vectors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Benches: replace free-function rand::random with local SmallRng
Addresses review: rand::random draws from the thread RNG (ChaCha12),
including inside the timed loop of the pq score benchmark.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Model tester: cover all quantization types
Add a `quantization` field to `VectorCandidate` so candidates can carry
any `QuantizationConfig` variant, and materialize the configs in the
fixture (`quantization_config`). The inline-storage vector "i" keeps its
scalar Int8 config, now declared on the candidate instead of hard-coded
in the fixture.
New candidates:
- "p" Dense(8) + Product x4
- "v" Dense(6) + Binary (non-byte-aligned dim, trailing-bit padding)
- "r" Dense(8) + Turbo (search-side TQ over Float32 storage)
Quantization x datatype combos:
- "l" Dense(6) Float16 + Binary
- "d" Dense(8) Turbo4 + Turbo default bits (keep-source-rotated branch
of `should_keep_source_rotated`)
- "g" Dense(8) Turbo4 + Turbo Bits1_5 (Padded rotation, rotate-back
branch)
This is model-safe: schema quantization keeps the original vectors, so
read-back predictions are untouched; the approximate quantized scoring
only feeds the membership-only Search/Query/Recommend checks.
`assert_candidates_predictable` enforces the wiring constraints:
quantization is dense-only (the fixture only wires the dense arm) and
requires `initially_active` (CreateVectorName's `DenseVectorConfig`
carries no quantization).
Verified: seeds 1/2/3/7/42 soaks (5k ops, restarts, optimizer on) green;
quantized codes confirmed on disk for every quantized candidate.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Model tester: make inline_storage a VectorCandidate knob, enable on "l" and "d"
Replaces the fixture's name-based INLINE_STORAGE_VECTOR special case with an
inline_storage field on VectorCandidate (requires quantization, enforced by the
startup assert). Enables it on "l" (Float16 base + padded Binary links) and "d"
(Turbo4 base + TQ links) to cover more (base layout, link encoding) pairs of the
CompressedWithVectors format; "v", "r", "g" and "p" keep the non-inline paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Fix resharding, on queries filter shards on all shard selectors
* Add failing consensus test: search during resharding with shard keys (#9880)
Reproduces a known bug: after resharding is initialized on a custom
sharded collection with a shard key, searches (with and without the
shard key selector) fail with "does not have enough active replicas",
because the new resharding shard is included in reads before it has
an active replica.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Exempt explicit shard id selection from resharding read filter
Explicit shard id selection is only used by internal per-shard
operations (local shard API, internal gRPC reads), including the
resharding driver reading back migrated points from the new shard.
These must reach the resharding shard before it becomes visible to
user-facing selectors, and filtering them also made per-shard reads
return silently empty results on peers lagging on hashring commits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Explicitly set resharding filtering per match branch
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The HashSet bought nothing: names are unique by construction (map keys)
and all callers only iterate. A sorted Vec skips the hashing and set
allocation, and makes the iteration order deterministic.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* DiskIdTracker: RAM-resident is_uuid stored-bitmask sidecar + module split
Move the is_uuid flags out of the i2e file into a separate
id_tracker.is_uuid file in the compact StoredBitmask format (#9871),
loaded whole into RAM as a RoaringBitmap on open and prefetched in
preopen, so slot decoding never reads the flag from disk. Bump the
on-disk format version to 2 (DiskIdTracker is unreleased; no migration).
Add StoredBitmask::read_ones() in common to normalize any stored
encoding into a bitmap of set positions, with logical_len validated
against the u32 position space at open.
Restructure disk_id_tracker: read_only.rs becomes read_only/{mod,
lifecycle,live_reload,id_tracker_read} mirroring the immutable tracker,
and reader.rs becomes reader/{mod,lifecycle,lookup,iter}.
Also unbox the DiskMappingsRef iterators (impl Iterator instead of
Box<dyn>), and make IdTrackerRead::iter_internal_versions fallible so
the read-only disk tracker propagates storage errors instead of
silently truncating; its implementation now reads the versions file in
one pass (cleanup-on-open drains it anyway).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Split disk_id_tracker mod.rs into lifecycle / read / write files
Mirror the read_only/ layout: mod.rs keeps the struct and resident-RAM
helpers, lifecycle.rs the build/open paths, id_tracker_read.rs the
DiskMappingsSource + IdTrackerRead impls, id_tracker.rs the mutable
IdTracker impl. Code moved verbatim; only imports and a field doc touched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Tighten disk_id_tracker docstrings around guarantees
State contracts (residency, laziness, error semantics, atomicity)
instead of narrating which structures hold or use what.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Deduplicate plain and raw upsert drivers via PointToUpsert trait
upsert_points and upsert_points_raw were ~60-line clones differing only
in how the point struct writes itself into a segment. Extract a private
PointToUpsert trait with the two variation points — upsert_into (in-place
write) and write_moved (CoW-move record transform) — implemented for
PointStructPersisted and PointStructRawPersisted, and fold the chunked
driver into a single generic upsert_points_impl.
The public functions keep their names and signatures as thin wrappers.
The duplicated upsert_with_payload/upsert_raw_with_payload tails collapse
into one shared set_full_or_clear_payload helper, which also carries the
single has_point debug assertion.
No behavior changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Deduplicate plain and raw sync drivers via PointToSync trait
sync_points and sync_points_raw were ~80-line clones of the same 5-step
algorithm, differing only in the retrieval call (retrieve vs retrieve_raw)
and the stored-record type compared against. Extend the upsert approach
with a PointToSync subtrait carrying an associated StoredRecord type,
retrieve_stored, and is_equal_to (delegating to the existing inherent
methods), and fold the drivers into a single generic sync_points_impl.
Step 5 calls upsert_points_impl directly; PointToUpsert and
upsert_points_impl become pub(super) to be visible within points/.
Public signatures unchanged.
No behavior changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
lib/shard/src/update.rs grew to 2200+ lines. Split it into an update/
directory by operation kind, moving code verbatim:
- mod.rs: process_* dispatch entry points + re-exports (public API and
crate-internal paths are unchanged)
- points/: upsert.rs (plain, conditional and raw), delete.rs (by id and
by filter), sync.rs (plain and raw)
- vectors.rs, payload.rs, field_index.rs: per-kind apply functions
- helpers.rs: shared filter-based point selection (incl. the deferred
points corner case) and check_unprocessed_points
- tests.rs: the test module, unchanged
Only additions are per-file imports, re-export lists and two
pub(super) visibility bumps for now-cross-module helpers.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The WAL is only truncated past operations whose segment flush was
confirmed, so first_index is a durable lower bound on the applied
sequence. The persisted applied_seq can legitimately lag behind it by
more than one save interval: it is saved every 64 update-worker calls
from a counter that restarts at zero on process start, and synchronous
WAL replay never feeds it. A replay target computed from such a stale
applied_seq can then sit before first_index, tripping the debug_assert
from #8454 (flaky model_testing gate, #9844) and, in release builds,
enqueueing already-truncated indices that fail with spurious
"Operation not found in WAL" errors.
Clamp the replay target at first_index: nothing before it ever needs
replay.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Compact stored bitmask for on-disk field index deleted masks
Add StoredBitmask: a compact persisted bitmask written and read as a
whole. The payload is a roaring bitmap of whichever bit value is the
minority (mostly-0 and mostly-1 masks both stay tiny), falling back to
raw dense bits when roaring would not be smaller, so the file is never
larger than the dense representation. Files are replaced atomically via
UniversalWriteFileOps::atomic_save; there is no in-place mutation.
Use it for the write-once "no values" masks of the on-disk numeric,
geo, map and full-text indexes, replacing the raw dense bitslice files
sized at point_count/8 bytes regardless of content.
Writing the new format is gated by the compact_bitmask feature flag
(default off; enabled by `all` and `serverless_compatible`). Reading is
format-agnostic regardless of the flag: the compact deleted_mask.bin is
tried first, then the index-specific legacy file, so old segments stay
readable and flag-off builds produce byte-identical legacy files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Split save_bitmask into named helpers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Regenerate OpenAPI spec for compact_bitmask feature flag
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review findings on stored bitmask
- Avoid u64 overflow in the payload bound check when opening a mask
with a corrupted payload_len.
- Reject roaring payload positions beyond logical_len at read time,
enforcing the BitmaskContent range contract for corrupted files.
- Remove the opposite-format mask file after a successful save, so a
rebuild with a flipped compact_bitmask flag can't leave a stale
compact file shadowing the fresh legacy one (or an orphaned legacy
file next to a compact one).
- Make the compact-open numeric test tolerate builds that already
wrote the compact format.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Every point id the workload draws now comes from an IdSpace pool
precomputed at startup. A --uuid-id-fraction (default 0.5) of the
--id-pool slots are well-formed v4 UUIDs built from the seeded rng via
uuid::Builder::from_random_bytes, the rest stay numeric. Precomputing
the pool keeps the id-reuse semantics (upserts overwrite live points,
deletes and retrieves hit them) that fresh per-op random UUIDs would
lose, and keeps runs seed-reproducible.
Sampling consumes a single range draw per id, exactly like the previous
NumId draw, so fraction 0 consumes no extra rng draws and reproduces
the numeric-only op stream byte-for-byte. The harness smoke tests run
with fraction 0.5, and the fraction is recorded in the trace header.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Preparation for capping appendable segment growth in the update path
(#9158): a dedicated error for "all appendable segments reached
max_segment_size", so the update pipeline can recognize it and provision a
fresh appendable segment before re-applying the operation.
Maps to a transient service error at the collection level: if it ever
escapes recovery, failed-operation recovery re-applies the operation.
Part 1/5 of the appendable segment overflow fix.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Every vector candidate now carries a distance metric instead of the
hardcoded Dot, threaded into the fixture schema, the CreateVectorName
generator and the read-back prediction.
The model predicts Cosine read-backs exactly by mirroring the engine's
ingestion preprocessing: metric_preprocess follows
NamedVectors::preprocess_dense_vector's per-datatype dispatch and calls
Distance::preprocess_vector itself, so predictions track the engine by
construction (including the identity preprocess of the byte metric,
which stores Uint8 vectors un-normalized). Stored vectors are
preprocessed exactly once (optimizer and CoW moves transfer raw bytes),
so predictions stay exact across moves, including Cosine + Float16.
New candidates: "e" (dense Cosine), "n" (multi-dense Cosine, per-row
normalization), "x" (dense Cosine + Float16), "o" (dense Cosine +
Turbo4, padding-free dim), "j" (dense Euclid), "k" (dense Manhattan).
Euclid/Manhattan preprocess is an identity, so their value is engine
side: Order::SmallBetter comparator coverage.
The startup predictability check now also rejects sparse + non-Dot
(sparse schemas carry no distance) and Turbo4 + Euclid/Manhattan (TQ's
L1/L2 modes store lengths differently from Dot/Cosine and their
copy-on-write re-quantization fixed point is not soak-validated yet).
Soak-validated on seeds 1/2/4/5/6/7/8 (30k ops), including two
restart runs (restart probability 0.002) with the optimizer enabled.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Profiling showed SipHash over JsonPath keys (RandomState::hash_one)
dominating field index lookups in
StructPayloadIndexReadView::estimate_field_condition: the map holds
only a handful of indexed fields, but it is queried per condition per
filter evaluation per query, and hashing the path is the entire
lookup cost.
Switch the field index lookup maps (IndexesMap, ReadOnlyIndexesMap,
the read view borrow, and the query_checker / condition_converter /
value_retriever signatures that receive them) from std HashMap to
AHashMap, already used throughout the crate. Serialized and API-level
schema maps keep std HashMap: they are cold and their type leaks into
conversions.
conditional_search bench (criterion, 100 samples, vs saved baseline):
- struct-conditional-search-query-points: 583.7 us -> 443.5 us (-23.1% median, p ~ 0)
- struct-conditional-search-context-check: 73.2 us -> 64.8 us (-12.6% median, p ~ 0)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
