mirror of
https://github.com/qdrant/qdrant.git
synced 2026-09-26 16:07:41 -05:00
model-testing-reload-postmortem
1961
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Arnaud GourlayandClaude Opus 5
|
bf8a1d0efc |
test(model_testing): log every segment flush execution
The snapshots-off A/B refuted the snapshot hypothesis: both passes of a disable_snapshots run failed with the identical reload-lost class. Its fully instrumented log then closed every other gap at once: no directory was deleted, the acknowledge window held, both the source's and the destination's directories reloaded - with the source's on-disk version past the CoW operation and the destination's below it, while no flush-pass decision was logged for the shard in the window, and the destination had left the holder dirty via a proxy wrap immediately after receiving the arrivals. Durable state therefore advanced through flush executions that pass-level decision logging does not attribute. Log each execution of a segment's flush closure with its directory and captured version, whichever caller captured it: the flush worker, the snapshot force-flush, or anything else holding a flusher. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Sonnet 5
|
309c64945a |
[UpdateOnly] wire every component into the appendable segment (#10152)
* [UpdateOnly] wire every component into the appendable segment `AppendableSegment::store_points` sheds its `todo!()`: the id tracker claims a fresh slot per point, every component writes its data at those slots — each named vector storage, the payload storage, the payload indexes — and only then do the versions cover them, the step that makes the points visible to readers. A crash anywhere in between leaves claimed, unpublished slots, which the next writer to open the segment retires. Each vector comes from whichever half of `FullyQualifiedPoint` holds it: the batch's decoded vectors win over the bytes carried from the point's previous slot, and a name in neither still takes its slot as a vector the point does not have. The store components open lazily, on the first `store_points`. A batch that only deletes writes nothing but the mappings log, so it never pays for those opens — and it keeps working against segments whose payload storage was created in mutable mode, which the append-only writers refuse and which is all any leader builds today. The writer now also remembers what it stored, so `tombstone_points` skips a point this very batch wrote instead of retiring its fresh slot; the caller can hand over every slot a stored point used to occupy without holding that rule. `UpdateOnlySegmentEnum::open` takes the segment config, which is where the writer learns which vector storages exist. The end-to-end edge tests now run stores the whole way through: located and resolved through the `LookupSegment`s, appended by the writer, and read back through an ordinary follower — a new point with its payload, a rewrite winning over the old copy, a replayed batch skipping on the published versions, and a second writer resuming every component where the first ended. The leader still writes its payload storage in mutable mode, so the tests recreate it empty in append-only mode, standing in for segment creation wiring that does not exist yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] drop the stored-ids guard from `tombstone_points` The caller already never asks to retire a point its batch stored — it has to hold that rule regardless, since `preview` mirrors it to count outcomes — so the writer-side set was redundant state, and it made `tombstone_points` silently drop requests instead of honoring a stated contract. The contract is now stated: only points the batch deletes go here, because a delete addresses the external id and would take a stored point's fresh slot along with the stale one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] gate the store tests off Windows The leader's writable storage preallocates chunk files, and the append-only writer cuts them back to end at the data — its append offset is a compare-and-swap token, so a file longer than the data would make every append conflict. That cut replaces the file, which Windows refuses while the writer's own `LookupSegment`s hold it memory-mapped; on Linux the old inode simply lives on under the mappings. Nothing to fix in the writer: Windows cannot shrink a mapped file, and the production target is object storage, where neither preallocation nor mmap exists. The delete tests keep running everywhere; the store tests move into a `#[cfg(not(windows))]` module together with the imports and helpers only they use, so the Windows build carries no unused-import warnings. Cross-checked with `--target x86_64-pc-windows-msvc`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] wire the quantized overlay into StoreComponents Opens UpdateOnlyQuantizedVectors alongside each dense, non-multivector, non-Turbo4-datatype vector's raw storage, when the segment's quantization config supports incremental appends (Binary/Turbo). Multivector and Turbo4 combinations are out of scope (see UpdateOnlyQuantizedVectors' own doc comment) — such a vector simply has no quantized overlay entry and stays searchable exactly through its raw storage alone, same as before. store_points keeps the overlay's row count in exact lockstep with the raw storage: every point takes a row in both, in the same order, at the same id (start_slot + offset) — a decoded vector encoded for real, a Raw-bytes-carryover blob decoded back to f32 per its actual storage datatype (mirroring QuantizedVectors::create_impl's use of PrimitiveVectorElement::quantization_preprocess for the same purpose on the non-update-only path), and a Missing vector as an all-zero placeholder. Skipping a row for the latter two cases would silently misalign every later quantized lookup — scoring one point's vector against another's quantized copy — so this mirrors the raw storage's own "every point takes its slot" rule exactly rather than only handling the common decoded case. UpdateOnlyQuantizedVectors now retains its resolved QuantizedVectorsConfig (exposed via quantization_config()/dim()) rather than discarding it after opening storage, since a reopened overlay's persisted config is the source of truth for how to decode carried-over bytes — not necessarily identical to whatever live config the caller has to hand. Its now-unused flusher() is dropped: like every other update-only storage in this stack, a write is already durable when append_many/upsert_vector returns. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
1a8c9a6397 |
[UpdateOnly] implement the appendable quantized-vector overlay (dense, Binary/Turbo) (#10161)
* [UpdateOnly] implement the appendable quantized-vector overlay (dense, Binary/Turbo) Appendable/plain segments can carry live quantized vectors today: PlainVectorIndex:: update_vector calls quantized_vectors.upsert_vector alongside the raw vector on every insert (lib/segment/src/index/plain_vector_index/lifecycle.rs), auto-created for a fresh segment when appendable_quantization is on and the method supports it (QuantizationConfig::supports_appendable — Binary and Turbo only; Scalar/Product are policy-gated off regardless of storage backend). The update-only vector-storage stack (this PR's base) had no equivalent: UpdateOnlyVectorStorage::open never read quantization_config, and nothing under vector_storage/*/update_only/ mentioned quantization at all — a segment configured with quantization would silently lose it end-to-end once written through this path. This adds UpdateOnlyQuantizedVectors, mirroring QuantizedVectors' auto-create/reopen behavior but scoped to dense (single-vector) Binary/Turbo — the two methods that support incremental appends, matching current capability exactly (multivector support is a follow-up: it needs its own append-only offsets storage, mirroring MultivectorOffsetsStorageChunked the same way this mirrors QuantizedChunkedStorage). The only new machinery is UpdateOnlyQuantizedChunkedStorage, an EncodedStorage backed by UpdateOnlyChunkedVectors (append-only, S: UniversalAppend) instead of ChunkedVectors' positional writes (S: UniversalWrite) — everything else reuses the quantization crate's EncodedVectorsBin::encode/load and EncodedVectorsTQ::encode/load completely unchanged, since both are already generic over the storage backend. It writes files in the exact layout QuantizedChunkedStorage reads, so a promoted segment's quantized data reads through the existing, unmodified reader with no new reading code. UpdateOnlyChunkedVectors gains one addition: a `get` method to read back a single vector, needed because EncodedVectors::load validates the storage's vector size by reading vector 0 (skipped when the store is still empty). Verified: the update-only writer's persisted bytes, read back through the standard (non-update-only) QuantizedChunkedStorage + EncodedVectorsBin/TQ::load, match a RAM-backed reference fed the same vectors one at a time through upsert_vector, byte-for-byte, for both Binary and Turbo. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * [UpdateOnly] fix quantized reopen: resume writing shouldn't validate stored reads The previous commit made reopening a non-empty quantized overlay panic (EncodedVectorsBin/TQ::load validates a non-empty store by reading its vector 0, which UpdateOnlyQuantizedChunkedStorage's write-only design cannot serve) and worked around it with a redundant pre-check plus a todo!(), narrowing the tests to single-session-only writes. Both of those were the wrong fix. A writer resuming appends doesn't need `load`'s read-and-validate — it only needs the fitted metadata (encoding, stats) to keep encoding consistently, and that invariant already holds by construction: every vector this writer ever encodes is sized from the same `quantized_vector_size` `load` and the new path both read. Added `EncodedVectorsBin`/`EncodedVectorsTQ::reopen_for_write` to the quantization crate — identical to `load` minus the validating read — and switched `open_existing` to it. `UpdateOnlyQuantizedChunkedStorage` stays write-only as originally designed; no new read capability, no pre-check, no todo. Tests restored to the original two-writer split (write half, drop, reopen, write the rest), now genuinely exercising resume-with-data instead of avoiding it, and still passing byte-for-byte against the reference. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * [UpdateOnly] split EncodedStorage into EncodedStorageWrite + EncodedStorage A write-only storage (the update-only quantized overlay) had to fake a full EncodedStorage impl with unreachable!() read stubs just to satisfy EncodedVectorsBin/TQ's generic bound. Split the trait so a write-only backend only needs to implement EncodedStorageWrite; EncodedStorage adds the read methods on top. The overlay now implements EncodedStorageWrite alone — no panicking stand-ins for methods that don't exist. * [UpdateOnly] remove UpdateOnlyQuantizedVectors::create Nothing in this stack builds the first appendable segment of a collection yet (that's still a todo!() in edge/src/update_only), so create() had no real caller and open() had to guess from file absence whether to invoke it. open() now only reopens an overlay create() already persisted; the bootstrap logic moved into tests.rs as a private fixture helper, since tests still need it to build fixtures. * [UpdateOnly] fix CI: codespell typo and lint dead-code on unwired write path codespell flagged "implementors" (wants "implementers") in two doc comments. Separately, CI's lint job runs clippy without --all-targets, so the update-only quantized write path — genuinely unreachable from any non-test code until #10152 wires it into a segment — trips -D warnings dead-code. Scope #![allow(dead_code)] to the two files that are only exercised by their own tests today, and allow the now test-only UpdateOnlyQuantizedChunkedStorageBuilder re-export. * [UpdateOnly] fix ast-grep: use expect(dead_code) instead of allow * fix CI: remove unused EncodedStorageWrite import in gpu vector storage Left over from splitting EncodedStorage into EncodedStorageWrite + EncodedStorage; only caught under --all-features since gpu is gated behind a feature flag. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: qdrant-cloud-bot <111755117+qdrant-cloud-bot@users.noreply.github.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
89f1ae939c |
[UpdateOnly] implement the appendable vector storages (#10151)
* [UpdateOnly] drop the `UniversalWrite` bound from `UpdateOnlyChunkedVectors` Nothing in it needs random-offset writes: the config, the chunk listing and the status file all go through `UniversalReadFs` / `UniversalWriteFileOps`, which `UniversalAppend` already provides. The bound excluded the object-store backend this writer exists for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] implement `UpdateOnlyDenseVectorStorage` The vectors go into `UpdateOnlyChunkedVectors` and the deleted flags into `UpdateOnlyStoredFlags`, both of which already append; what this adds is the directory layout and the rule for a point with no vector under this name. Such a point still takes its slot, holding a placeholder, and is flagged deleted — slots are shared across every named storage of the segment, so skipping one would shift every later vector of this storage against the id tracker. Only the missing ones are flagged: an unflagged slot reads as present, and the mask is explicitly allowed to be shorter than the vector count, so a batch where every point has a vector rewrites no mask at all. `VectorToStore` is the input, mirroring the two halves of `FullyQualifiedPoint`: vectors the batch decoded, and storage-native bytes carried over from a point's previous slot which are appended without a decode round-trip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] implement `UpdateOnlyMultiDenseVectorStorage` Rows here are not indexed by point slot — a point owns a run of them — so this writer tracks where the row space ends and places each run itself, reading the end from the chunked storage on open. A run that would straddle a chunk skips to the next one, as the writable side does, since a read of a multi-vector assumes its rows are contiguous within a chunk. The rows of a batch are therefore not always one span, and each span is appended on its own; the gap a skip leaves is zero-filled by the append that follows it. A point with no multi-vector here owns no rows at all: its offset entry says so. Unlike the single-vector storages there is no row to keep aligned, because the offsets are what map slot to rows. Adds `stored_len` and `remaining_chunk_keys` to `UpdateOnlyChunkedVectors` — the vector count read that #10114 dropped as unused now has a user. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] implement `UpdateOnlySparseVectorStorage` The vectors go into `UpdateOnlyBlobstore` — the sparse storage is the one that was already blob-backed — and the flags into `UpdateOnlyStoredFlags`. A point with no sparse vector stores nothing at all, since the storage is keyed by slot and an unwritten slot is already "no vector"; it is flagged instead. `UpdateOnlyStoredFlags::open` now materializes its directory rather than waiting for the first flag. Storages use that directory as the marker that they exist: `MmapSparseVectorStorage::open_or_create` takes its absence for "not created yet" and starts a fresh storage over the top of the old one. A batch that flags nothing must still leave it behind. Caught by the resume test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] implement the TurboQuant vector storages `UpdateOnlyTurboVectorStorage` and its multivector counterpart. The quantizer is rebuilt from the dimension and distance rather than read back — it carries no learned state, so the two sides encode identically, which the test asserts by comparing the encoded bytes against what the writable storage produces for the same vector. The multivector one places runs of rows exactly as the plain multivector storage does, skipping to the next chunk rather than straddling one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] add `UpdateOnlyVectorStorage`, the dispatch over the five families Selects the writer from the vector config the way the writable side selects the storage, and refuses a storage type an update-only segment cannot have: the mmap ones are built whole rather than appended to, and the empty placeholder has no files. Sparse gets its own opener, since sparse vectors are configured separately from dense ones rather than through `VectorDataConfig`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fix clippy under `-D warnings` - `is_multiple_of` in place of the manual remainder checks in the two multivector writers. - Drop the `dead_code` expectations on `UpdateOnlyChunkedVectors`: the vector storages use it now, so the expectation no longer holds. - Drop a `TypedMultiDenseVectorRef::from` that converts to its own type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] fix a stale doc and an inconsistent guard The doc on `UpdateOnlyStoredFlags::open` still said nothing is created until the first flush, from before open started materializing the directory eagerly. And the span-merge guard in the multivector writer hedged with `dim.max(1)` while the same function divides by bare `dim` three lines up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] append a multivector batch's rows once, with the gaps as zero rows The chunk layer packs rows consecutively while runs must not straddle a chunk, so a batch's rows are not gapless. The old bridge grouped them into contiguous spans and appended each on its own, leaning on `ensure_chunk_lengths` — the repair path — to zero-fill the gap before every span, and saving the status once per span. Making the gaps explicit zero rows removes all of that: one append per batch, through the normal write path, one status save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
e95f56861b |
[UpdateOnly] implement UpdateOnlyStructPayloadIndex, the per-segment fan-out (#10150)
* [UpdateOnly] implement `UpdateOnlyStructPayloadIndex`, the per-segment fan-out Every field index of one segment, opened for a batch and dropped with it — the update-only counterpart of `ReadOnlyStructPayloadIndex`, and the level `AppendableSegment` needs: it takes the points a batch stores and leaves every index of every indexed field current. It reads which indexes a field has from the payload config, exactly as the read-only side does, and holds nothing else. No payload storage, no id tracker, no vector storages: those are there to answer queries and to work out what an update means, and by the time a batch reaches here that is settled — each point arrives with the payload it will be stored with. Every field is offered every point, including points whose payload holds nothing under it. An index that stores values per point stores none for those; the null index records that the point has no value there, which is the whole reason it exists. That is simpler than the writable path's add-or-remove split, which is only needed because a slot there may already hold something. A field whose index types the config does not spell out is refused. That config predates those types being recorded, and the writable index repairs it by deriving them from the schema on its next open; this writer builds no indexes, so it cannot, and going on would leave whatever is on disk to rot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] trim the doc comments on the payload index fan-out Keep the guarantees and the non-obvious rationale, drop the restatements. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
9d1894d311 |
[UpdateOnly] implement UpdateOnlyFieldIndex for the appendable payload indexes (#10147)
* [UpdateOnly] implement `UpdateOnlyFieldIndex` for the appendable payload indexes The payload index write half for the update-only segment writer, over a backend that only appends. An appendable field index keeps two things: the values it persists per point, and the in-memory structure it answers queries from. Only the first is state — the second is rebuilt from it on every open, by the mutable index and by its read-only counterpart alike. A writer that never answers a query therefore holds nothing: it turns a point's payload into the values its index would persist, appends them at the point's slot, and is done. What differs between index types is only that translation, so that is all `UpdateOnlyIndexKind` captures; `UpdateOnlyValueIndex` is the storage around it, the same for all of them, and each kind lives next to the index it writes for as the read-only counterparts do. The extraction itself is taken from the index types' own `ValueIndexer` and `NumericIndexIntoInnerValue` impls rather than restated, so the two sides cannot drift apart. `UpdateOnlyFieldIndex` dispatches over the nine covered index types, mirroring `ReadOnlyFieldIndex`. What the writer emits is the append-only mode of the very same storage the mutable index writes, and `Blobstore` selects the mode from the persisted config, so the read side needs no change: every test here writes through the update-only writer and reads back through the ordinary appendable index, opened on the directory the writer produced. The boolean and null indexes are not covered and are refused loudly rather than skipped. They keep a bitmask over all points instead of values per point, and persist it through random-offset writes, which an append-only backend does not offer. A skipped index goes stale and then answers queries wrongly, and the null index complements every other index of every indexed field — so a caller that took a silent skip for "nothing to do" would leave every field it touched wrong. Covering them needs an append-only bitmask representation first. That is also why this stops short of the struct-payload-index fan-out: until bool and null can be written, a component that claims to keep a field's indexes current could not. `UpdateOnlyPayloadStorage` moves onto the shared `UpdateOnlyBlobstore` extracted here, which is what it already was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] share the flattening, tokenization and flush guard Cleanup pass over the field-index writers, from a reuse/simplification review. Share what was restated: - `ValueIndexer::flatten_values` is now a provided method holding the loop that `add_point` had inlined. The update-only kinds call it instead of the free `extracted_values`, which built one throwaway `Vec` per input value on top. - `FullTextIndex::tokenize_document` and `serialize_stored_document` hold the sentinel placement and the phrase-matching order-vs-sort decision that the update-only kind had copied out of `MutableFullTextIndex::add_many`. Both sides call them, so a document written by one always matches the phrases the other would. Simplify: - `UpdateOnlyFieldIndex::open` matches on the index type alone and takes the text params via `TextIndexParams::try_from`, as `ReadOnlyFieldIndex::open` does. That drops the schema tuple, the nine-arm mismatch block and the `Option` return. - The `UuidIndex` variant is gone: that discriminant is historically map-backed, and both the writable selector and the read-only mirror already collapse it into `UuidMapIndex` — its `storage_dir` is `map_dir`, so a numeric-kind writer was writing into a directory everyone else opens as a map index. - Why bool and null cannot be written append-only now lives in `PayloadIndexType::is_append_only_writable`, next to `storage_dir`, so that whoever decides a field is update-only-serviceable can ask rather than rediscover it; `open` consults it as a backstop. - Dead `new()` constructors on the two zero-sized kinds. Skip the flush when nothing was buffered, in `UpdateOnlyBlobstore` rather than in one caller: a flush with nothing to write still syncs every page file of the storage, and for a field index an empty batch is the common case — every point that lacks the field, or holds a value the index rejects, stores nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] move each index kind under the appendable index it writes for `numeric_index/update_only.rs` and its three siblings sat at the index-type level, next to the enum over all three storage variants, although each writes for the appendable variant alone. They now live at `<index>/mutable_<index>/update_only/`, beside that variant's `read_only/` counterpart, which is the same split for the same reason. `mutable_text_index` is private, so the text kind is re-exported from `full_text_index` for the dispatch enum to name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] cover the bool and null indexes by rewriting their masks whole These two keep a bitmask over all points rather than values per point, so keeping one current means changing bytes in the middle of it — which an append-only backend cannot do. It can replace a file outright, and that is enough: `UpdateOnlyStoredFlags` reads the mask into memory on open, sets the batch's bits, and writes both files back whole, in the same format `DynamicStoredFlags` uses. A reader cannot tell which side produced them. The mask goes out before the length that publishes it, so a torn batch falls back to the shorter mask rather than to flags that were never written, and the whole-file write is charged to the hardware counter at flush — it is the write that actually happened, not the handful of bits the batch touched. `UpdateOnlyBoolIndex` and `UpdateOnlyNullIndex` sit on that, next to their mutable index like the other kinds. The null classification (which values count as present, which as null) moves into `classify_payload`, shared with `MutableNullIndex::add_point`, and the boolean one reuses that index's own `ValueIndexer`. Both are recorded for every point of a batch, including those whose field holds nothing: "this point has no value here" is precisely what these indexes are asked. With that, `UpdateOnlyFieldIndex` covers every index type `ReadOnlyFieldIndex` does, so the refusal and `PayloadIndexType::is_append_only_writable` are gone. The cost is that a batch rewrites the entire mask however few bits it touched — about 1.2 MiB per flag set for a segment of ten million points. Documented on the writer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] trim the doc comments on the field index writers Keep the guarantees and the non-obvious rationale, drop the restatements and the comments that narrate the next line. One code change: `values.contains(&true)` in place of `values.iter().any(|value| *value)` on the boolean index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
c8261beaec |
[UpdateOnly] implement UpdateOnlyPayloadStorage (#10146)
* [UpdateOnly] implement `UpdateOnlyPayloadStorage` The payload write half for the update-only segment writer: a short-lived storage opened for one batch and dropped with it, over a backend that only appends. Backed by a `Logstore` — the append-only mode of the same storage the writable `PayloadStorageImpl` uses — so a slot's payload is written once and never rewritten. `append_many` takes one payload per point at the slot the ID tracker claimed for it and flushes, so a batch is durable when the call returns and nothing is buffered across calls. Puts only buffer, so the flush is what touches the files: one append per touched page file plus one to the tracker, regardless of how many points the batch holds. A point with an empty payload is skipped, since an unwritten slot already reads back as an empty payload, and so is any gap between slots, which the tracker materializes as unmapped entries. `Logstore` had to leave the `Blobstore` facade for this: `Blobstore`'s type is bound at `UniversalWrite + UniversalAppend` for the sake of its `Gridstore` variant, so it cannot be named on a backend that only appends. Its cross-crate surface is `open_or_create`, `put_value` and `flusher`, nothing more; the new `open_or_create` mirrors `Blobstore`'s and rejects a storage created in mutable mode rather than opening it. Not wired into `AppendableSegment` yet — `store_points` stays `todo!()` until the vector storages and field indexes exist, as with `UpdateOnlyChunkedVectors`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] trim the doc comments on the payload storage writer Keep the guarantees and the non-obvious rationale, drop the restatements — the merged-baseline style of `UpdateOnlyChunkedVectors` and `AppendableSegment`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Jojii
|
4c9d124ab4 | Move deferred iter into PointMappings. (#10156) | ||
|
Andrey VasnetsovandClaude Opus 5
|
1d18e46551 |
[UpdateOnly] split UpdateOnlySegment into its lookup and writer phases (#10142)
* [UpdateOnly] split UpdateOnlySegment into its lookup and writer phases Applying a batch runs in two phases that agree on almost nothing, and `UpdateOnlySegment` was both: `resolve.rs` used every field, `append.rs` used none of them and could not — a `ReadOnlyPayloadStorage` has no append path. The `fs` field existed only for the writes that were never wired up. Split along that line: * `LookupSegment` (was `UpdateOnlySegment`) is the read phase. Every segment of a shard is opened as one, on read-only bounds, and the phase above them aggregates. Loses the dead `fs` field. * `DeleteOnlySegment` and `AppendableSegment` are the write phase, one segment each, `UpdateOnlySegmentEnum` over the two. Opened for one batch and dropped with it, matching the append-only components, which buffer nothing across calls. The phases meet at `SegmentWriterState`, produced by `LookupSegment::writer_state` and consumed by `UpdateOnlySegmentEnum::open`. It carries the mappings-log tail an appendable writer resumes from, which `UpdateOnlyAppendableIdTracker::new` requires to come from one and the same read of that log. The writer kind follows the id-tracker format that was loaded, not the segment config: the format decides how a point is retired. That difference makes `tombstone_points` take both ids, `(external, slot)`; an immutable segment marks the slot in its deleted-points bitmask, an appendable one records a retirement for the id in its mappings log. The appendable half is implemented — deletes now run end-to-end. `store_points` and the immutable bitmask remain `todo!()`, still waiting on the append-only storages and field indexes. Two bugs surfaced while wiring it up: * A point stored into the write target must not have its old slot retired there: appending records a mapping that supersedes it, and retiring the id on top would take the new slot with it. * A second `apply_batch` through one writer resurrected deleted points. It resumed the log from the `mappings_end` its own first batch had moved past, and appending there cut that batch's entries off. Refused now; lifting it means reloading the segments after a batch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] one batch per writer, enforced by the type system Cleanup pass over the phase split. `apply_batch` now takes `self`. It could only ever serve one batch — the segments are read when the writer opens, and that read is both what a batch resolves against and what its writers resume from — and the runtime guard enforcing that cost a flag, its doc, two imports, a hand-maintained `writes_anything` condition, an error branch and a test. Consuming the writer makes the second call a compile error instead. Also: * drop `LookupSegment::uuid`, which nothing ever read, along with the two parameters and the argument that fed it; * `AppendableSegment::tombstone_points` was a copy of the tracker's own `retire_pending_inserts`; both now go through `delete_points`; * fold the duplicated "segment disappeared mid-batch" error into `LookupSegmentHolder::get`, and restore `write_target_uuid` as an `Option`, which is what two of its three callers wanted; * one fixture helper for the writer tests instead of three copies; * state the mappings-log co-read invariant once, with pointers, instead of three times. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * [UpdateOnly] cut the writer surface down to what it does * `flush()` is gone from both writers and the enum. Both bodies were `Ok(())` and would stay that way: the id tracker persists what it writes before returning, and the deleted-points bitmask writer does not exist yet. The ordering it looked like it enforced — new slots durable before the tombstones retiring the old ones — falls out of call order, since every write is durable when it returns. Bring it back with the first storage that buffers. * `SegmentWriterState` was an enum of one unit variant and one payload, which is `Option`. `writer_state()` returns `Option<AppendableIdTrackerState>`, and `None` reads as what it means: no mappings log to resume, so a delete-only writer. * `LookupVectorData` wrapped a single `Arc<AtomicRefCell<_>>`; the map holds it directly now. * `appendable` joins the five `pub` fields around it, and `is_appendable()` goes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
qdrant-cloud-bot
|
2805fe4e2a |
fix: stop underestimating is_empty / not-null cardinality by 1/3 (re-introduce #10128) (#10141)
* fix: stop underestimating is_empty / not-null cardinality by 1/3 Re-introduce #10128 after its revert so CI can exercise payload_index_test::test_read_operations / test_is_empty_conditions. * test: stop requiring is_empty struct exp to beat plain NullIndex complement estimates use an indexed upper bound (may include soft-deletes); that is not guaranteed to be closer to truth than plain's available/2 guess. Assert upper-bound semantics instead. * test: drop is_empty exp==max assertion That locked in NullIndex implementation detail. Keep result parity and min/max bounds only; document why exp-vs-plain is not checked. |
||
|
|
2c024ba037 |
[UpdateOnly] implement UpdateOnlyChunkedVectors (#10114)
* AI + manual: impl `UpdateOnlyChunkedVectors` * AI: simplify * graceful handling of unexpected file lengths fix test * incorporate updates from #10119 * drop the unused status read on open The vector count loaded at open was never consulted: every batch carries the offset it starts at, and the chunks are reconciled against that offset. Drop the field and the read, and fold both watermark writes into `save_len`. A corrupt status file no longer blocks opening the writer — the first batch overwrites it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix clippy: ensure_chunk_lengths no longer needs &mut self Dropping the status field left it with nothing to mutate. `append_many` keeps `&mut self` — nothing in this module is exported, so the lint reaches it too, but the exclusive borrow is what enforces the single-writer contract the appends rest on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
qdrant-cloud-bot
|
34d3f35fd7 |
Revert "fix: stop underestimating is_empty / not-null cardinality by 1/3 (#10128)" (#10140)
This reverts commit
|
||
|
qdrant-cloud-bot
|
50fe2e8139 |
fix: stop underestimating is_empty / not-null cardinality by 1/3 (#10128)
NullIndex used an arbitrary `exp = 2/3 * estimated` heuristic for complement conditions (`is_empty=true`, `is_null=false`), which caused steady-state approximate counts to under-report by ~35% even with no deletes (see #10120). Use the indexed upper bound as the expected count instead. |
||
|
|
b60f298dea |
Add UpdateOnlyAppendableIdTracker, the append-only ID tracker writer (#10093)
* Add UpdateOnlyAppendableIdTracker, the append-only writer The write counterpart of `ReadOnlyAppendableIdTracker`, producing the two files that tracker already consumes — `mutable_id_tracker.mappings`, an append-only log of mapping changes, and `mutable_id_tracker.versions`, a dense array of one version per slot — through `UniversalAppend`, so the same code drives a local file and an object store. `insert_operations` records a batch of `MappingOperation`s in order: an insert claims the next slot above the highest one in use and reports it, a delete retires an external id and claims nothing. Nothing is rewritten in place, so re-inserting a live id moves it to a fresh slot and supersedes the old one — the update-only shape of an update. `set_internal_versions` extends the versions array. Ids may come in any order but must be exactly the slots the array does not cover yet: a slot below the end would need an in-place overwrite, and a hole would have to be zero-filled — and since "covered by the versions file" *is* the commit signal for readers, that would publish a slot as a live point of version 0 before its data exists. Both are rejected rather than written. Both methods append at an offset they probed for, never at an implicit end: the offset is a compare-and-swap token, so a file that has moved on since the probe is rejected instead of being written twice or in the wrong place. Both have persisted what they wrote when they return `Ok` — append, then run the handle's flusher — and nothing is buffered across calls. The order of the two calls, claim the slot then commit the version, is what makes a crash in between safe: readers ignore slots the versions array does not cover. Cleaning up the slots such a crash abandons is left to the opener, along with repairing a torn tail; the writer fails loudly rather than guessing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Share the versions file format between both write paths `set_internal_versions` was reimplementing what `versions_storage` already knew: entries are `SeqNumberType`-sized, slot `n` lives at `n * VERSION_ELEMENT_SIZE`, a file length that is not a whole number of entries has a torn tail. Every one of those facts existed in two to four places, spelled as inline `/`, `%` and `write_u64::<FileEndianess>`, so the append-only writer could drift away from the in-place one silently. Move them into `versions_storage`, which now owns the format for both writers and both readers: - `write_version` / `read_version`, the entry codec, with a static assertion tying its `u64` to `VERSION_ELEMENT_SIZE` so a change to `SeqNumberType` cannot silently shrink every offset; - `version_offset` and `versions_byte_len`, the slot arithmetic; - `VersionsLayout`, which splits a file length into committed entries and a partial tail. The two writers still react differently — the in-place one truncates the tail, the append-only one refuses it, because an append cannot — but they no longer each work out what the tail is. `store_version_changes`, `load_versions`, `set_internal_versions` and the read-only tracker's live reload all go through it. The write loops themselves stay separate: one seeks to sparse offsets, the other emits a validated consecutive run, and merging them would obscure both. What they share is where the bytes go, which is the part that must not diverge — and a new test pins it down by writing the same versions through both writers and comparing the files byte for byte. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Append mapping and version entries as batches Both writers built one concatenated buffer and handed it to `append`. `append_batch` takes the entries as separate buffers and places them in a single operation — a vectored write locally, one request on an object store — so the entry boundaries reach the backend instead of being flattened away first. Versions are fixed-size, so the entries are the payload's `chunks_exact`. Mapping changes are variable-length, so their bounds are recorded as they are serialized. Both keep the compare-and-swap offset, which `append_batch` validates the same way `append` does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Test the writer against MutableIdTracker end to end The suite leaned on round-trips through the storage loaders, which only restated what the writer had just written. Four such tests are replaced by one that checks the property actually worth having: drive the append-only writer and `MutableIdTracker` with the same points, versions and deletes, open each segment through `ReadOnlyAppendableIdTracker`, and require the two views to be indistinguishable — counts, deleted state, external ids, live points' versions, and id resolution. Versions are compared for live points only. `MutableIdTracker::drop` overwrites the slot with `DELETED_POINT_VERSION`, which an append cannot do, so the append-only writer leaves the point's original version there; neither is observable for a point that is gone. The remaining tests keep what a round-trip cannot show: slot allocation across calls, instances and deletes (three tests folded into one), the rejection of holes and rewrites, and the byte-for-byte agreement of the two writers on the versions file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * better buffer names * fmt * Heal a torn versions tail instead of refusing to write A writer that dies mid-entry leaves the versions file ending inside a slot. The in-place writer already truncated that tail before writing; the append-only writer refused, which left the file unwritable forever since an append cannot truncate. Share the decision — what counts as torn, the healthy length, the warning — in `heal_versions_tail`, and let each writer supply the shrink its backend can do: `set_len` in place, or reading the committed prefix back and putting it in place as a whole file where there is no truncate. Dropping the tail loses nothing: the array covers a slot only once its whole entry is there, so a partial entry belongs to a slot no reader ever saw and no writer counted as committed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Append mappings at the end of the log, not the end of the file Mapping entries vary in length, so no length tells you whether the log ends on an entry boundary. Appending at the file's end therefore could not fail: a torn entry was silently appended after, and every entry from there on was framed off the stray bytes. Carry the boundary instead. `new` takes the offset just past the last complete entry — `ReadOnlyAppendableIdTracker::mappings_read_to`, from the same view that supplies `max_internal_id` — and appends there, which turns a file ending elsewhere into an append offset conflict. On that conflict `heal_mappings` cuts the file back to the log's end, the same read-prefix-and-rewrite the versions file heals with, and writes the batch again. A torn entry and a batch that landed unacknowledged are indistinguishable without parsing, and need not be told apart: neither `max_internal_id` nor `mappings_end` moves before an append is durable, so the retry writes the same bytes at the same offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Move the healing functions into their own file `update_only/mod.rs` had grown to hold the writer, its two public write paths and the two repair routines they fall back on. Split the latter out: `heal_versions` and `heal_mappings` move verbatim into `update_only/heal.rs`, as a second impl block, following the layout the read-only half already uses (`lifecycle.rs`, `live_reload.rs`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Retire inherited pending inserts when opening the writer A slot is spoken for from the moment the mappings log claims it — components may already have written data at it — so the writer must resume above every slot the log ever handed out. Deriving that bound from a reader's point set undercounts it twice over: a claimed slot whose version was never committed is not in the mapping, and one whose external id was deleted afterwards is not among the pending inserts either. Track it in the log instead, as `ReadOnlyAppendableIdTracker::max_claimed_internal_id`, bumped on every insert entry regardless of what becomes of the point, and take it as `UpdateOnlyAppendableIdTracker::new`'s bound. The points on those claimed-but-unversioned slots are the other half. They cannot be adopted: a writer stopped partway through them, so some components hold their data and others do not, and which is unknowable here. They cannot be left alone either, the versions array being dense — covering any slot above one of them publishes it, half-written. So `new` now takes the pending inserts explicitly and retires them, recording a `Delete` per id before the writer can be used at all, which is what makes it fallible. Doing it at construction rather than lazily on the first write means no write path can be added later that forgets to. `set_internal_versions` accordingly stops rejecting holes: it writes the whole run from the end of the array through the highest id given, covering skipped slots with `DELETED_POINT_VERSION` as the in-place writer's seek already does. It gains an upper bound in exchange — publishing a slot means covering every slot below it, so an id the log never claimed is refused. Live-reload semantics are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Skip UpdateOnlyAppendableIdTracker heal tests on Windows Healing replaces the file via atomic_save while an mmap handle is still open, which Windows denies with os error 5. * Drop mmap handles before healing ID tracker files atomic_save cannot replace a path while an mmap is still open on Windows. Copy the committed prefix, drop the handle, then rewrite and reopen. * Trim ID tracker docs and drop redundant helpers Condense the doc comments on the update-only tracker to the style of the sibling modules, merge a duplicate impl block, and remove `read_version` and a debug assert that restates its own operands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Drop the versions layout helpers and inline the arithmetic VersionsLayout, versions_byte_len and version_offset wrapped one divmod and one multiplication between them, and adopting the struct made the live-reload hunk longer than the line it replaced. Compute the committed length where it is needed instead, and let heal_versions_tail and heal_versions return unit, since no caller used the layout they handed back. Also drop the writer's unused max_claimed_internal_id accessor, and fold retires_inherited_pending_inserts_at_construction into retires_inherited_pending_inserts: the merged test asserts the retirement happened before the writer did anything, and commits a real version to the retired slot rather than letting it take the filler, so it still shows the Delete is what hides the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: qdrant-cloud-bot <111755117+qdrant-cloud-bot@users.noreply.github.com> |
||
|
Jojii
|
7c0948628d |
Batched deletion checks in full-scan/exact search (10-35% faster Turbo4) (#10042)
* Batched bitmap checks in full-scan/exact search Instead of walking the deleted bitmap bit by bit and re-checking every point, the new peek_top_visible reads all bitmaps one u64 at a time and handles 64 points in one step. About 1.5x QPS on unfiltered full scans. * Remove debug statement * Extract parallel bitmap scaning into separate helper * Optimizing `BatchedBitmapScan` * Also apply to HNSW full-scan fallback |
||
|
xzfc
|
c8e7ef80a8 |
Batched HNSW: preliminary refactoring (#10052)
* [1] refactor: EntryPoint: derive Copy
* [2] refactor: extract GraphLayers::probe_links_format
* [3] refactor: merge …/graph_links/{links, storage}.rs
* [4] refactor: GraphLinks: inline GraphLinksEnum methods
* [5] refactor: graph_links/view.rs split into view_utils.rs
Later these utils would be used in links_file.rs.
* [6] spelling: clarify error_size
* [7] refactor: TestGraphLinksVectors::{assert_base_vector, assert_link_vector}
* [8] refactor: extract entry-point selection out of GraphLayers::search
* [9] refactor: Introduce GraphWithVectorScorers
* [10] refactor: extract load_or_derive_config
|
||
|
|
0a6cb3b4cf |
[Raw payloads]: read payload as stored bytes in retrieve_raw (#10040)
* segment: read payload as stored in retrieve_raw `retrieve_raw` already hands back vectors as stored; let the caller ask for the payload the same way, so a reader that only relocates a point parses nothing. `RawPayloadFormat` states what the caller wants — no payload, parsed, or as stored — and replaces the `WithPayload` argument, which could express a key selection that a raw read cannot serve anyway. [`MaybeRawPayload`] states what came back, which can differ from the request in one direction only: a payload storage that keeps payloads parsed cannot answer `Raw` with a blob, and now says so instead of encoding a payload for a reader that would parse it straight back. The raw path reaches the blobstore through `read_payloads_maybe_raw`, mirroring `read_payloads` down the payload storage and payload index traits, so it keeps the batched read. Every caller asks for `Parsed`, so this changes no behaviour: the copy-on-write move and the sync comparison need the parsed payload anyway, and the shard transfer switches over with the feature flag that ships the blob to another node. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * segment: always hand out the stored payload blob from retrieve_raw Review follow-up: instead of telling `retrieve_raw` in which form to return the payload, it always returns it as stored and a caller that needs the parsed form decodes it itself. - Drop `RawPayloadFormat` and the payload parameter it replaced: no production caller ever asked for anything but the whole payload, and a selector cannot be applied to an opaque blob anyway. - Drop `MaybeRawPayload` / `MaybeRawPayloadRef`: only `InMemoryPayloadStorage` could produce the parsed variant, and no segment can be built with that storage (`PayloadStorageType` is `Mmap` or `InRamMmap`, both blobstore-backed). `SegmentRecordRaw` carries a plain `Option<RawPayload>`. - `PayloadStorageRead::read_payloads_maybe_raw` becomes `read_payloads_raw` and hands out `Option<&[u8]>`. The in-memory storage keeps payloads parsed, so it encodes on read, producing the bytes an on-disk storage would have written. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * api: decode a received raw payload with the shared decoder `decode_payload` at the gRPC boundary matched on the encoding and parsed the blob itself, duplicating `RawPayload::decode`. Add the inbound conversion from the wire type and let the one decoder do the reading, so another encoding has a single place to be taught. The conversion also rejects an encoding number no variant maps to, which prost would otherwise hand out as the default encoding — a blob from a node that writes payloads some other way must not be read as JSON. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * simplification --------- Co-authored-by: Ivan Pleshkov <ivan.pleshkov@qdrant.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
aa6c5d8403 |
Global quota API (#10035)
* feat: global quota API Memory and disk are node-wide resources, so configuring their thresholds per collection through strict mode makes little sense. Move them behind a single cluster-wide `QuotaManager`. The quota config is seeded from `storage.quotas` in the settings (and so from env vars), overridden by `quota.json` in the storage directory, and updated cluster-wide through a new `SetQuotaConfig` consensus operation which rewrites that file on every peer. Raft snapshots carry it too, so a peer that joins by snapshot picks it up. Quotas are enforced wherever the strict mode memory and disk checks used to run, but no longer gated behind `strict_mode.enabled`: a value set in an enabled strict mode config still wins per resource, the quota is the default. Rejections name both the condition that tripped and the config that governs it. `GET /quotas` reports the config plus current utilization to global read users; `PUT /quotas` replaces it for global manage users. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: cover the quota endpoints in the API consistency checks `test_all_rest_endpoints_are_covered` and the OpenAPI endpoint count both break on any new REST endpoint. Add `GET`/`PUT /quotas` to `ACTION_ACCESS` with their JWT access tests, and bump the expected API count. The quota endpoints stay out of `REST_ENDPOINT_WHITELIST`: that list is for data-plane endpoints reported per-endpoint in metrics. Also add a Raft snapshot CBOR compatibility test — snapshots are exchanged between peers of different versions during a rolling upgrade, so `quota_config` must be absent-tolerant in both directions. Review feedback: persist through `SaveOnDisk`, which already implements the write-before-swap protocol this was doing by hand; validate the config at both persistence boundaries, since a hand-edited quota file or a config arriving through consensus does not pass the REST handler's validation, and a `0%` limit would reject every update forever. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: assert seeding a quota from invalid settings persists nothing Follow-up to review feedback claiming `SaveOnDisk::load_or_init` writes the init value before it is validated. It does not — only `SaveOnDisk::new` persists — but the property matters: were seeding to persist first, invalid settings would leave a `quota.json` that fails validation on every subsequent start, and the node could only be recovered by deleting it by hand. Pin it down with a test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: make QuotaManager the single reader of memory and disk The quota checks measured memory and disk themselves, while the optimizer and the WAL disk watcher each called `fs4::available_space` behind their own ad-hoc caches. Fold all of it into QuotaManager: it owns the readings, the freshness policy, and the limits they are compared against. Moves the module to `lib/shard`, since the optimizer sits below `storage` and has to reach it; `storage::quota` re-exports it, so consensus, the `/quotas` API and StorageConfig are unchanged. The manager is installed as a process singleton by TableOfContent, ahead of loading any collection. - Callers hand in QuotaLimits overrides instead of a StrictModeConfig, and an override can now only tighten. A collection-level admin could raise `max_disk_usage_percent` past a cluster-wide limit that needed global manage rights to set; ties resolve to the quota so the rejection names the knob that actually has to change. - Measurements are cached for 5s, but a reading at or above its limit is never reused: a rejected client retries, and freeing the resource has to take effect on the next request rather than a TTL later. - `fits_on_disk` sizes an optimization against physical free space only, never the configured limits. Optimizations are what free a full disk, so the quota must not be what stops one. - `percent_of` widens to u128 instead of saturating the multiply, which under-reported utilization (failing open) above ~184 PB. - StorageConfig::quotas is optional; absent means no quota is enforced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: don't recover dead replicas onto a node at a resource limit Recovering a dead replica pulls a whole copy of its shard onto this node. If it is already at its memory or disk quota that transfer cannot finish, and starting it only pushes the node further past the limit. Skip it and reconsider on a later sync, once the resource frees up. Adds QuotaManager::check_capacity for work that lands bytes here without being an update. Unlike fits_on_disk the configured limits do apply: taking on a replica is not what frees a full node, so there is no deadlock to avoid by letting it through. The check is hoisted out of the per-shard loop because a node over its limit re-measures on every call, so checking per dead shard would cost a statvfs each. It is free when no quota is configured. Also trims the comments across the quota module, which had grown well past what the code needs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: drop trivial and duplicated quota tests Six tests removed, ~140 lines, with no loss of coverage: - a_rejection_names_the_knob_that_has_to_change asserted that a format! contains its own literals; the message is covered end-to-end by the override test and by test_global_quota.py. - a_node_over_its_quota_has_no_capacity_to_take_on_a_replica was 30 lines for check_capacity, a one-line delegation to check_update the test above it already calls. - a_rejecting_measurement_is_never_served_from_the_cache duplicated the meter test, which proves the same rule with an injected reader instead of inferring it from the real filesystem. - free_space_is_reported_without_enforcing_anything covered a one-line accessor, and its point is what the fits_on_disk test is for. - The two resolve tests and the three meter tests each collapse into one. DiskFit::Unknown keeps its coverage as two lines inside the fits_on_disk test rather than its own fixture. The snapshot compat pair becomes one test: the second only asserted cluster_metadata.is_empty(), which says nothing about quotas — the real check was the deserialize, now an expect that states it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: re-measure free space as the disk fills, and drop a Windows-only assert Two CI failures, both from this branch. e2e test_low_disk: the DiskUsageWatcher I replaced escalated to checking on every call once free space fell below 512 MB. Folding it into the quota manager lost that — available_bytes passed no limit, so a reading was reused for the full 5s however little space was left. On a disk filling as fast as that test fills it, 5s blind is enough to actually run out and the WAL write dies instead of returning "No space left on device". available_bytes now takes a watch_below level and never reuses a reading under it, which is what the old ladder was expressing. The watcher passes max(min_free, 512 MB), so the escalation point is back; above it the 5s cache still costs fewer syscalls than the old 128-call ladder. fits_on_disk gets the same rule by passing required_bytes, so a merge that does not fit re-checks rather than sitting on a stale sample. Windows: fits_on_disk on a missing path was asserted to be Unknown, but GetDiskFreeSpaceEx resolves up to the containing drive and succeeds — as common::disk_usage's own test documents. Dropped; the branch is a two-line else and is not portably reachable. Also renames an_optimization_is_sized_against_the_disk_not_the_quota, which needed explaining to be understood. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: report the quota config in telemetry Reads it from the quota manager rather than the settings, so it is the config the node is actually enforcing: a peer that missed a consensus update reports what it is applying, not what the cluster agreed on. Gated on global access, the same access `GET /quotas` requires, and left out of `PeerTelemetry` — a quota is per-node state, so each peer reports its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: regenerate OpenAPI, and cover the quota in the telemetry key sets Two CI failures from the previous commit. Referencing QuotaConfig from TelemetryData moves its definition earlier in `components/schemas`, because TelemetryData is generated ahead of QuotaStatus. Regenerated rather than hand-patched, so the schema is a pure move. test_telemetry_detail asserts the exact set of top-level telemetry keys. The quota is reported at every level, including 0 — it is three scalars, it is the default the endpoint serves, and it is what explains an update being rejected — so both key sets gain it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: remove `max_disk_usage_percent` from strict mode Disk is a node-wide resource, so a per-collection percentage of it never meant anything a caller could act on: the limit describes how full the *node* is, and which collection the write happens to target has nothing to do with it. The global quota is where it belongs. It shipped in 1.18.2 without documentation, so this drops it outright rather than deprecating. Removal is soft in every direction: StrictModeConfig has no `deny_unknown_fields`, so a client still sending it gets it ignored rather than a 400, and the same struct deserializes the persisted collection config, so collections created on 1.18.2+ keep loading. Proto field 22 is reserved so the number is never reused. The e2e test becomes a quota test — the fixture and the timing are the interesting parts and they carry over unchanged; only how the threshold is configured differs. `max_resident_memory_percent` was documented and stays for now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: enforce the strict mode memory limit outside the quota `max_resident_memory_percent` was folded into the quota as an override, which meant the quota check had to know about strict mode, and retiring the setting would mean unpicking `EffectiveLimit` and `LimitSource` from the resolution logic. It is now a check of its own in `verification/mod.rs`, next to the strict mode checks it belongs with, borrowing only the measurement from the quota manager — which stays the node's single reader of process memory, so both checks still share one reading. Deleting the setting later is deleting one function and its one caller. `QuotaManager::check_update` takes no arguments and consults the quota alone. A collection can still only tighten the limit for itself, because its own check runs in addition rather than in place of the quota's, and each rejection now names the config that has to change without having to carry a `LimitSource` to say so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: enforce the quota on the update path, not in strict mode The quota check sat inside `check_strict_mode_toc_batch` only because that was the one place holding the collection's strict mode config. It doesn't need one any more, and the placement had a real cost: coverage depended on each handler remembering to ask for a strict mode check, and four of the internal update RPCs do — `sync_internal`, which moves the most bytes onto a node, does not. It now runs in `Collection::update_from_client` and `update_from_peer`, which every update passes through. `update_from_client` checks ahead of the shard split, so an operation is accepted or refused whole rather than landing on some shards and being refused by others. Classification moves with it, from ~10 `consumes_memory` impls on request DTOs to one exhaustive `CollectionUpdateOperations::consumes_quota`. The internal enum has variants — raw upserts, conditional upserts, the syncs — that have no client-facing request type, so per-DTO impls structurally could not classify them. Shard-transfer syncs stay excluded, as they are today: a transfer is sized up once before it starts, and refusing its batches partway abandons work that is nearly done only for it to restart from the beginning. Index and named-vector creation reach shards through consensus, past this check — a peer must not refuse what the cluster agreed to — so they keep their pre-consensus check, now against the quota manager directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: deprecate `max_resident_memory_percent` in strict mode Same reason the disk threshold went: memory is node-wide, so a per-collection percentage of it caps how full the *node* is, which has nothing to do with which collection is being written to. The node-wide quota caps it once for everything. Unlike the disk threshold this one shipped documented, in 1.18.0, so it keeps working — as a limit a collection can tighten for itself, never lift — and gets the usual markers: `#[deprecated]` on both Rust structs, `[deprecated = true]` on proto field 21, and `deprecated: true` in the OpenAPI schema, which schemars derives from the attribute. The note names 1.21 as the removal. Recording a version matters here: the audit in docs/plans/overdue-deprecations.md found that this repo has never written a removal deadline down, and members of the 1.15.0 deprecation batch are still in tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: reconcile the quota readers with #9891 #9891 landed effective (cgroup) figures in telemetry while this branch was making QuotaManager the single reader of memory and disk. Two collisions, neither of which git sees. `segment::utils::mem::total_memory_bytes` is now a shared accessor with a 5s TTL, so a cgroup resize is picked up. The quota module had its own `OnceLock` copy that froze the value at startup — exactly what #9891 set out to fix — so it delegates to the shared one instead. Telemetry's new `disk_size` called `common::disk_usage::disk_usage` directly. That reader lost its TTL cache on this branch when the caching moved into the quota manager's meter, so it would have taken an uncached `statvfs` on every telemetry request, and it put a second disk reader back in the tree. It goes through `QuotaManager::disk_capacity_bytes` now, sharing the reading the quota check already takes. Verified it still reports the storage filesystem, matching `df`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: split the quota manager by what each half does `manager.rs` had grown to 450 lines holding three separate jobs: owning the config and its file, taking the readings, and comparing one against the other. - `manager/store.rs` — the `Store` enum, `QUOTA_CONFIG_FILE`, and config validation, which is now the store's own business rather than something every caller has to remember to do first. - `manager/measure.rs` — every reading, and `DiskFit`. The "nothing else calls `statvfs` or reads process RSS" claim is now checkable by looking at one file. - `manager/enforce.rs` — `check_update` / `check_capacity` and the threshold comparison. - `manager/mod.rs` — the struct, its construction, and the config accessors: what a reader needs to see first. Tests move with their subject. No behaviour change: `set_config` used to validate before delegating to the store, and now the store validates on write, which is the same order of operations from the outside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: count copy-on-write deletes toward the quota Dropping a vector or a payload key does not free anything on its own: copy-on-write rewrites the point to produce the version without that field, so storage grows first and is only reclaimed once the optimizer gets to it. Gating those as if they were reclaiming space let a full node keep taking writes that make it fuller. Deleting whole points stays exempt. That is the one operation that has to work on a node at its limit, or there is no way back under it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: say that the quota's reported usage is per node `GET /quotas` returns one cluster-wide config and one set of utilization figures, which reads as though both describe the cluster. They do not: memory and disk are node-local, so `usage` is whatever the peer that served the request is seeing, and a peer under its limit says nothing about the others. Also corrects `resident_memory_percent`, which claimed to be a share of total system memory. It is a share of the memory available to the process, which under a cgroup is the limit rather than the host's RAM. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: treat a node over its quota as a failed replica, not a bad request A quota rejection described the request as invalid (400) and was classified non-transient, which is how the replica set recognises errors that every replica would produce alike. A quota is the opposite: the input is fine and the answer depends on which machine you ask. On the default `wait=false` path that combination silently dropped the write — `update.rs` only deactivates transient failures when nothing completed — leaving the replica Active and permanently missing data its co-replicas had. It is now `InsufficientStorage`, transient, HTTP 507 / gRPC `ResourceExhausted`. So a node that is out of room is handled like one that is offline: - last active replica, or every replica over quota: nothing could take the write, and the client is told the cluster is out of room. - more than one replica: the full node is deactivated through the same path a dead peer takes, and the update stands if enough replicas accepted it. `check_capacity` already keeps recovery off that node until it has room. The check also moves off `update_from_client`, which applied the coordinator's own limit to the whole operation even when it held no replica of the shards being written. Each replica set now gates its own local write and records the refusal as a failure of this peer, so a node only ever answers for itself. `ResourceExhausted` is shared with rate limiting, and the reverse conversion mapped it straight to `RateLimitExceeded` — a forwarded rejection came back as 429. Statuses now carry a marker so the two stay distinguishable across the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: report quota pressure per node, and across the cluster A quota is node-local, so finding out which node has hit one meant asking each of them in turn — and nothing at all showed up in monitoring. `/metrics` gains a `quota_exceeded` gauge for the local node. It is emitted only while the quota is enabled: with it off the value would be a constant 0 that says nothing about the node, and an alert built on it would go quiet rather than fire if someone disabled the quota. Telemetry's `quota` field carries the same verdict alongside the config, since that is where the metric is derived from. `GET /quotas` now answers for the whole cluster. A new `GetQuotaUsage` RPC on the internal `QdrantInternal` service returns what one peer is using, and the handler fans it out to every known peer in parallel. Peers that do not answer are left out rather than failing the request — the nodes that are out of room are exactly the ones most likely to time out, and a partial answer still names them. Outside distributed mode the field is absent rather than a map of one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: report the quota metric per resource `quota_exceeded` was one flag for the whole node, which does not say what to go and fix — disk is freed by deleting or optimizing, memory by unloading. It now carries a `resource` label: quota_exceeded{resource="memory"} 0 quota_exceeded{resource="disk"} 1 A resource with no limit gets no series at all, for the same reason the metric is absent while the quota is disabled: a series that can never reach 1 reads as healthy and would quietly carry an alert that cannot fire. `QuotaManager::exceeded` returns the per-resource verdict, with `None` for a resource this node does not cap. Telemetry reports the same breakdown, since the metric is derived from it. The peer usage RPC keeps a single flag — it sits next to both percentages, so it only has to answer "is this peer refusing writes". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop a no-op error conversion the linter caught `check_global_access` already returns a `StorageError`, so mapping it through `StorageError::from` converted the type to itself and tripped `clippy::useless_conversion`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: hold a tripped quota until usage clears a release margin A resource resting on its limit crosses it in both directions on the noise between two readings, and each crossing is expensive: the node refuses a write, its replica is deactivated, usage dips, recovery starts sending a whole shard copy back, and the arriving data pushes it over again. The loop sustains itself, and every lap costs a shard transfer. A limit now trips at its configured value but only clears once usage has fallen 5 percentage points below it, so the crossing has to be real. The margin is floored at 1%, since a limit smaller than the margin would otherwise be impossible to fall back under and would strand the node. The verdict is carried on the manager rather than recomputed, which makes it the thing reporting shows: expect `exceeded` to be set while the utilization next to it is already back under the limit. Rejections say so too, rather than claiming a limit that is no longer exceeded: Disk usage is at 87% of total capacity. It reached the configured limit of 90% and has to fall below 85% before this node takes writes again. Changing the config clears the verdicts. New limits are a deliberate act, and should not be held back by the margin of a limit that no longer exists. Both resources are now evaluated on every check instead of stopping at the first failure, so a verdict is never left behind reporting a reading that has since been superseded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: make the quota release margin configurable 5 points is a guess about how noisy a deployment's usage is, which is not something one number can be right about: a node whose disk moves in gigabyte steps needs a wider margin than one that creeps, and an operator who wants the old flip-on-every-reading behaviour should be able to ask for it. `release_margin_percent` joins the rest of the quota config, so it seeds from `QDRANT__STORAGE__QUOTAS__RELEASE_MARGIN_PERCENT`, replicates through consensus, and changes with `PUT /quotas`. Defaults to 5 and is filled in when a request omits it, so it always answers with the margin actually in force rather than leaving the caller to assume one. `0` releases as soon as usage is back under the limit. `QuotaConfig` grows a hand-written `Default` for it, since deriving one would have quietly defaulted the margin to 0 and disabled the hysteresis for anyone constructing a config in code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: leave the release margin unset by default, and hold verdicts in atomics `release_margin_percent` is `null` unless someone sets it, rather than materialising 5 into every config. A quota written today then does not pin a number a later release may want to revise, and `{"enabled": false}` still round-trips as itself. `QuotaConfig::limits` resolves it, next to `enabled`, so enforcement never sees the unset case. The verdicts move from a `Mutex<QuotaExceeded>` to one `AtomicBool` per resource. They are judged independently and nothing reads them as a pair, so the lock only added contention to the path every update takes; a verdict that races a concurrent check is re-decided by the next one from a fresh reading. That also drops the tri-state. Only "was this over its limit" has to survive between checks — whether a resource is enforced at all follows from the config and the reading, so it is derived when reporting rather than stored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: drop a comment arguing with a design that was never here Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Andrey VasnetsovandClaude Opus 5
|
4a453329fc |
Reorganize chunked_vectors: dedicated read-only module (#10062)
* Move chunked-vectors read-only code into a dedicated module `chunked_vectors` kept the read-only view (`ChunkedVectorsRead`), its `LiveReload` impl and the writer in flat sibling files. Reshape it to the layout the other components use: a `read_only/` submodule split into `mod.rs` / `lifecycle.rs` / `read_ops.rs` / `live_reload.rs`, with the writer's impls split into `lifecycle.rs` and `write_ops.rs` next to the struct in `mod.rs`. Two things did not survive as pure code motion: - The file-path and metadata readers (`config_file`, `status_file`, `load_config`, `read_status_len`) were associated fns on `ChunkedVectorsRead` that the writer reached through the type. They are now free `pub(super)` fns in `config.rs`, next to the types they read, so both sides get at them without widening visibility across the read-only boundary. `ChunkedVectorsRead::status_file` was `pub` but unused outside the module. - `preopen_chunks` moved from `chunks.rs` to `read_only/lifecycle.rs` beside its only caller; `chunks.rs` keeps the shared chunk-name matching and gains a `chunks_prefix` helper for the two listing sites. `ChunkedVectorsRead` is no longer re-exported from the component root, so importers spell out `chunked_vectors::read_only::ChunkedVectorsRead` — same as the sibling `dense` / `multi_dense` / `sparse` / `turbo` read-only modules. No behaviour change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rename ChunkedVectorsRead to ReadOnlyChunkedVectors Read-only structs are named `ReadOnly*` across the codebase (`ReadOnlyChunkedDenseVectorStorage`, `ReadOnlySparseVectorStorage`, `ReadOnlyDiskIdTracker`, `ReadOnlyNumericIndexInner`, ...), while the `*Read` suffix marks the read-side traits (`VectorStorageRead`, `NumericIndexRead`, `PayloadFieldIndexRead`). `ChunkedVectorsRead` is a struct wearing the trait suffix; rename it to match its peers now that it lives in a `read_only` module. Pure rename, no other change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Kumar ShivenduandClaude Opus 4.8
|
7743616b3b |
Report effective (cgroup) CPU, RAM and disk in telemetry (#9891)
* Report effective (cgroup) CPU, RAM and disk in telemetry
The `system` block reported host-level figures that ignore the limits the
kernel actually enforces on the process:
- `cores` <- sys_info::cpu_num() (host socket count)
- `ram_size` <- sys_info::mem_info() (host total RAM)
- `disk_size` <- sys_info::disk_info() (container root fs)
On any cgroup-limited deployment (containers, Kubernetes pods, systemd
slices) these overstate what Qdrant can use, are misleading for capacity /
oversubscription analysis, and don't match how Qdrant sizes itself.
Report the effective values instead, reusing existing helpers:
- `cores` -> common::cpu::get_num_cpus() (already drives sizing)
- `ram_size` -> segment::utils::mem::total_memory_bytes()
(cgroup limit via cgroups_rs, else sysinfo host total)
- `disk_size` -> common::disk_usage::disk_usage(storage_path)
(data-volume capacity, cached; sys_info host disk fallback)
`Mem::new()` is not free (it builds a sysinfo System and loads the cgroup
memory controller), so `total_memory_bytes()` caches with a 5s TTL — matching
the disk-usage cache — instead of recomputing per call. The TTL (rather than
caching once) means an in-place cgroup memory resize is reflected within a few
seconds, consistent with how `disk_size` and `cores` already behave. The
strict-mode helper in `collection` now delegates to this shared accessor
(dropping its own OnceLock), so it too becomes resize-aware.
ram_size/disk_size stay in KiB to match the previous unit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Simplify comments and code
* Simplify openapi spec
* minor comment improve
* mem: cache total_memory_bytes like disk_usage (5s TTL)
`total_memory_bytes()` mirrors `common::disk_usage::disk_usage`: a small 5s
TTL cache over `Mem::new().total_memory_bytes()`. `Mem::new()` is not free
(builds a sysinfo System + loads the cgroup controller), and the short TTL
keeps the value in step with an in-place cgroup memory resize rather than
freezing at startup. Shared by telemetry and strict-mode, like the disk cache.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Regenerate OpenAPI spec
Add the ram_size / disk_size field descriptions produced by schemars from the
updated telemetry doc comments, keeping the generated spec consistent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* mem: address review — parking_lot mutex, hold lock, saturating_duration_since
- Use parking_lot::Mutex (no poisoning; lock() returns the guard directly).
- Hold the lock across the whole method — single acquisition, simpler.
- saturating_duration_since instead of duration_since (no panic on a cached
timestamp spuriously ahead of now).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
xzfcandLuis Cossío
|
a6a9143afc |
bitpacking_ordered: batched reads (#10038)
* bitpacking_ordered: batched reads * Replace ranges with pairs * "unsufficient" -> "insufficient" --------- Co-authored-by: Luis Cossío <luis.cossio@outlook.com> |
||
|
Luis Cossío
|
c07d57bd8f | chore: fix dead code lints on macos (#10045) | ||
|
|
9f3c07b00d |
feat: UpdateOnlySegment / UpdateOnlyEdgeShard batch writer skeleton (#10021)
* feat: `UpdateOnlySegment` / `UpdateOnlyEdgeShard` batch writer skeleton Mirror image of the read-only pair, for the serverless updater: a shard/segment whose public surface is writes only, built for batches of many tiny operations against remote, append-only storage. Implemented: * `UpdateOnlySegment<S>` with a deliberately narrow open — id tracker, payload storage and one storage per named vector, all cold. No vector index, no quantized vectors, no payload index on the segments the writer only reads from. * `SegmentUpdateView`, the shared home of resolution logic, generic over the component traits (`VectorDataStorageRead` is a `VectorDataRead` without the index, so a segment that opens no index can produce the view). Batched `locate_points` / `point_versions` / `read_stored_points`. * `UpdateOnlyEdgeShard<S>::apply_batch`: fold the batch to one entry per point, locate the points, read only the ones that cannot be resolved from the batch alone, materialize `FullyQualifiedPoint`s, append them and tombstone the slots they replace. `todo!()`, pending the append-only components on the roadmap (appendable `DynamicStoredFlags` and `ChunkedVectors`, an appendable payload blobstore and field indexes): `store_points`, `tombstone_points`, `flush`, and creating the first appendable segment. Filter-selected operations, point sync, conditional upserts and the schema-level operations are rejected up front rather than silently skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: codespell implementor → implementer in SegmentUpdateView docs Co-authored-by: Cursor <cursoragent@cursor.com> * docs: trim update-only writer docstrings to guarantees Less verbose throughout: state each function's contract — ordering, absent-value behavior, preconditions, durability — and drop narration about where types are used or why alternatives were rejected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: fold SegmentUpdateView into UpdateOnlySegment as inherent methods The view was premature: it had exactly one producer, and its trait bounds bought an unexercised option. Resolution (locate / versions / read raw) now lives as inherent methods on UpdateOnlySegment, still generic over the backend. A shared view can be extracted when a second producer appears, e.g. batched CoW moves out of regular segments. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: split edge update_only batch.rs into a module Pure move: mutation.rs (PointMutation fold + materialize), plan.rs (UpdateBatchPlan operation intake), tests.rs. PointUpdates::new/push narrowed to pub(super). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: parallel per-segment batch reads + tombstone every copy of a point locate_points and read_stored_points visit segments in parallel on a dedicated edge-update rayon pool (build_search_pool generalized to build_segment_pool with a thread-name prefix). locate_points now keeps every slot a point occupies, not just the newest copy: a rewrite or delete retires all of them. Tombstoning only the newest slot would let an older duplicate left by an interrupted move outlive the point — and resurrect it after a delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: point_versions returns a map keyed by internal id The id tracker's batch read is keyed by internal id already; returning AHashMap drops the positions_of reverse-lookup adapter. Absent key = unwritten slot, defaulted to version 0 at the caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: accept a deferred threshold when opening UpdateOnlySegment Groundwork for an external rebuilder working the same directory: the cutoff loads slots at or above it into the appendable id tracker's deferred track (same appendable-only filter as ReadOnlySegment). It hides nothing from the writer — resolution runs WithDeferred, so every point still locates at its latest slot. The edge shard passes None until the rebuilder coordination exists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: preview_batch — resolve a batch without writing anything apply_batch and the new preview_batch share one resolution stage (resolve_batch: locate, read, materialize into per-point PointActions), so a dry-run reports exactly what an apply would do. Plus segment_configs(): per-segment configs with the write target marked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: prefetched + parallel segment opens for the update-only writer UpdateOnlySegment::open now mirrors ReadOnlySegment::open: a per-segment CachedFs primed by preopen, config parsed once and handed to open_via. The edge shard opens segments in parallel on its pool, keeping fail-hard semantics. With Populate::No throughout, prefetches transfer no data-file content — only configs, the id tracker and the deleted flags, whose opens consume them whole anyway. Also: ReadOnlyAppendableIdTracker::preopen now tolerates the not-yet-created mappings/versions files of an empty appendable segment, matching its open's contract — previously unreachable because followers skip appendable segments on error, while the writer must open them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: edge-shard-update — dry-run batch upserts against a shard Counterpart of edge-shard-query for the write path: opens an UpdateOnlyEdgeShard over a local directory or S3/GCS object storage, generates random points shaped by the shard's own schema (segment config + payload-index schema), and logs what applying them would do — locations, versions, actions, tombstones — via preview_batch. Nothing is written: the write half is still todo!(). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: box PointAction::Store to appease clippy::large_enum_variant A resolved point is ~384 bytes while every other variant is empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: adapt to dev's dead-code sweep (#10030) Restore NamedVectors::remove_ref — removed as dead on dev, but the batch fold's DeleteVectors arm is now its first caller. Drop the allow(dead_code) on segment::update_only (no longer needed) and switch the writer's unread fs field to expect(dead_code), per the new ast-grep rule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
xzfc
|
75385df69f |
Remove dead code (#10030)
* Remove dead code * Remove unused dependencies * `allow(dead_code)` -> `expect(dead_code)` * ast-grep: rule-tests/*-test.yml => tests/*-test.yml For brevity. * ast-grep: forbid allow(dead_code) |
||
|
|
0a16a62f99 |
feat: io_uring setting to control which components use the io_uring backend (#10008)
* feat: `io_uring` setting to control which components use the io_uring backend A few components have both an mmap and an io_uring variant reading the very same files: the immutable dense vector storages, the single-file TurboQuant storage, and the mmap payload storage. Until now the choice was a side effect of `async_scorer` — a vector-search knob — plus, for the payload storage, a feature flag that was parked off because io_uring is ~2x slower than mmap when the data fits the page cache (#9310, #9409). Add `storage.performance.io_uring`, optional, with two modes: - unset (default): unchanged behaviour. The vector storages keep following `async_scorer`; the payload storage stays on mmap. - `disabled`: no component uses io_uring. - `auto`: a component uses io_uring when its memory placement is `cold` (data is left on disk, so reads hit the disk and there is something to gain), its feature flag allows it, and the kernel supports io_uring. Components meant to sit in RAM keep using mmap. The decision lives in one place, `segment::common::io_uring::use_io_uring`, so the openers no longer each reach for the async-scorer global. Kernel support is now probed up front through `is_io_uring_supported()` instead of opening a file and falling back on error. `async_payload_storage` now defaults to on: it no longer decides anything by itself, it only lifts the ban, and the payload storage no longer follows `async_scorer` at all — so turning it on cannot silently move an existing `async_scorer: true` deployment onto the slower path. Which backend a component ended up on depends on the config, the placement and the kernel at once, so report it in `SegmentInfo`: `vector_data[name].io_backend` and `payload_storage_io_backend`, both `"mmap" | "io_uring"`, absent for components that have no such choice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Trim comments, drop trivial tests Two tests were only restating their own implementation: `test_mode_round_trip` round-tripped the encode/decode pair next to it, and `test_io_uring_config` checked that serde deserializes a two-variant enum. The mode matrix test stays, it is the one that pins the semantics. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Flatten `IoBackend` in OpenAPI, derive `JsonSchema` for `IoUringMode` Per-variant doc comments on a plain string enum make schemars emit a `oneOf` of anonymous single-value objects instead of a flat `enum`. Move the variant descriptions into the enum doc, as `Memory` and friends already do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Update lib/segment/src/vector_storage/turbo/turbo_vector_storage.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Update lib/segment/src/types.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Update lib/segment/src/types.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Update lib/segment/src/types.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Update lib/segment/src/types.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * upd openapi schema * Update lib/common/common/src/flags.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Require kernel io_uring support in the async-scorer fallback `use_io_uring` returned `get_async_scorer()` verbatim when the `io_uring` setting is unset, so an enabled async scorer on a kernel without io_uring opened the io_uring storage, failed, and fell back to mmap with an error log per segment. Gate that branch on `is_io_uring_supported()` too, like `Auto` already is, so the component just stays on mmap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * upd openapi schema --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> |
||
|
Andrey Vasnetsov
|
9e53738c19 | do not generate oneOf (#10011) | ||
|
Luis Cossío
|
1df16a50e4 | flush after prefilling deleted vectors (#9992) | ||
|
Jojii
|
9df011ab3d | Make folder structure consistent (#9990) | ||
|
|
be543561e5 |
Add Logstore and Blobstore wrapper (#9673)
* Gridstore: introduce storage operating mode in config Add a mode field to the gridstore config, selecting between the dynamic mode (current behavior, the default) and the upcoming serverless mode. The mode is specified through StorageOptions on creation, persisted in config.json, and read back first when opening so the correct variant can be selected automatically. Configs written before this field existed deserialize as dynamic. For now, selecting the serverless mode returns an error; the variant itself is added in follow-up commits. * Gridstore: move dynamic implementation into dedicated module Mechanical move of the current Gridstore implementation into gridstore/dynamic.rs as DynamicGridstore. The public Gridstore struct becomes a thin wrapper holding a mode variant enum, propagating every call into the selected variant. For now the enum only has the dynamic variant; the serverless variant is added in follow-up commits. No logic changes to the dynamic implementation itself: only visibility, the config parameter now passed into open (the wrapper reads it first to select the mode), and open_or_create staying on the wrapper. * Gridstore: add serverless tracker Add the append-only mapping tracker for the serverless storage mode. The tracker file is a plain array of 16-byte mapping entries without any header: the number of mappings is defined by the exact file length, and the entry index is the point offset. The file starts empty and only ever grows by appending, existing bytes are never rewritten. Mappings must be set in monotonically increasing point offset order; skipped offsets are backfilled as zeroed entries which decode as None. New mappings are buffered in memory and appended with a single write per flush. A flush with a stale target is a no-op so bytes are never written twice. A torn trailing entry (file length not a multiple of the entry size) is ignored when reading and truncated away when opening writable. Unlike the dynamic tracker, the file is read and written directly with positional file IO instead of memory mapping, as serverless environments do not handle memory mapped files well. * Gridstore: add serverless storage variant Add the append-only gridstore variant for serverless deployments, which restrict IO to appending to files: existing bytes can never be rewritten, and IO is expensive so as few files as possible are used. The variant stores all value data in a single page file next to the serverless tracker and the storage config, three files in total. Both data files start empty and only ever grow by appending; there is no preallocation, no used-block bitmask and no gap/region bookkeeping. Values are appended at put time at the next block aligned offset, with the zero padding included in the write so it lands exactly at the end of the file. Mappings are buffered and appended to the tracker with a single write per flush, after the page file is synced, so a mapping on disk never points at data that is not durable. Values cannot be updated or deleted, and must be put at monotonically increasing point offsets; violations are rejected before any data is written. Files are read and written directly, never memory mapped. The mode is selected through StorageOptions on creation and picked up automatically from the persisted config when opening. * Gridstore: serverless support in reader and view Extend the read-only GridstoreReader and the GridstoreView with the serverless mode, keeping both public types unchanged: like the writable Gridstore they now hold a mode variant internally, selected automatically from the persisted config when opening. The serverless reader holds the tracker and page directly and reads the files positionally, without memory mapping. A live reload re-reads the mapping count from the exact tracker file length (there is no size header), ignoring a torn trailing entry, and never truncates as it is read-only. Value reads always go directly to the file, so newly appended data is readable without remapping anything. * Gridstore: document storage operating modes * Gridstore: review fixes for the serverless mode Hardening and cleanup from a review pass over the new serverless storage variant: - Batch the reader side iteration like the writer already did, instead of materializing tracker mappings for the full range in one go, which could transiently allocate gigabytes on large storages. - Recover the append cursors when a positional write fails partway: truncate the file back to the tracked length so a retried append or flush never rewrites bytes that already landed in the file. - Validate page addressability before appending value data, a rejected put must not grow the page file. - Cross-check tracker and page consistency when opening: mappings that reference value data past the end of the page file (e.g. after a partial copy or restore) now fail fast instead of surfacing as opaque read errors per point. - Reject value pointers into any page other than page 0 on the serverless read path with PageNotFound, matching the dynamic mode contract, instead of silently reading from a wrong location. - Refresh the reported storage size on reader live reload even when no new mappings were flushed, unflushed value data may have grown the page file already. - Validate configs read from disk: a corrupt config with zero sized blocks, pages or regions is now rejected when opening instead of panicking on a division by zero later. - Classify rejected serverless puts as UnsupportedOperation, consistent with rejected deletes, so they don't surface as user-facing validation errors at the segment level. - Deduplicate the compression dispatch into Compression::compress and Compression::decompress, and the serverless file create/open patterns into shared direct IO helpers, so the two modes and files can't silently drift apart. * Gridstore: cover both operating modes in mode-agnostic tests Parameterize the gridstore tests that exercise mode-agnostic behavior over both the dynamic and serverless mode with rstest, using a single and bulk put/get roundtrips, storage files, basic persistence, corrupt config rejection, batched read congruence, reader live reload, and the different block sizes. Mode specific expectations branch inside the tests: expected file names, storage size semantics (whole blocks vs exactly packed bytes), value pointer layout (page spill over vs a single packed page), and gaps (created by deletes in dynamic mode, by skipped puts in serverless mode). Dynamic-only internals assertions are kept behind a mode check. Tests around updates, deletes, page spanning, block reuse and other dynamic-only behavior intentionally stay dynamic; the serverless specific format invariants remain covered by the dedicated serverless tests. * Gridstore: port serverless specific tests from sibling branch Source the serverless specific test cases that the serverless-gridstore-updates branch added, adapted to the dedicated variant implemented here (distinct file names, headerless tracker with 16 byte entries, a single packed page without trailing padding, and rejected re-puts): - writes only ever append: tracker and page files only grow and previously written bytes stay byte-for-byte untouched - new mappings land exactly at the end of the tracker file, which always covers the exact number of mappings - mapping gaps are zero-padded on disk and survive reopening - values are packed back to back at block aligned offsets, the page file ends exactly at the last value - serverless mode never creates nor reports block flag files - a flusher persists exactly the mappings that existed at its creation, later puts stay pending - a config claiming the wrong mode fails loudly in both directions instead of loading the incompatible file format of the other mode Tests around their mode switching, page spanning and tolerated deletes don't apply to this design and are intentionally not ported. * Gridstore: test serverless production risk scenarios Add tests for the operational aspects that matter before serverless mode goes to production, each covering a scenario that wasn't evaluated yet: - Replayed puts of already persisted offsets (a WAL redo after a crash where the flush completed but was never acknowledged) are rejected without appending anything, and max_point_offset is the exact offset a replay must resume at. - The accepted crash case of a tracker file extended with zeroed bytes: the entries count as permanent None mappings, can never be put again, and the storage stays consistent and writable past them. - The read-only reader never modifies the files: opening over a torn tracker tail, reading, iterating and live reloading leave both files byte-for-byte untouched. - A multi-round put/flush/reopen cycle always exposes exactly the flushed prefix, with the mapping count matching the exact tracker file length and unflushed offsets reusable. - An append beyond the maximum addressable block offset is rejected before writing anything, keeping retried puts from growing the page file unboundedly. * Gridstore: rename serverless mode to append-only, split into module Rename the mode after its defining characteristic instead of its deployment target: files only ever grow, existing bytes are never rewritten. Renames Mode::Serverless to Mode::AppendOnly (persisted as "mode": "append_only") and the on-disk file names to append_only_tracker.dat and append_only_page_0.dat. The serverless deployment motivation stays in the documentation. Also split the single 2300 line serverless.rs into an append_only module with dedicated files for the storage, page, view, reader and tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Use universal IO in Gridstore * Include upstream preopen logic in new Gridstore variant * Gridstore: buffer append-only value writes until flush In append-only mode, put previously wrote the value data to the page file right away, one write operation per put, while mappings were already buffered and batch persisted on flush. Buffer value writes the same way: both the value and its mapping now only land on disk once a flush cycle executes. This batches all new value data into a single write operation per flush, which is significantly more efficient on S3 based storage where every write is a costly operation. A flush now performs exactly two writes: one appending all buffered value data to the page file, one appending all pending mappings to the tracker file, in that order, so a mapping on disk never points at value data that is not durable. The page mirrors the tracker's pending mechanism: an in-memory buffer that is byte for byte the next append (zero padding between block aligned values included), a watermark captured at flusher creation so puts made during a flush stay buffered, a stale-flush no-op guard so appended bytes are never written twice, and truncate-back recovery on failed writes. Reads transparently serve buffered values from memory. As a side effect, a crash between flushes now leaves nothing on disk at all, where the write-through approach left orphaned value bytes in the page file. The buffered data is held in memory until the next flush, bounded by the flush cadence. Universal IO filesystem handles are now required to be Send + Sync, so the flusher closure can carry one to grow the page file at flush time; all existing backends already satisfied this. * Gridstore: rename inner DynamicGridstore to Gridstore The dynamic variant keeps the Gridstore name; the outer dispatching type will be renamed to Blobstore in a follow-up. Until then the inner type is referred to as dynamic::Gridstore to distinguish it from the outer type. * Gridstore: rename append-only variant to Arenastore The append-only variant stores all value data in a single ever-growing page, allocating space by appending, hence: arena store. * Gridstore: rename outer storage type to Blobstore The outer type dispatching between the two storage variants is now called Blobstore, being more generic than Gridstore. This frees up the Gridstore name, which now exclusively refers to the dynamic mode variant, next to Arenastore for the append-only variant. Storage components keep using the outer type, so they now use Blobstore. The gridstore crate name, GridstoreError, and the persisted names (config.json mode, payload config storage_type) are unchanged. * Gridstore: split Gridstore and Arenastore into dedicated modules The outer module is now blobstore, matching the Blobstore type it defines. The two storage variants each get their own submodule: the dynamic Gridstore moves from dynamic.rs into gridstore/ with its reader and view extracted from the shared files, mirroring the arenastore/ module (previously append_only/) which already had this layout. * Rename gridstore crate to blobstore The crate is named after the outer Blobstore storage type it provides. The gridstore name lives on in the dynamic mode variant. GridstoreError and the persisted names (config.json mode, payload config storage_type) are unchanged. * Arenastore: pack values back to back across multiple pages Drop the block alignment from the append-only mode: values are packed byte to byte, without blocks, and the tracker offset is now a plain byte offset within the page. Blocks and regions are dynamic mode concepts; their page size constraints no longer apply to append-only configs. Bring back support for multiple pages. Once appending a value would grow the current page beyond the configured page size, a new page is started, bounding the size of and the number of appends to each file: object stores like S3 Express limit the number of appends per object. A value larger than the page size gets a page of its own; values never span pages. A rollover creates the new, empty page file at put time; the value data itself stays buffered until the next flush, which appends to each touched page with a single write, using per-page watermarks captured at flusher creation. The reader scans for consecutively numbered page files when opening, validates the most recent mappings against them, and adopts pages created since on a live reload. * Blobstore: rename dynamic mode to mutable Rename Mode::Dynamic to Mode::Mutable, and the persisted config value with it: config.json now writes "mode": "mutable". There is no compatibility alias for "dynamic", released versions never wrote the mode field (a missing field still defaults to mutable), only unreleased storages did. The Gridstore type and module names for the mutable variant are unchanged. * Fix Edge compilation due to package rename * Review remarks * Extract Gridstore preopen into module * Rename Arenastore files * Use universal IO for append operations * Rename GridstoreError to BlobstoreError The error type belongs to the Blobstore crate and is shared by both the Gridstore and Arenastore variants, so it follows the crate naming. Also update the user-facing error messages that referred to the old name. * Split config into per-variant types * Rename Arenastore to Logstore Rename the Arenastore type to Logstore, including the reader, view, config, module and variant names. The storage file names follow: log_page_{n}.dat and log_tracker.dat. The persisted mode tag stays "append_only". * Move bitmask module into the Gridstore variant The bitmask tracks free blocks, which only exists in the mutable mode. Move the module from the crate root into the Gridstore variant that owns it. It stays re-exported at the crate root because the bitmask benchmark needs a public path. * Move pages module into the Gridstore variant Like the bitmask, the block based pages module is only used by the mutable mode. Move it from the crate root into the Gridstore variant that owns it. The Logstore variant has its own page implementation. * Use universal IO for every Logstore operation Replace the direct_io module with universal IO in the append-only tracker, making the whole Logstore go through a universal IO backend bounded by UniversalRead and UniversalAppend: - The tracker is generic over the backend now. Reads go through UniversalRead with the caller's access pattern, flushes land as one atomic append with the same offset compare-and-swap recovery as the pages: a retried append after a lost acknowledgement is adopted instead of appended twice. A torn trailing entry is still truncated away on writable open, through a fresh handle since shrinking is not supported through an open one. - The reader now schedules a prefetch for the tracker file too, it no longer bypasses the backend. - The config write, clear and wipe use the backend file operations instead of local filesystem calls, matching the Gridstore variant. * Batch reads in Logstore read_values Apply the same batching logic as the Gridstore variant: resolve all mappings first, then fetch the value data, both through the backend's read pipeline so async backends can serve the reads in parallel. The tracker gains a batched lookup mirroring the mutable tracker's iter, serving pending mappings and out of range point offsets directly from memory. The pages gain a batched value read; unflushed values are served from the in-memory buffers, and since values never span pages each value is a single read without reassembly. Like in the Gridstore variant, the callback may now be invoked in a different order than the requested point offsets. * Better describe logstore live reload ordering * use enum for options, swap `*Options`<->`*Config` naming * don't wrap enum in struct * ditch unused `StorageConfig`, make deserialization more ergonomic * rename `*Options`->`*Config` * make `preopen` non-blocking * fixup! ditch unused `StorageConfig`, make deserialization more ergonomic * fixup! use enum for options, swap `*Options`<->`*Config` naming * fixup! don't wrap enum in struct * fix rebase * use `populate` param in Logstore * test: failing repro of stale page after live reload across rollover A reader that live-reloads between a page rollover and the following flush adopts the new, still empty page. The previous page is then no longer the last one and is never reloaded again, so the tail that the next flush appends to it stays invisible to the reader forever: value pointer at byte 100 with length 100 is out of range AppendOnlyPages::live_reload only reloads the last held page, assuming earlier pages never change once a newer page exists. But the rollover creates the new page file eagerly at put time, while the previous page's buffered tail only lands at the next flush (see test_rollover_writes_no_value_data_before_flush), so a page can keep growing on disk after its successor exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: reload all pages that grew * use Fs in `open_or_create` * fix: publish tracker mappings only after the pages reload `AppendOnlyTracker::live_reload` observed the mapping count and made it visible in one step, before `LogstoreReader::live_reload` reloaded the pages. Every failure path in the page reload -- `list_files`, reopening a grown page, opening an adopted one, the truncation check -- therefore left the reader with mappings referencing value data it never loaded, so reads in the new offset range fail until a later reload happens to succeed. The edge refresh loop keeps a segment whose reload failed, expecting it to keep serving its pre-refresh state, which it then does not. Split observing from publishing: `reload_count` refreshes the handle and returns the count as a `PendingReload` token, `commit_reload` publishes it. The reader still observes the tracker first, as the writer persists pages before the mappings referencing them, but only commits once the pages are loaded. Reopening without committing is harmless: reads stay bounded by the unchanged count, and the bytes below it never change. A partial failure inside the page reload needs no unwinding, pages running ahead of the tracker is the safe direction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: batch the value reads in Logstore iteration `LogstoreView::iter_range`, the path behind `Logstore::iter` and `LogstoreReader::iter`, fetched the mappings for the whole range with a single read but then read the values themselves one at a time, serially. Gridstore routes its `iter` through `read_values` and pipelines both stages, so a full scan of an append-only storage was the one read path without batching -- one blocking round trip per value on the object store backends this variant exists for. It is reached by payload storage iteration and by the payload index build, which scans every payload. Feed the pointers into `read_batch_values` instead, keeping the single contiguous tracker read, which is better than the per-offset pipeline scheduling Gridstore does on that side. Values are now delivered through the read pipeline, so the callback may be invoked out of order, as it already could be for Gridstore's `iter` and for `read_values` in both variants. Both segment callers are order independent. Tests that happened to rely on the mmap backend completing reads in scheduling order now sort before comparing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: don't run the failed-page-reload test on Windows The test shrinks a page file out of band to make the page reload fail, but Windows refuses to resize a file while the reader holds it mapped, which it does by construction here: "the requested operation cannot be performed on a file with a user-mapped section open". The panic is on the injection itself, the code under test never runs. There is no portable injection. Truncating a page the reader holds is what the check under test detects, so the mapping cannot be avoided; failing the adopted page open instead needs a listed but unopenable file, and `local_list_files` descends into matching directories rather than listing them; failing the directory listing needs the storage directory removed, which Windows also refuses while pages are mapped. The storage itself is fine on Windows, its append path grows mapped pages there and every other Logstore test passes. The logic under test is platform independent and stays covered elsewhere, with the tracker half of the guarantee pinned by `test_live_reload`, which runs on every target. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Luis Cossío <luis.cossio@outlook.com> |
||
|
Ivan PleshkovandClaude Opus 4.8
|
7469834d9b |
[TQDT] TQ dense/multi vector storage consistency (#9953)
* [TQDT] Align TQ vector storage layout with the reference dense storage
Make the TurboQuant vector storage structurally mirror the reference dense
(and multi_dense) storages, down to file names and their contents:
- Rename files + structs to the dense convention:
- immutable.rs -> turbo_vector_storage.rs (ImmutableTurboVectorStorage ->
TurboVectorStorageImpl)
- appendable.rs -> appendable_turbo_vector_storage.rs
(AppendableTurboVectorStorage -> AppendableMmapTurboVectorStorage)
- multi.rs -> multi_turbo/appendable_mmap_multi_turbo_vector_storage.rs
(TurboMultiVectorStorage -> AppendableMmapMultiTurboVectorStorage)
- ReadOnlyTurboMultiVectorStorage -> ReadOnlyChunkedMultiTurboVectorStorage
- Thin out turbo/mod.rs to module declarations + re-exports: open_* fns move
into their storage files, consts + turbo_storage_roundtrip into shared.rs,
and TurboScoring / TurboMultiScoring join the other TQ traits in
vector_storage_base.rs.
- Split read_only/ into the chunked storage (read_only/) and the single-file
storage (read_only/immutable/), each with the mod/lifecycle/live_reload/
read_ops 4-file layout, mirroring dense/read_only/.
- Introduce multi_turbo/ mirroring multi_dense/, with its own read_only/
submodule holding ReadOnlyChunkedMultiTurboVectorStorage.
- Relocate the storage test suites to
tests/test_appendable_turbo_vector_storage.rs and
tests/test_appendable_multi_turbo_vector_storage.rs, paralleling the
dense/multi_dense integration test files (tests moved verbatim, no new
tests added).
- Fix a gpu-gated VectorStorageEnum match that referenced stale DenseTurbo /
DenseTurboAppendable variant names.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* are you happy fmt
* fix after rebase
* [TQDT] Address review feedback on TQ vector storage split
- gpu tests: use the real `VectorStorageEnum::DenseTurboAppendableMemmap`
variant (the old `DenseTurboAppendable` name never existed post-rename, so
the gpu-feature test failed to compile — missed because `cargo build
--features gpu` does not compile the `#[cfg(test)]` code).
- memory_reporter: report `DenseTurboUring` files as `FileStorageIntent::OnDisk`
like the other io_uring variants; io_uring never mmap-caches, so delegating
to `is_on_disk()` could wrongly report `Cached` for a populated backend.
- turbo_vector_storage: fix the misleading `insert_tq_bytes` doc comment — the
single-file backend rejects the upsert via `?`, so `set_deleted` is never
reached.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [TQDT] Fix clippy::wildcard_enum_match_arm in read-only routing test
Spell out the non-routing `VectorStorageType` variants instead of `_`, so a
future added variant fails the match rather than silently mapping to `false`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [TQDT] Fix stale Turbo4 storage-variant assertion in quantization test
The segment is built with the default (appendable/chunked) storage type, so a
Turbo4 datatype now lands in `DenseTurboAppendableMemmap`, not the single-file
`DenseTurboMemmap`. The assertion was left on the pre-split variant; align it
with the non-turbo branch, which already expects the appendable variants.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* are you happy fmt
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Andrey VasnetsovandClaude Fable 5
|
94fdd0e746 |
Support memory placement in service-level storage config defaults (#9950)
* Support memory placement in service-level storage config defaults Follow-up to #9684: `storage.payload.memory` and `storage.collection.vectors.memory` set service-wide placement defaults for newly created collections, deprecating `storage.on_disk_payload` and `storage.collection.vectors.on_disk`. Defaults resolve as: request `memory` > request legacy flag > service `memory` > service legacy flag; exactly one level is filled to avoid spurious memory-vs-legacy mismatch warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Mark hnsw_index.on_disk deprecated in config.yaml, document memory option Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Daniel Boros
|
3b77388f7e |
fix: fold appended points' deletion into read-only live_reload (#9948)
* fix: fold appended points' deletion into read-only live_reload * fix: batch appended-deletion reads in read-only live_reload * fix: clamp appended-deletion reload to persisted flag length |
||
|
Luis Cossío
|
d32f738c1f |
[CI] Enforce no default impl for batch methods (#9939)
* [AI] Add ast-grep rule to avoid default batch trait methods * reword * fix `FullTextIndexRead::check_match_batch` * move to `tools/ast-grep/` * Add tests * pin ast-grep version * fix spelling |
||
|
Jojii
|
89cf9295a9 | Remove wrong debug assertion (#9945) | ||
|
|
542f722105 |
[TQDT] Support Turbo4 in the read-only vector storage path (#9925)
* feat(segment): support Turbo4 in the read-only vector storage path Read-only segments (UniversalRead-backed: mmap / cache / remote) could not open a `Turbo4`-typed vector storage — every dispatch arm returned "not yet supported". This adds the missing read-only TurboQuant storages so read-only segments can retrieve and score TQ-typed vectors. - Reuse the existing, correct TQ scoring: extract `TurboScoring` / `TurboMultiScoring` traits and generalize `TurboQueryScorer` / `TurboCustomQueryScorer` (+ multi) and `raw_turbo_*_scorer_impl` from the concrete `TurboVectorStorage` to `&impl TurboScoring`. Scoring logic is untouched. - Split `DenseTQVectorStorage` / `MultiTQVectorStorage` into read-only (`*Read`) + write supertraits, mirroring the dense storages, so the read-only storage need not implement the ingest path. - Add `ReadOnlyTurboVectorStorage<S>` (single-file + chunked backends) and `ReadOnlyTurboMultiVectorStorage<S>` over `QuantizedStorage<S>` / `QuantizedChunkedStorageRead<S>` + `InMemoryBitvecFlags`; the quantizer is rebuilt from `(dim, distance)`, so nothing beyond the encoded bytes and flags is read from disk. - Wire `DenseTurbo` / `MultiDenseTurbo` variants into `VectorStorageReadEnum` (dispatch, scorer, live-reload) and replace the `Turbo4` stubs in the read-only `open` / `preopen`. - Round-trip tests: writable TQ -> reopen read-only, asserting byte-exact raw TQ bytes, identical dequantization, deletion flags and per-point scores matching the writable scorer (dense single-file + chunked, and multi). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * review remarks * Split turbo read_only.rs into a module Mirror the layout of the other read-only vector storages: split the oversized read_only.rs into mod.rs (struct definitions + shared encoded backend enum), lifecycle.rs (preopen/open), read_ops.rs (retrieval + scoring trait impls) and live_reload.rs. Co-authored-by: Cursor <cursoragent@cursor.com> * review remarks * fix after rebase --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Arnaud GourlayandClaude Fable 5
|
f8b5a9ce60 |
Tests: use SmallRng for RNG-bound test data generation in segment (#9889)
The turbo model tests, HNSW graph properties tests, and id-tracker mapping tests generate their datasets with StdRng (ChaCha12 in rand 0.10). Switching to SmallRng (Xoshiro256++) makes the turbo model tests ~10-14% faster and the 400k-mapping id-tracker test ~30% faster. All affected tests pass with the changed seeded sequences, including the tolerance-carrying turbo model comparisons. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
xzfc
|
48e710ce0b |
Cleanup UniversalRead methods interface (#9934)
* use common::generic_consts::{Random, Sequential};
* UniversalRead::read_batch: generic over E
* UniversalRead::read_batch: pass `AccessPattern` as ZST arg
* UniversalRead::read_bytes_iter: pass `AccessPattern` as ZST arg
* UniversalRead::read_iter: pass `AccessPattern` as ZST arg
* UniversalRead::read: pass `AccessPattern` as ZST arg
* UniversalRead::read_bytes: pass `AccessPattern` as ZST arg
|
||
|
xzfc
|
7e99cdd86a | UioResult (#9933) | ||
|
Luis CossíoandClaude Fable 5
|
fca67539f4 |
Mandatory batch impls (#9935)
* make `EncodedStorage::for_each_batch` mandatory * make `DenseVectorStorageRead::for_each_in_dense_batch` mandatory * make `DenseTQVectorStorage::for_each_in_dense_batch` mandatory * make `DenseTQVectorStorage::read_dense_tq_bytes` mandatory * make `QueryScorer::score_stored_batch` mandatory ...and implement for tq multivectors * [AI] make `IdTrackerRead::internal_versions_batch` mandatory Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * [AI] make `IdTrackerRead::external_ids_batch` mandatory Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * [AI] make `DiskMappingsSource::resolve_internal_batch` mandatory Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
xzfc
|
ba3043b343 | ConditionChecker::check_batched: use in hnsw (#9845) | ||
|
xzfcandLuis Cossío
|
9bb75ea2d1 |
Batched ConditionChecker (#9740)
* ConditionChecker::check_batched: trait + ConditionCheckerEnum * check_batched for OptimizedFilter * OnDiskPointToValues::values_iter_batch: improve performance * OnDiskPointToValues::values_iter_batch: update interface Accept bitvec, call on every point, pass UserData. * check_batched for geo index * geo_index tests: add same_geo_index_between_points_with_dups_test Catches broken load_from_on_disk/for_all_points_values. * check_batched for map index * check_batched for numeric index * check_batched for full-text index * tests for check_batched * Review fixups - default_check_batched: avoid running pred twice at boundary - condition_checker: use explicit match over matches!/if-else - Partitioner: use assert! over debug_assert! Co-authored-by: Luis Cossío <luis.cossio@outlook.com> * ConditionChecker::check_batched: &mut self -> &self * ConditionChecker::check_batched: make mandatory --------- Co-authored-by: Luis Cossío <luis.cossio@outlook.com> |
||
|
Jojii
|
0745c36c8f | Rename WrongVectorBytesSize -> MalformedVectorBlob (#9929) | ||
|
Jojii
|
c561e433c6 |
[TQDT] upsert raw malformed blob multi + sparse vectors (#9904)
* Fix malformed vector upsertion for multivecs and sparse-vecs too * Shorten comment * Fix after rebase |
||
|
Ivan Pleshkov
|
1353d54eb5 | [TQDT] Fix/upsert raw malformed blob badinput (#9886) | ||
|
Andrey VasnetsovandClaude Fable 5
|
0595333a65 |
Batch IdTracker read operations, pipelined in disk-resident trackers (#9809)
* Add batch id-tracker lookups, pipelined in the disk-resident trackers
Introduce batch counterparts for the per-point IdTracker read operations
(external->internal resolution, internal->external lookup, version lookup,
deleted checks) and implement them with pipelined reads in the disk-resident
trackers, where the per-point path costs one storage round-trip per lookup:
- StoredBitSlice::get_bits_batch: dedupe the containing u64 elements and
fetch them through one read_batch pass.
- DiskMappingReader::lookup_batch: group e2i keys by run block, read each
unique block once, binary-search all its keys.
- DiskMappingReader::external_ids_batch: schedule all i2e data slots plus
the deduplicated is_uuid bitmap bytes in one pass.
- DiskMappingsSource::{points_deleted_batch,resolve_internal_batch,
resolve_external_batch}: shared batched resolution for both disk trackers.
- IdTrackerRead::{external_ids_batch,internal_versions_batch}: trait-level
batch methods with loop defaults for in-RAM trackers; overridden (along
with resolve_external_ids) in DiskIdTracker and ReadOnlyDiskIdTracker,
and forwarded through both tracker enums so the overrides dispatch.
Wire the batch operations into the read paths: search result processing
(external ids + versions), scroll filtered_read_by_index (chunked), and
HasId condition conversion/cardinality estimation. retrieve() already goes
through resolve_external_ids and picks up the batched dispatch.
Batch read-by-id keeps the read-only tracker's laziness: no full
deleted-set materialization (covered by test).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Rework batch id-tracker interfaces to iterator inputs and callback output
Review follow-up: the slice-in/Vecs-out shapes forced intermediate collects
at every call site, costing the in-RAM trackers (whose defaults are plain
loops) allocations the pre-batch code never had.
- resolve_external_ids now takes `impl IntoIterator<Item = PointIdType>`
and delivers `(id, offset)` pairs through a callback, in input order:
has_id conditions stream the id set straight into the offsets set/vec,
retrieve fills its parallel vectors directly.
- external_ids_batch / internal_versions_batch take iterator inputs but
keep `Vec<Option<_>>` outputs (search result processing needs one aligned
slot per input); search drops its two offset collects, scroll passes the
itertools chunk directly.
- In-RAM trait defaults stream with no intermediate allocation; the disk
overrides buffer the input once internally, where the block-grouped
reads need the whole batch anyway.
Also document on the writable DiskIdTracker why points_deleted_batch and
internal_versions_batch are deliberately not overridden there: deleted and
versions are RAM-resident by design, so the default loop has no IO to
pipeline; only the read-only tracker keeps them on disk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove accidentally committed local example
inspect_mutable_id_tracker.rs is a local debugging harness that was swept
into the previous commit by a directory-level git add; it is not part of
the batch-interface change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add PointIdBatch and thread it through resolve_external_ids
Replace the `impl IntoIterator` / `&[PointIdType]` inputs on the
id-tracker external->internal resolve path with a small `PointIdBatch`
trait (Copy + num_ids + iter_ids), implemented for `&[PointIdType]` and
`&AHashSet<PointIdType>`. Being re-iterable and length-known lets the
disk trackers batch the reads without first collecting the input.
On the disk path this drops several intermediate allocations:
- no input `Vec<PointIdType>` collect in resolve_external_ids_batch;
- lookup_batch returns compact `(id, offset)` pairs (id rebuilt from
is_uuid + key) instead of a size-N `Option` reorder buffer;
- lookup_batch no longer groups keys by block up front: a block is one
~16 KiB DiskCache block, so same-block reads are deduplicated by the
cache (piggybacked in flight, a plain hit once resident);
- resolve_internal_batch filters deleted pairs in place and feeds
points_deleted_batch an offset iterator, dropping the offsets Vec.
Delivery is no longer input-ordered; the id travels with the offset into
the callback, so callers that need positions still have them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream disk id resolve through a callback; prefetch deleted flags whole
Follow-up to the PointIdBatch rework, removing the last buffers from the
disk external->internal resolve path and dropping the redundant deleted
batching.
- `lookup_batch` and `resolve_internal_batch` now hand each resolved
`(id, offset)` to a callback as its block completes instead of
returning a `Vec`; `resolve_external_ids_batch` just forwards the
caller's callback and logs on error. No buffer is built on the path.
- Drop `points_deleted_batch`: the deleted set is resident (writable
tracker) or prefetched whole (read-only), so a per-point `point_deleted`
check is as cheap as a pipelined read would be. `resolve_internal_batch`
captures the first check error out of band and surfaces it after the
pass.
- Prefetch the deleted flags whole (`Populate::PreferBackground`) in both
`try_preopen` and `try_open`, via a shared `deleted_open_options`, so
the per-point checks stay off remote storage.
- `PointIdBatch` loses the now-unused `num_ids`; only `iter_ids` remains.
Delivery is fully streamed, so on a mid-pass storage error the callbacks
for already-resolved live points have fired before the error surfaces —
acceptable at the best-effort `IdTrackerRead` boundary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Surface id resolve errors to the trait; drop redundant batch wrapper
`IdTrackerRead::resolve_external_ids` now returns `OperationResult<()>`
instead of swallowing storage errors. The disk path previously logged
and dropped a `resolve_internal_batch` error at this boundary; now it
propagates so callers see failed id resolution.
- The in-RAM default returns `Ok(())`; both enum forwarders and both disk
overrides propagate. The three call sites (retrieve, HasId cardinality,
HasId checker) are already in `OperationResult` functions, so they just
gain a `?`.
- Drop the `resolve_external_ids_batch` free function: with its
log-and-swallow gone it only forwarded to `resolve_internal_batch`, so
the two overrides call that directly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream internal_versions_batch through a callback
Give internal_versions_batch the same treatment as resolve_external_ids:
it takes a callback and returns OperationResult<()> instead of a
Vec<Option<..>>, so the disk override no longer allocates. It walks the
input lazily (no collect), tags each pipelined read with its internal_id,
and streams (internal_id, version) to the callback as reads complete;
out-of-range offsets are skipped and a storage error propagates instead
of being logged and swallowed.
The in-RAM default and both enum forwarders match. process_search_result
now builds a small internal_id -> version map from the callback and looks
up by scored offset (handling duplicate offsets naturally), keeping the
same missing-version-is-an-error behaviour.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stream external_ids_batch through a callback; drop chunked scroll batching
Align external_ids_batch with internal_versions_batch: iterator input,
(offset, id) callback output, no internal buffering, storage errors
propagated. The deleted filter runs inside the lazy range iterator, so
each tracker implements the override directly against its own deleted
source (resident bitvec vs prefetched on-disk file) and the redundant
DiskMappingsSource::resolve_external_batch pass-through is removed,
along with the now-unused log_lookup_err_batch.
filtered_read_by_index feeds the whole candidate iterator into one
batch pass and lets the IO layer pipeline the reads; the limit case
keeps the top-smallest ids in a bounded priority queue. This retires
ID_TRACKER_BATCH_SIZE. Search result processing pairs external ids by
internal id, mirroring the versions map.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove accidentally committed local example, again
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop unused StoredBitSlice::get_bits_batch
Its only caller, the read-only tracker's points_deleted_batch override,
was removed when the deleted flags became whole-file prefetched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Write deleted_open_options fields explicitly, comment the why
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Simplify read-only external_ids_batch error handling via try_filter
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add infallible point_deleted helper to the writable disk tracker
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop PointIdBatch in favor of impl IntoIterator<Item = PointIdType>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Andrey VasnetsovandClaude Fable 5
|
446d140c2d |
Slice filtering condition: sliced scroll / deterministic sampling (#9899)
* feat: slice filtering condition for sliced scroll and deterministic sampling
Add a `slice` filter condition selecting points where
`stable_hash(point_id) % total == index`. The hash is SipHash-2-4 with a
zero key over canonical id bytes (8 LE bytes for numeric ids, 16 RFC 4122
bytes for UUIDs) — a frozen public contract, independent of the internal
resharding ring hash, reproducible by clients to predict membership.
For a fixed `total`, slices are disjoint and cover all points, enabling
parallel scroll streams (ES sliced-scroll style) and reproducible sampling
that composes with any other filter condition.
- REST: `{"slice": {"total": N, "index": R}}`; gRPC: `SliceCondition` in
the condition oneof (tag 8)
- Evaluated per point via id_tracker external-id lookup; no payload index
needed; cardinality estimated as `points / total` with no primary clause
- `total >= 1` enforced by NonZeroU32 at parse time, `index < total` by
validation in both REST and gRPC paths
- Hash contract locked by test vectors independently reproduced with a
reference SipHash-2-4 implementation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tests: minimal OpenAPI test for slice filter condition
Scrolls all slices of a fixed total over numeric + UUID ids asserting
disjointness and full coverage, checks must_not inversion, and pins the
two rejection paths (422 for index >= total, 400 for total = 0). Requests
and responses are validated against the regenerated OpenAPI spec by the
test harness.
Note: the spec cannot itself reject total = 0 client-side — the Condition
anyOf falls through to the permissive Filter schema, as with any invalid
condition — so rejection is asserted via the server response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Luis Cossío
|
956e5dde29 |
make VectorStorageRead::read_vector_bytes mandatory (#9893)
Additionally: - homogenize enum dispatch - impl batch multivector read bytes |
||
|
xzfc
|
9d4b4b91a0 |
Optimize debug build time (less generics) (#9895)
* Non-generic TransformInto::transform I think dyn is fine here because it is executed once per query, not in a hot loop. * Query::score_by impls: manual loops Iterator chains (`.iter().map(..).sum()`) produce a lot work for the compiler to do. * QuantizedCustomQueryScorer: move generics from struct to `new` method |
||
|
Luis Cossío
|
5c269b9525 |
Misc nits (#9894)
* use `WithVector::is_enabled` * suppress unused var lint * fix non linux "useless mut" lint |
||
|
Jojii
|
c03ee39456 | Less generics in retrieve raw (#9874) |