mirror of
https://github.com/qdrant/qdrant.git
synced 2026-07-31 23:20:52 -05:00
astgrep-rules-test
266 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Arnaud Gourlay
|
dd84044a3e |
refactor: replace StructPayloadIndex::open bool flags with StorageType and IndexLoadMode (#9754)
StructPayloadIndex::open took two adjacent bools (is_appendable, create)
and call sites passed every literal combination: (true, true),
(true, false) and (false, true) all exist. A transposed pair compiles
and silently yields e.g. non-appendable + create instead of
appendable + load-only.
The target enum already existed: open immediately converted the bool
into the private StorageType { Appendable, NonAppendable }, so the bool
survived only at the API boundary, exactly where the swap hazard lives.
Make StorageType public, take it directly, and introduce
IndexLoadMode { CreateIfMissing, LoadExisting } for the create flag.
create_segment had the same trailing create: bool with bare literals at
both callers, so its parameter is lifted to IndexLoadMode as well:
load_segment passes LoadExisting, build_segment passes CreateIfMissing.
No behavior change.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Jojii
|
68e7efa6ee | Rename ambiguous build_multivec_segment test fixture (#9713) | ||
|
Andrey Vasnetsov
|
ab0d3ecc62 |
Add unified memory: cold|cached|pinned placement parameter for collection components (#9684)
* Add unified `memory: cold|cached|pinned` placement parameter for collection components
Introduce a single `memory` parameter that controls how each collection
component's data is held in RAM, replacing the inconsistent zoo of
`on_disk` / `always_ram` / `on_disk_payload` flags:
- `cold`: not pre-loaded from disk, cached with usage
- `cached`: pre-populated into page cache on load, evictable under pressure
- `pinned`: materialized on heap, never evicted by cache pressure
The parameter is available on dense vectors, HNSW config, all quantization
configs, the sparse index, all payload field index types, and payload
storage (as a new `payload: { memory }` sub-object on collection params).
When set, it overrides the deprecated legacy flag; when unset, behavior is
unchanged. Legacy flags are marked deprecated (Rust + proto) but keep
working; conflicts are resolved in favor of `memory` with a warning.
New capabilities enabled by the tri-state model:
- HNSW graph links can be pinned (first production caller of the existing
`GraphLinksResidency::Pinned`)
- sparse mmap index, quantized vectors and on-disk payload field indexes
gain a `cached` tier (mmap + populate on open)
`pinned` is rejected by API validation for components without a heap
variant (dense vector storage, payload storage). Low-memory mode degrades
placements at load time via `Memory::clamp_to_low_memory`, matching the
existing `prefer_disk`/`skip_populate` behavior. Effective-placement
comparison in the config-mismatch optimizer avoids spurious rebuilds when
the same placement is expressed through the new parameter.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix gpu-gated tests for the new `memory` field
CI clippy runs with --all-features, which compiles the gpu-gated tests
that were missed locally: add the `memory` field to config literals and
allow deprecated placement params, same as in the rest of the tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add OpenAPI tests for memory placement, keep sparse config downgrade-clean
- OpenAPI tests: create/update collections with `memory` on every component,
assert the parameters are echoed in collection info, assert legacy-only
collections expose no new fields, and assert `pinned` is rejected (422)
for dense vector storage and payload storage on both create and update.
- Persist only the explicitly requested `memory` parameter in
`sparse_index_config.json` instead of the legacy-resolved placement, so
configurations using only the deprecated `on_disk` flag keep byte-identical
files that older Qdrant versions load without unknown fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Validate collection meta ops at construction, not only in the API layer
The `memory: pinned` rejection for dense vectors and payload storage
lived in `Validate` impls on the internal request types, which only ran
through the REST actix extractor. gRPC validates just the proto message,
so a gRPC client could persist `pinned` where it is not supported and
have it silently treated as `cached`.
Run the derived validation in `CreateCollectionOperation::new` and
`UpdateCollectionOperation::new` instead: the constructors are the
common chokepoint for all API paths, before the operation is proposed
to consensus. This covers every validator on these types, not just the
`memory` checks, and keeps consensus-apply unaffected so mixed-version
clusters never reject already-committed operations.
`UpdateCollectionOperation::new` becomes fallible; `remove_replica` now
uses `new_empty` since it carries no user config. Regression tests drive
the gRPC conversion path and assert `InvalidArgument` for `pinned` on
create and update, with `cold`/`cached` accepted as a control.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Andrey Vasnetsov
|
8a5d9ac1fc |
Add disk-resident id tracker (#9657)
* Add disk-resident id tracker Introduce a disk-resident id tracker family that keeps the point-id mapping on disk instead of loading it into RAM, so resident memory no longer scales with total point count. This removes the last component of a segment that forces a full RAM load, enabling object-storage / edge followers and cutting RAM for regular deployments that opt in. New on-disk mapping format (`disk_id_tracker/on_disk_format.rs`): random-access `id_tracker.i2e` (internal->external) + `id_tracker.e2i` (external->internal as sorted num/uuid runs with a resident sparse block index). `id_tracker.versions` and `id_tracker.deleted` are reused unchanged. Trackers (sharing a lazy `DiskMappingReader` core and a `DiskMappingsSource` trait): - `DiskIdTracker` — writable, deletion-only; a new `IdTrackerEnum` variant used by regular segments, keeping deleted+versions resident and the mapping on disk. - `ReadOnlyDiskIdTracker` — read-only live-reload mirror for followers; per-point `get_bit` deletion checks on read-by-id, full deleted set materialized lazily. Selection: created when the `serverless_compatible` feature flag is set (in `segment_builder`); loaded by attempting each format in `ReadOnlyIdTrackerEnum:: detect_and_load` (no per-file `exists` round-trips) and by file-presence detection unified in `IdTrackerFormat`. `PointMappingsRefEnum` is generic over the read backend so the disk variant is a concrete `DiskMappingsRef` (no trait object). The `DiskMappingsSource` read surface returns `OperationResult`; errors are only swallowed at the infallible `IdTrackerRead` boundary. `ImmutableIdTracker` is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: apply rustfmt Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix clippy: collapse nested if in DiskIdTracker::drop Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Align on-disk id tracker headers and sections to 16 bytes Pad the i2e header to 32 bytes and the e2i header to 48 bytes, and zero-pad between e2i sections so every section starts on a 16-byte boundary. This keeps the files mmap+transmute-friendly: the u128 arrays (i2e slots, e2i uuid sparse index) need 16-byte alignment in Rust. Add on_disk_sections_are_aligned test pinning the invariant and the store/parse padding agreement via exact file-length checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Arnaud Gourlay
|
0346ea66cd |
fix: unblock optimizer after deleting a named vector (#9641)
* fix: unblock optimizer after deleting a named vector Deleting a named vector could permanently block the config-mismatch optimizer. The source-superset check in SegmentBuilder::update cancelled every rebuild that found the deleted vector still in old segment files, and each retry cancelled again, so optimizations got stuck forever. Removing the check (as in #9609) would fix delete but reintroduce data loss for the CreateVectorName race. Instead, tell the two cases apart with the live collection schema: prune a source vector that is gone from the schema (a real deletion), but cancel when it is still present (a freshly created vector this optimizer has not yet seen). This is safe because the schema is persisted before the op reaches segments, and the live schema is read after the source segments are frozen. The live set covers dense and sparse vectors, since a segment stores both together. When no live source is wired in, the conservative always-cancel behavior is kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: wire live vector names into edge optimizers Deleting a named vector left the edge path with the pre-fix behavior: segment_optimizer_config hardcoded live_vector_names to None, so a merge touching a segment that still carried the deleted vector cancelled, and EdgeShard::optimize() propagated the cancellation as a hard error forever. Share the shard config behind an Arc and hand the blocking optimizers a provider that reads the current vector names on every call. Same safety argument as the server wiring: update() holds the segments read guard across both the segment application and the config update, so any name a frozen source segment carries is visible to the live read. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: share vector-name enumeration via CollectionParams::vector_names The optimizer's live-schema set and the WAL-recovery valid-name set are the same dense+sparse enumeration and must stay in lockstep; a drift between them would reintroduce a wrong prune/cancel decision. Replace the private helper in optimizers_builder and the inline block in WAL recovery with a single CollectionParams method. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: drop SegmentOptimizer::live_vector_names forwarding hop The default trait method only forwarded to the config getter and had a single caller; ShardOptimizationStrategy now reads the config directly, removing one layer of indirection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Andrey Vasnetsov
|
852ca200b5 |
feat: feature-flagged segment manifest for LocalShard (#9530)
* feat: feature-flagged segment manifest for LocalShard
Add an on-disk segment manifest (`segments/manifest.json`) that lists a
shard's segments and their state, so out-of-process readers (e.g. a
read-only follower, possibly over object storage) can discover segments
without scanning the filesystem. Gated by the new `write_segment_manifest`
feature flag (off by default).
shard: define the structure + helpers (`SegmentsManifest`,
`SegmentManifestState`, `from_segment_holder`) in a new `segment_manifest`
module, plus the `SEGMENT_MANIFEST_FILE` constant and path helper. The
manifest is a flat `{ "<uuid>": "<state>" }` map; only `active` is written
today, with `under_construction`/`retiring` defined so the format can be
extended without breaking compatibility.
collection: LocalShard owns the writing logic. The manifest is persisted
via `SaveOnDisk<SegmentsManifest>`, initialized from the live segment set
on load/build and refreshed by the optimization worker whenever the
segment set changes (the helper re-derives from the holder and no-ops when
unchanged). No changes to `lib/shard/src/optimize.rs` internals.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Emj8TFxdtrf3K32eWhGgor
* feat: make append_only_mutations a proper feature flag
Replace the debug-only `QDRANT_APPEND_ONLY_MUTATIONS=1` env-var escape
hatch in segment construction with a `FeatureFlags::append_only_mutations`
flag, so it works in release builds and is configurable like the other
flags (config / `QDRANT__FEATURE_FLAGS__APPEND_ONLY_MUTATIONS`).
Deliberately left out of `FeatureFlags::all()`: it changes mutation
semantics and `all` is enabled in dev and e2e configs, so it stays
explicit opt-in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Emj8TFxdtrf3K32eWhGgor
* upd openapi schema
* feat: register segments in manifest via must-use token, holder builder
Wire segment-manifest maintenance through the segment lifecycle so a
newly created segment is registered as soon as it exists on disk, and
construction can't silently skip it:
- build_segment now returns a #[must_use] NewSegmentToken carrying the
new segment UUID; the lint forces callers to register or drop it.
- SegmentHolder owns the manifest and reconciles it on sync; new
segments are registered ASAP (even before being added to the holder)
via the token, before they can receive writes.
- SegmentHolderBuilder is the only way to obtain a shard's holder; its
build() wires up the manifest, so it can't be forgotten. init/set
manifest helpers are now private / test-only.
- Optimization registers the optimized segment before dropping the
superseded segments' data; deletion is intentionally lenient.
- Document the consistency assumptions on SegmentsManifest: it is a
superset-biased view that may list not-yet-finalized or already-deleted
segments, which readers must tolerate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Andrey Vasnetsov
|
7b52f04dae |
refactor: move version() from ReadSegmentEntry to StorageSegmentEntry (#9527)
`version()` is the segment's update version — only meaningful for the mutable/storage path. Every real caller already reaches it through `StorageSegmentEntry` (flush), `SegmentEntry` (update), or a concrete `Segment`/`ProxySegment`; none use it through the read-only base trait. Move the declaration down to `StorageSegmentEntry` and relocate the `Segment`/`ProxySegment` impls accordingly. `ReadOnlySegment` no longer needs it, so drop the method and the write-only `version`/`initial_version` fields it carried (`live_reload` never refreshed `version`, and neither field was ever read). This leaves `ReadSegmentEntry` a clean read-only surface and stops a read-only segment from exposing a meaningless (stale) version. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
xzfc
|
bf1aa818c6 |
Sparse InvertedIndex: bring back generic methods (#9485)
* sparse InvertedIndex: bring back generics * nits --------- Co-authored-by: Luis Cossío <luis.cossio@outlook.com> |
||
|
Ivan Pleshkov
|
d9729925cc |
[TQDT] Wire TQ multivectors storage (#9439)
* multi tq scorers * add vector storage enum * Rebase fixes * Improve scoring --------- Co-authored-by: jojii <jojii@gmx.net> |
||
|
Daniel Boros
|
4d6fb4e0ab |
feat: add open for read-only sparse vector index (#9435)
* feat: add open for read-only sparse vector index + enum sparse dispatcher * fix: universal-IO loads for read-only sparse index open * refactor: rename load_via/open_via to load_universal/open_universal * refactor: drop StorageVersion::load in favor of load_universal The regular-IO `load` duplicated `load_universal` over plain `File` IO. Remove it and route all callers through `load_universal(&MmapFs, ..)`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(sparse): decouple inverted index from concrete storage; split segment_constructor_base (#9461) Make the read-only index enum generic over storage `S` (no concrete MmapFile), and remove construction callbacks from the sparse index open paths. - InvertedIndex is now a pure read/search trait: `open`/`from_ram_index`/ `type Fs` moved off the trait to inherent methods on each concrete index type, so construction no longer requires `S::Fs: Default`. - SparseVectorIndex open split into a generic `plan` (load-vs-build decision + RAM-index build) and generic `finish` (assembly); callers do the concrete per-type construction, so no construction callbacks are needed. - ReadOnlySparseVectorIndex::open takes the already-constructed inverted index and caller-loaded config instead of a `load_inverted_index` callback. - VectorIndexReadEnum is generic over `S: UniversalRead`; sparse mmap variants hold `InvertedIndexCompressedMmap<_, S>` rather than a concrete `MmapFile`. - Split the 1182-line segment_constructor_base.rs into a module (paths, vector_storage, payload_storage, id_tracker, vector_index, sparse_vector_index, create_segment, segment, legacy_state); the sparse dispatcher's match arms collapse into three per-family helpers. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat: LiveReload (no-op) dispatch for read-only vector index enum (#9436) --------- Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Roman Titov
|
2fd1b5371a | Add async_payload_storage feature flag (#9409) | ||
|
Jojii
|
ff4d24cbd0 |
[TQDT] Integration into VectorStorageEnum (#9399)
* Integrate TurboVectorStorage into VectorStorageEnum # Conflicts: # lib/segment/src/segment_constructor/batched_reader.rs # lib/segment/src/vector_storage/vector_storage_base.rs * Review remarks * Allow Turbo4 datatype as inlined HNSW vector |
||
|
generall
|
a3fcb597f6 |
fix: post-rebase trait integration for batched_reader and TQDT
Import the *Read traits in batched_reader (get_dense/get_multi/ get_sparse_opt moved onto them) alongside the write traits it still needs for update_from, and drop DenseTQVectorStorage's duplicate size_of_available_vectors_in_bytes default now that VectorStorageRead mandates it (it was ambiguous for TurboVectorStorage). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ivan Pleshkov
|
6a20fc1df5 |
[TQDT] Vector storage update from without CowVector (#9325)
* [TQDT] Vector storage update_from without CowVector Move update_from to the kind-specific DenseVectorStorage, SparseVectorStorage and MultiVectorStorage traits in their native element type, and merge through batched_reader::merge_from. This drops the CowVector-based VectorStorageEnum::update_from shim: sources are copied in their native representation, with no f32 round-trip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * review remarks * remove obsolete comments --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Roman Titov
|
1e5013bcae | Add io_uring based payload storage type (#9310) | ||
|
Andrey Vasnetsov
|
ba3472ea49 |
feat(id_tracker): deferred-aware mutable id tracker (#9249)
* feat(debug): QDRANT_APPEND_ONLY_MUTATIONS env override Debug-only escape hatch so newly built segments default to append-only mutation routing when QDRANT_APPEND_ONLY_MUTATIONS=1 (or true/yes) is set in the environment. Lets us run the existing test suites against the append-only path without wiring a collection-level config knob first. Release builds compile this out — the function is a const false. Logs a single warn-level message the first time the override fires. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(append-only): skip deleted vectors on snapshot, always write payload Two correctness fixes to clone_and_mutate_point uncovered by running the openapi suite with QDRANT_APPEND_ONLY_MUTATIONS=1: 1. The snapshot loop read every named vector via get_vector_opt, which returns slot bytes even when the per-vector deletion bit is set. For sparse-only points (or any point that had update_vector(_, None) applied) this materialised the default-zero vector as if it were real data, wrote it at new_id, and never re-tombstoned the slot — so dense search started scoring phantom vectors. Now we check is_deleted_vector(old_id) and skip the read, letting the writer loop emit update_vector(new_id, None) and re-mark the slot deleted. 2. The payload write was skipped when the snapshot ended up empty. That dropped two side effects the field indexes rely on: payload_storage.overwrite(new_id, empty), and the remove_point fan-out across configured field indexes that bumps each index's total_point_count to cover new_id. Without the bump the null index doesn't see new_id, so is_empty / is_null filters lose the point even though its mapping is live in the id tracker. The skip was an optimisation, not a contract; remove it so the field indexes get the same registration they'd get from the standard clear_payload path. Also collapses the debug env override to an inline cfg!()-gated check at the struct literal — the helper with one-time logging and multi-value matching was disproportionate for a debug-only escape hatch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(id_tracker): split active vs deferred maps in PointMappings (PR A) First step of moving the mutable id tracker to a "two-track" model so append-only mutations into a deferred segment can keep both the visible (active) version and the latest mutated (deferred) version of a point at the same time. This PR is purely structural. On-disk format is unchanged: the loader still produces a single combined map, and `PointMappings::new` partitions it at construction time based on `deferred_internal_id`. Every observable behaviour at the existing API surface is preserved. Concretely: - New fields: `external_to_internal_num_deferred`, `external_to_internal_uuid_deferred`, and a `shadowed: BitVec` for future use by PR B (lazy-grown, default-false). - `internal_id(ext)` checks active first, falls through to deferred — matches the pre-split "any matching id" contract for ext ids whose internal id sat above the cutoff. - `set_link(ext, new_id)` now routes by cutoff: writes below the cutoff land in active, writes at or above land in deferred. Any prior head in the other track is tombstoned, so each ext still owns exactly one slot — same observable result as the pre-split single-map insert. PR B replaces the cross-track tombstone with a shadow-bit flip; PR A keeps current semantics on purpose. - `drop(ext)` clears entries from both tracks and tombstones each one, again matching the prior single-map behaviour. - `iter_external` and `iter_from` merge the active and deferred BTreeMap views into one sorted-by-key stream (dedup'd in case an ext exists in both tracks). - `available_point_count` counts distinct external ids across both tracks — preserves the prior observable count for segments where some entries used to sit above the cutoff in the single map. No write-path or read-path behaviour change. Reads still filter `internal_id >= cutoff` exactly as before; mutations still tombstone prior heads. The shadow bit and the deferred-aware lookup wiring land in PR B and PR C. All existing id_tracker tests pass against the split layout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(id_tracker): shadow active head on deferred writes (PR B) Step 2 of the deferred-aware mutable id tracker. Replaces PR A's "tombstone the other track" cross-track cleanup with shadow-bit logic so a deferred mutation no longer hides the visible active version. Behaviour by case: - Active write (`internal_id < cutoff`, or no cutoff): unchanged semantically. Any prior deferred head for the same ext is dropped and tombstoned (the new active is the single visible head). Same observable result as PR A. - Deferred write (`internal_id >= cutoff`): the prior deferred head (if any) is dropped and tombstoned. The active head, if it exists, is **shadowed** — its bit is set in `shadowed: BitVec` but its slot stays alive in the active map. Read paths in `Exclude` mode continue to return that active version; PR C will teach `IncludeAll` paths (the optimiser) to skip shadowed actives and prefer the deferred head. Also adds `is_shadowed(internal_id)` and `shadowed_bitslice()` accessors (the latter for PR C's filter pipeline). New unit tests cover the routing matrix: - no cutoff — active replacement, no shadow, - below-cutoff replacement — active path, no shadow, - deferred-on-top-of-active — shadow set, active retained, - two deferred writes — prior deferred tombstoned, shadow persists, - fresh insert above cutoff — no shadow, - `drop(ext)` clears both tracks plus the shadow bit. WAL replay reuses the same `set_link`, so deferred routing on replay falls out of this change with no extra wiring. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(id_tracker): IncludeAll skips shadowed actives, deferred-aware lookups (PR C) Step 3 of the deferred-aware id tracker stack. PR B introduced the shadow bit on `set_link`; this PR teaches the read paths to honour it so the optimiser (and any other `DeferredBehavior::IncludeAll` consumer) sees each external id exactly once, with the deferred head winning over the now-shadowed active. Concretely: - `PointMappings::internal_id_with_behavior(ext, behavior)`: * `Exclude` returns the active head only — `None` for a deferred-only ext, so query paths never see a deferred mutation; * `IncludeAll` prefers the deferred head and falls back to active for points that never crossed the cutoff. Yields at most one internal id per ext. - `IdTrackerRead::internal_id_with_behavior` mirrors the new method with a default impl that delegates to `internal_id` for trackers that don't carry deferred mutations. - `PointMappingsRefEnum::iter_internal_with_behavior(IncludeAll)` now filters shadowed actives via the new `PointMappings::shadowed_bitslice()` accessor. - `PointMappingsRefEnum::filter_deferred_and_deleted(IncludeAll)` also filters shadowed actives — same single-yield-per-external guarantee for external iterator sources like field-index outputs. - `IdTrackerRead::resolve_external_ids` switches to the deferred-aware lookup. No more post-lookup `id >= cutoff` filter — the behaviour enum lookup gets it right at the source. New unit tests cover the two new entry points: - IncludeAll prefers the deferred head when an active is shadowed; - Exclude returns None for deferred-only ext ids; - `filter_deferred_and_deleted` over a mixed candidate list yields the expected per-mode result (Exclude: actives below cutoff; IncludeAll: every visible head, no shadowed actives). No queries observable behaviour change today — production Exclude paths still resolve via `internal_id` and the active head. The optimiser will start using `IncludeAll` (and reap the dedup) in follow-up work. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(id_tracker): expose shadowed_point_count in SegmentInfo (PR D) Final step of the deferred-aware id tracker stack. Adds a single new counter — number of active heads currently shadowed by a deferred mutation — and plumbs it through the id-tracker trait, the segment read-view helper, and `SegmentInfo` for telemetry. Preserves existing semantics: - `available_point_count` is unchanged. Each distinct external id still counts once, regardless of which track holds its head. - `deferred_point_count`, `deferred_internal_id`, `num_deleted_deferred_points` keep their current values. - Non-appendable trackers default `shadowed_point_count()` to `0`, so the new `SegmentInfo.num_shadowed_points` is `None` for them. Concrete changes: - `PointMappings::shadowed_count()` — popcount of the shadowed bitslice. - `IdTrackerRead::shadowed_point_count()` trait method with a `0` default; wired through `MutableIdTracker`, `InMemoryIdTracker`, the mutable read-only tracker, and both enum dispatchers. - `SegmentReadView::shadowed_point_count()` helper. - New `SegmentInfo.num_shadowed_points: Option<usize>`, populated with `Some(_)` for appendable segments and proxied through `ProxySegment` from the wrapped segment's value. New unit test covers the counter lifecycle: - active-only writes don't grow it, - a deferred write over an active adds one shadow, - a second deferred write supersedes the prior deferred head but the shadow stays put (still one active being shadowed), - `drop(ext)` clears the shadow bit, - a fresh deferred insert with no active prior doesn't add a shadow. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(id_tracker): rename DeferredBehavior, push behavior down to PointMappings Three connected pieces of polish on top of the deferred-aware id tracker stack: 1. Rename `DeferredBehavior::Exclude` → `VisibleOnly` and `IncludeAll` → `WithDeferred`. The pre-PR-C semantics of "include/exclude the deferred cutoff" became misleading once `IncludeAll` started skipping shadowed actives — it doesn't "include all" anymore, it yields one slot per external (deferred head preferred, active fallback). New names describe what each variant returns instead of how it relates to the cutoff. The helper method `include_all_points` becomes `with_deferred_points`. Updates ~90 call sites across the workspace; behaviour is unchanged. 2. Push `iter_internal_with_behavior` down from `PointMappingsRefEnum` into `PointMappings`. The per-mode logic (cutoff `take_while`, shadowed `filter`) now lives next to the data it consults; the enum layer becomes a two-arm `Either` dispatcher. `CompressedPointMappings` short-circuits to `iter_internal()` since compressed mappings can't carry deferred mutations. 3. Add a short docstring on `internal_to_external` describing the two-track model: no active-vs-deferred bias, shadowed pairs occupy two slots with the same value, and reads must gate on `deleted` because `set_link`'s same-track replacement leaves a real-looking stale ext id in place. Returning `impl Iterator` instead of `Box<dyn Iterator>` for `iter_internal`, `iter_internal_excluding`, `iter_internal_visible`, and `iter_internal_with_behavior` removes the double-Box at the enum boundary. The original branching structure is preserved with `itertools::Either` instead of restructured into one big filter chain. All existing `id_tracker` tests still pass (47 total). `cargo check --all-targets` + `cargo clippy --all-targets` clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(id_tracker): drop shadowed API surface, unbox iter_from/iter_random Cleanup pass on top of the deferred-aware id tracker stack. API surface: - Drop SegmentInfo.num_shadowed_points + its trait method, read-view helper, tracker impls and ProxySegment passthrough. Cross-segment shadow already exists in the appendable update flow (the source segment keeps its copy when the new write lands above the cutoff, see SegmentHolder::apply_points_with_conditional_move) and is handled at the aggregation layer, not reported per-tracker. - Keep PointMappings::shadowed_count() as a private popcount helper guarded by expect(dead_code) so a future caller can opt back into the dedup'd count without re-plumbing the trait. available_point_count: - Replace the cross-track dedup (active + deferred-only via contains_key filter) with a straight 4-term sum. A shadowed ext contributes two slots, matching the two non-tombstoned internal ids it actually occupies. Old dedup broke the invariant that deleted_point_count == deleted_bitslice.count_ones(): for each shadow it overcounted deletions by one without any tombstone actually being set. DeferredBehavior pushdown: - iter_random_with_behavior: caps the sampling range at the deferred threshold in VisibleOnly mode (no wasted samples above cutoff), filters shadowed actives via the bit in WithDeferred. - iter_from_with_behavior: VisibleOnly walks the active maps only (no merge with deferred); WithDeferred delegates to iter_from's existing merge. - scroll.rs read_by_id_stream / filtered_read_by_id_stream collapse their manual if-deferred-behavior branching into a single iter_from_with_behavior call. - Old iter_random (no behavior) at PointMappings + ref enum was unused after the migration, deleted. Unboxing iter_from / iter_random / iter_from_with_behavior: - PointMappings::iter_from returns impl Iterator + '_ via Either inside the merged_num/merged_uuid closures (BTreeMap::iter vs range), Either at the outer match (num+uuid chain vs uuid-only). - PointMappings::iter_from_with_behavior unboxed with a triple Either (behavior, external-id arm, closure start). - CompressedPointMappings::iter_from unboxed (Either over None/Some). - PointMappingsRefEnum::{iter_from, iter_from_with_behavior, iter_from_visible} all return impl Iterator + 'a via Either on the Plain/Compressed dispatch. Other: - Update outdated PR-A/PR-B comment on PointMappings::drop. - Make the max_internal match in iter_random_with_behavior exhaustive (VisibleOnly+None | WithDeferred+_ instead of `_`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(id_tracker): unbox iter_external Return impl Iterator from PointMappings::iter_external, CompressedPointMappings::iter_external and the PointMappingsRefEnum wrapper, matching the style of the other iter_* helpers. The wrapper dispatches via Either. The remaining Box::new at Segment::iter_points stays because self_cell's BoxedPointIdIterator alias needs a sized type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(id_tracker): count set_link tombstones in deferred_deleted_count PointMappings::set_link tombstones the prior slot at two sites — when an active write supersedes a deferred head (cross-track) and when any write replaces a same-track head — using a bare deleted.set(old, true). Neither path updated deferred_deleted_count, so tombstones above the cutoff weren't reflected in the counter. The append-only flow exposes this constantly: every set_full_payload after upsert_point routes through clone_and_mutate_point, which re-issues set_link with a fresh internal id and tombstones the prior one. Most tombstones land above the cutoff, so deferred_point_count (total - cutoff - deferred_deleted_count) over-reports by the missing count. The openapi test_deferred_points integration test caught this as `num_points - num_deferred_points = -1800` across two segments. Extracted the tombstone bookkeeping into PointMappings::tombstone_slot and routed both set_link sites + drop's loop through it. Behaviour on drop is unchanged; set_link now bumps the counter on the live → tombstoned transition for slots at or above the cutoff. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(id_tracker): collapse set_link tombstone if-let chains Clippy's collapsible_if on the two if-let blocks added by the deferred_deleted_count fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update lib/segment/src/id_tracker/point_mappings.rs Co-authored-by: Tim Visée <tim+github@visee.me> * refactor(id_tracker): size shadowed BitVec once instead of growing lazily Collect the shadowed active ids up front and allocate the BitVec to the highest offset in a single resize, avoiding repeated reallocations while marking shadows. Addresses review feedback. Co-authored-by: Cursor <cursoragent@cursor.com> * revert: restore lazily-grown shadowed BitVec Roll back the up-front sizing of the shadowed BitVec; last_entry doesn't fit here and swapping one allocation for another isn't worthwhile. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(id_tracker): prefer deferred head in iter_from merge When an external id has both an active and a deferred head, iter_from's merge collapsed the pair to the active (stale) offset. That contradicts the WithDeferred contract used everywhere else: internal_id_with_behavior and iter_random_with_behavior both surface the deferred head (the latest mutation) over the shadowed active. Consumers that use the returned internal id (payload-filter checks, the optimizer's version merge, HNSW old->new mapping) therefore saw the stale copy. Flip the Both arm to take the deferred operand and document the rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(id_tracker): preserve active+deferred heads when loading mappings read_mappings replayed the persisted change log into a single flat external -> internal map per id-type, then split it by the deferred cutoff in PointMappings::new. A flat map holds one head per external id, so it collapsed the active+deferred coexistence case (an external id linked first to an active slot, then to a deferred one via sequential set_link) down to the last write — silently dropping the other head, orphaning its slot, and leaving the shadowed bit unset. The split in new() could not recover what was already lost before it. Replay the log through the canonical set_link/drop mutators on a PointMappings seeded with the cutoff instead. The log is the sequence of set_link/drop calls that produced the live in-memory state, so this reconstructs that state exactly — both heads, the shadowed bit, and deferred_deleted_count — with no logic duplication. Drops the debug_assert-guarded corruption-recovery branch (subsumed by set_link's re-link handling) and the now-unused Uuid/PointIdType imports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(id_tracker): make deferred behavior explicit at point resolution Replace the ambiguous "any matching head" point resolution with an explicit DeferredBehavior at every resolution boundary, so callers no longer rely on a hidden active-vs-deferred policy. - Remove PointMappings::internal_id and the bare IdTrackerRead::internal_id (the active-first-else-deferred hybrid). internal_id_with_behavior is now the single required resolution method; immutable/compressed trackers implement it by ignoring the behavior (they never carry deferred heads). - Migrate every caller to an explicit behavior, audit-driven: - writes (upsert/delete/payload/vectors), point_version, point_is_deferred, get_internal_id, drop, consistency + builder dedup -> WithDeferred (the latest/live head); - single-point payload/vector retrieval and formula rescore -> VisibleOnly; - HasId/CustomIdChecker/cardinality resolution -> the request's behavior, threaded through the filter chain from iter_filtered_points (other entry points default to VisibleOnly). - lookup_internal_id takes an explicit DeferredBehavior instead of assuming VisibleOnly internally. - has_point takes an explicit DeferredBehavior (drop the has_point_with_behavior wrapper). Thread it through read_points/_read_points/read_points_locked so retrieve_blocking passes its request behavior to the existence filter; all other existence/dedup callers pass WithDeferred (unchanged behavior). - set_link now detaches a stale live occupant of a reused internal id, keeping the forward and reverse maps consistent when recovering a corrupted log. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(id_tracker): failing tests for two correctness findings in #9249 (#9311) * test(id_tracker): failing tests for two review findings Two intentionally-failing tests pinning correctness gaps in the deferred-aware id tracker (#9249). Both fail on the final assertion; the earlier assertions establish the expected/consistent behavior. 1. iter_from_with_behavior(WithDeferred) resolves an active/deferred shadow collision to the stale ACTIVE internal id, while its siblings internal_id_with_behavior and iter_internal_with_behavior correctly surface the DEFERRED (latest) head. Consumers that use the yielded internal id (optimizer merge via for_each_unique_point, filtered_read_by_id_stream) therefore observe the pre-mutation version. left: [(NumId(7), 2)] right: [(NumId(7), 9)] 2. The PR-B shadow/visible invariant is not durable: the on-disk single combined map cannot represent a shadowed ext, so a plain mappings flush + reload collapses the shadow to deferred-only and the visible (active) head is lost (VisibleOnly resolves None where it resolved Some(2) live). Restoration then depends entirely on WAL replay, i.e. on flush-vs-WAL-truncate ordering. left: None right: Some(2) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(id_tracker): strengthen shadow tests + merge-primitive proof Follow-up to the two failing tests, addressing self-review: - Add for_each_unique_point_keeps_deferred_head_for_shadowed_point: the optimizer merge primitive (used by segment_builder::update_from) yields the stale active copy (internal 2, version 5) for a shadowed point and drops the deferred latest (internal 9, version 8). This directly exercises the data-loss consequence of finding #1 at the merge layer. left: [(NumId(7), 2, 5)] right: [(NumId(7), 9, 8)] - Tighten shadow_visible_head_survives_mapping_flush_reload: pin the exact reload failure mode. After flush+reload the mapping collapses to deferred-only (internal_id == Some(9)) and the active slot survives as a live orphan in the inverse map (external_id(2) == Some(7), not deleted) — a torn state where a VisibleOnly scroll still surfaces the stale copy while by-id VisibleOnly resolution breaks. Reframe as the live-vs-reload divergence the PR introduces (Some(2) live -> None reload; dev is consistently None). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix test --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: generall <andrey@vasnetsov.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Tim Visée <tim+github@visee.me> Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com> |
||
|
Andrey Vasnetsov
|
84712e3a93 |
feat: append-only mutation mode for mutable segments (#9245)
* feat: append-only mutation mode for mutable segments Adds an opt-in per-segment switch so every mutating op tombstones the old internal_id and writes the result at a fresh one, instead of overwriting the underlying vector storage / payload row / field index in place. Intended for S3-backed storages that prefer pure appends — the on-disk structures are never rewritten. Runtime flag, not persisted: `Segment::append_only_mutations` lives next to `appendable_flag` on the struct, defaults to false in the builder, and `Segment::is_append_only()` only returns true when the segment is also appendable (clone-and-tombstone needs growable storages). The core helper is `Segment::clone_and_mutate_point`: snapshots the point at `old_id` into owned `NamedVectors` + `Payload`, hands them to a closure for op-specific in-memory modification, allocates `new_id`, writes all configured vector storages and the payload at `new_id`, and repoints the id tracker via `set_link` (which auto-tombstones `old_id` in the deleted bitslice). Stale slots and field-index postings keyed by `old_id` are filtered by readers via `filter_deferred_and_deleted` and reclaimed at optimization. The routing policy concentrates in one new dispatcher, `Segment::handle_point_mutate`, which takes two closures (in-place and snapshot-mutate) and picks the path based on `is_append_only()`. All six existing entry points in `SegmentEntry` (`upsert_point`'s existing-pid branch, `update_vectors`, `delete_vector`, `set_full_payload`, `set_payload`, `delete_payload`, `clear_payload`) collapse to this dispatcher; `upsert_point`'s insert path stays on the non-mutating fast path. `delete_point` gets a tombstone-only variant (`delete_point_tombstone_only`) that only flips the id-tracker deleted bit, leaving the payload row and field-index postings at `internal_id` in place. The append-only path skips the in-place `clear_payload` call entirely, matching the "id tracker is the only thing we write" intent. `NamedVectors::into_owned()` is added so the snapshot closure can take ownership of a borrowed input wholesale. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(debug): QDRANT_APPEND_ONLY_MUTATIONS env override Debug-only escape hatch so newly built segments default to append-only mutation routing when QDRANT_APPEND_ONLY_MUTATIONS=1 (or true/yes) is set in the environment. Lets us run the existing test suites against the append-only path without wiring a collection-level config knob first. Release builds compile this out — the function is a const false. Logs a single warn-level message the first time the override fires. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(append-only): skip deleted vectors on snapshot, always write payload Two correctness fixes to clone_and_mutate_point uncovered by running the openapi suite with QDRANT_APPEND_ONLY_MUTATIONS=1: 1. The snapshot loop read every named vector via get_vector_opt, which returns slot bytes even when the per-vector deletion bit is set. For sparse-only points (or any point that had update_vector(_, None) applied) this materialised the default-zero vector as if it were real data, wrote it at new_id, and never re-tombstoned the slot — so dense search started scoring phantom vectors. Now we check is_deleted_vector(old_id) and skip the read, letting the writer loop emit update_vector(new_id, None) and re-mark the slot deleted. 2. The payload write was skipped when the snapshot ended up empty. That dropped two side effects the field indexes rely on: payload_storage.overwrite(new_id, empty), and the remove_point fan-out across configured field indexes that bumps each index's total_point_count to cover new_id. Without the bump the null index doesn't see new_id, so is_empty / is_null filters lose the point even though its mapping is live in the id tracker. The skip was an optimisation, not a contract; remove it so the field indexes get the same registration they'd get from the standard clear_payload path. Also collapses the debug env override to an inline cfg!()-gated check at the struct literal — the helper with one-time logging and multi-value matching was disproportionate for a debug-only escape hatch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * make test_move_points_to_copy_on_write work with copy-on-write --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Jojii
|
7a8703f166 |
[TQDT] API (#9172)
* [ai] TQDT in the API * [ai] unify new sparse error for TQDT * Rename to `turbo4` * Add `Turbo4` to comments and doc strings. * Also validate named sparse vector creation |
||
|
Andrey Vasnetsov
|
daa22f15e6 |
[UIO] Split UniversalReadFileOps into filesystem + file traits (#9151)
* [UIO] Split UniversalReadFileOps into filesystem + file traits
`UniversalReadFileOps` is now an instance-based trait describing a
filesystem handle (list/exists with `&self`, plus `from_context`). A new
`UniversalReadFs: UniversalReadFileOps` subtrait adds the
`open(&self, path, options) -> Self::File` capability with `type File:
UniversalRead`. `UniversalRead` no longer extends `UniversalReadFileOps`
and is purely a file-handle trait.
This separates "filesystem instance" from "file handle". Backends that
need per-instance configuration (S3 bucket name + credentials, mmap
default advice, io_uring runtime, block-cache controller `Arc`) gain a
typed home in `Self::ContextConfig`, and `list_files`/`exists`/`open`
become `&self` methods on the filesystem handle.
Concrete filesystem handles introduced for the three existing backends:
- `MmapFs` — unit struct; `ContextConfig = ()`; produces `MmapFile`.
- `IoUringFs` — carries `prevent_caching`; `ContextConfig =
IoUringConfigContext`; produces `IoUringFile`. `IoUringConfigContext`
becomes the construction input rather than a per-open argument.
- `BlockCacheFs` — carries `Arc<CacheController>`; `ContextConfig =
BlockCacheConfigContext`; produces `CachedSlice`.
`TConfigContext` (universal builder methods) is kept so generic-over-`Fs`
code can still set cross-backend knobs via
`Fs::ContextConfig::default().with_prevent_caching(true)`.
Wrappers (`ReadOnly<S>`, `TypedStorage<S, T>`, `StoredStruct<S, T>`),
higher-level storages (`StoredBitSlice<S>`, `UniversalHashMap<K, V, S>`)
keep `<S: UniversalRead>` parameterization but their `open(...)`
constructors now grow a `fs: &Fs` argument bound by
`Fs: UniversalReadFs<File = S>`. `read_json_via` becomes
`read_json_via(fs: &Fs, path)`.
All test code and benches in `common` updated to construct
`MmapFs`/`IoUringFs` inline as needed. `common` compiles cleanly with
tests and benches. `gridstore`, `segment`, and `tonic` caller updates
are in flight in subsequent commits.
* WIP: gridstore + segment caller sweep (partial)
Threads `fs: &Fs` through gridstore's `BitmaskGaps`, `Bitmask`, `Pages`,
and `Gridstore::new`/`open`/`create_new_page`. Most segment callers
have `OpenOptions { extra: ... }` removed and `S::open(path, opts, ctx)`
sites updated mechanically but the trait change is not yet propagated.
Does NOT compile yet. Tracker still has static `S::open` calls, segment
generic constructors (`MmapInvertedIndex<S>`, `UniversalMapIndex`,
`StoredGeoMapIndex`, etc.) still call `S::open`/`S::list_files`/`S::exists`
statically — they need an `fs: &Fs` parameter added. Tonic API
`StorageReadService<S>` also unconverted.
Committed as branch checkpoint; cascade continues in subsequent work.
* gridstore: thread `fs: &Fs` through Bitmask, BitmaskGaps, Pages, Tracker
Per the new `UniversalReadFs` shape, every constructor/method that opens
files takes an `fs: &Fs` parameter. Gridstore is currently mmap-only,
so the top-level `Gridstore` / `GridstoreReader::open` callers in the
crate pass `&MmapFs` inline. Tests do the same.
gridstore lib + tests now compile cleanly. Segment + tonic cascade
still pending.
* WIP: segment caller sweep — dynamic_stored_flags first
* WIP: segment cascade - id_tracker partial
* common benches: update to new UniversalReadFs::open shape (clippy clean)
* segment flags: thread Fs through BufferedDynamicFlags / Bitvec / Roaring
DynamicStoredFlags::set_len now takes `fs: &Fs`. BufferedDynamicFlags
stores an `Arc<Fs>` so the flusher closure can call `set_len` on resize.
BitvecFlags and RoaringFlags expose a new `Fs` type parameter and the
flag tests now pass `Fs::default()` (MmapFs/IoUringFs via duplicate_item).
Concrete consumers (bool/null index, mmap dense/multi/sparse storages)
pin `Fs = MmapFs` and pass `&MmapFs` to inner opens.
* segment: thread Fs through field-index lifecycle methods
Apply the new UniversalReadFs::open shape across:
- full_text_index (MmapInvertedIndex, MmapFullTextIndex, UniversalPostings)
- geo_index (StoredGeoMapIndex build/open + tests + builders)
- numeric_index lifecycle (UniversalNumericIndex build/open)
- map_index lifecycle (UniversalMapIndex build/open)
- stored_point_to_values (open / from_iter)
Concrete consumers pin Fs = MmapFs and pass &MmapFs inline; generic
open paths thread `fs: &Fs` where Fs: UniversalReadFs<File = S>.
* segment: thread Fs through chunked vectors and id-tracker callers
- ChunkedVectors gains `Fs` generic so add_chunk can call create_chunk
after open. ChunkedVectorsRead/load_config and chunks::{read_chunks,
create_chunk} take `fs: &Fs`. Concrete callers (dense / multi-dense /
sparse / quantized) pass MmapFs inline.
- DenseVectorStorageImpl stores `fs: Fs` so `update_from` can reopen
ImmutableDenseVectors. ImmutableDenseVectors::open takes `fs: &Fs`.
- VectorStorageEnum DenseUring* variants thread IoUringFs alongside
IoUringFile.
- segment_builder + segment_constructor_base pass MmapFs to
ImmutableIdTracker::{new, open}.
- QuantizedStorage::from_file takes `fs: &Fs`; quantized_vectors callers
pass &MmapFs.
* segment: apply nightly rustfmt after Fs refactor
cargo +nightly fmt --all over the segment crate after the
UniversalReadFs cascade. No semantic changes.
* segment: thread Fs through benches and id-tracker tests
Update the dynamic-mmap-flags and buffered-update-bitslice benches to
the new UniversalReadFs::open shape (pass `&MmapFs`). Update the
immutable-id-tracker test suite to forward `&MmapFs` to
`from_in_memory_tracker` / `open`.
* uio: pin Fs via UniversalRead::Fs assoc type; per-call OpenExtra
Two design changes that fall out of the per-instance Fs refactor:
1. Bidirectional Fs ↔ File pinning. `UniversalRead::Fs:
UniversalReadFs<File = Self>` lets generic-over-`S` code refer to
`S::Fs` directly instead of carrying an extra `<Fs: UniversalReadFs<File = S>>`
generic param. `ReadOnly<S>` wraps a file but has no natural
filesystem; a phantom `ReadOnlyFs<S::Fs>` satisfies the constraint
while inherent `ReadOnly::open` keeps taking `&S::Fs` directly.
2. `prevent_caching` moves from filesystem-instance state to per-call
`UniversalReadFs::OpenExtra: Default`. Was previously a knob on
`IoUringConfigContext` / `IoUringFs`, conflating "how this fs is
built" with "how this file is opened." Now `IoUringFs::OpenExtra =
IoUringOpenExtra { prevent_caching }`; mmap and block-cache use `()`.
`IoUringConfigContext` is gone, `TConfigContext` slims to a `Default`
marker.
Tonic StorageReadService holds `Arc<S::Fs>` (was `PhantomData<S>`); its
`new()` builds via `S::Fs::from_context(default)` and the spawn_blocking
closures clone the Arc to call instance methods.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: cfg-gate IoUringOpenExtra import for non-linux builds
The IoUringOpenExtra reexport from `universal_io` is gated on
`target_os = "linux"`. The previous commit left an unconditional import
in `persisted_hashmap/tests.rs`, breaking macOS/Windows CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: migrate simple_disk_cache to per-instance Fs API
PR #9097 (merged into dev concurrently with this branch) introduced a
`DiskCache<R: UniversalRead>` using the pre-refactor trait shape:
file-handle-as-filesystem (`R::open`, `R::list_files`), an
`OpenOptionsExtra` field on `OpenOptions`, and trait methods without
`&self`. The Fs-instance refactor on this branch removed all three.
Reshape `simple_disk_cache` to match the new design without breaking the
lazy mirror semantics:
- New `DiskCacheFs<R>` is the filesystem handle. Holds a clone of the
remote `R::Fs`; `list_files`/`exists` delegate; `from_context`
forwards to the inner Fs context. `open` constructs a `DiskCache<R>`
via the global `DiskCacheConfig`.
- `DiskCache<R>` now stores `remote_fs: R::Fs` + `remote_extra:
<R::Fs as UniversalReadFs>::OpenExtra`, so lazy remote opens go
through `self.remote_fs.open(path, options, extra)` instead of the
removed `R::open`. `open_with_config` takes the remote Fs + extra
explicitly (no more hard-coded `prevent_caching: true`; callers pass
the appropriate `OpenExtra`).
- `UniversalRead for DiskCache<R>` now declares `type Fs =
DiskCacheFs<R>` (no more `open` method on the file trait).
- Propagate the necessary bounds (`R::Fs: Clone`,
`<R::Fs as UniversalReadFs>::OpenExtra: Clone`,
`R::OwnedReadPipeline<u8, Range<u32>>: Send`) through `pipeline.rs`
free functions and impl blocks that reach into `DiskCache::remote` /
`local_state`.
- Drop the now-removed `extra: _` destructure in `LocalState::new`.
- Tests construct the remote Fs via `R::Fs::from_context(Default::default())`
and exercise `DiskCache::open_with_config`. 17 simple_disk_cache
tests pass; the 3 `empty_read_does_not_materialize_local_file`
failures pre-exist on dev (verified) and are unrelated.
`mold -run cargo clippy --all-targets` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt on simple_disk_cache migration
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt for local_state imports after rebase
Co-authored-by: Cursor <cursoragent@cursor.com>
* uio: split DiskCacheFs into its own module
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: replace TConfigContext with OpenExtra trait; move DiskCacheConfig onto DiskCacheFs
- Drop the empty TConfigContext marker; ContextConfig is now unconstrained
so backends can require explicit construction.
- Add OpenExtra trait with with_prevent_caching for backend-agnostic
per-call knobs; impl for () (no-op) and IoUringOpenExtra.
- DiskCacheFs now carries Arc<DiskCacheConfig> via the new
DiskCacheFsContext<C>; the prefill flow moves from the deleted
open_with_config into DiskCacheFs::open so Populate::Blocking /
PreferBackground work through the trait API.
- Remove the DiskCacheConfig global; callers must construct the context
explicitly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt after OpenExtra refactor
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: fix spelling — Implementors → Implementers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: possible panic insetad of error propagation
* fix: missing cfg annotation
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Daniel Boros <dancixx@gmail.com>
|
||
|
Andrey Vasnetsov
|
2f815b751a |
Clear id tracker page cache after building a segment (#9137)
The segment builder evicts the page cache of vector storage, quantized vectors, payload, payload index and the vector index after a build to avoid cache pollution, but the id tracker was never cleared. Its on-disk files (mappings, versions, deleted bitslice) are written during the build and stay resident in the page cache, so after each optimization the id tracker files linger as cache even though they are meant to be on-disk only (expected_cache_bytes == 0). Add `IdTracker::clear_cache` (default no-op) plus a `clear_cache_if_on_disk` policy wrapper, and implement the eviction for the mutable and immutable trackers: - ImmutableIdTracker pages out its two mmap-backed storages via madvise and drops the RAM-loaded mappings file via fadvise(DONTNEED). - MutableIdTracker drops its append-only log files via fadvise(DONTNEED). The builder calls `clear_cache_if_on_disk`, mirroring the payload index. The id tracker has no on-disk mode yet, so this always clears for now; a TODO marks where to gate it once an on-disk mode exists. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Arnaud Gourlay
|
386e222b42 |
fix: reject source-superset schema mismatch at SegmentBuilder (#9110)
* fix: serialize optimization proxy install against shard updates execute_optimization captures `target_config` from the optimizer's frozen config and then wraps source segments in proxies. Between those two points, `CollectionUpdater::update` can apply a `CreateVectorName(V)` to the source segments via `apply_segments`, leaving the optimizer with sources that have V but a target_config that does not. The optimization then produces a merged segment without V, and a follow-up optimization (running with the refreshed config that includes V) fails to use that segment as a source: "Cannot update from other segment because it is missing vector name X". Close the race by extending the scope of the existing `LockedSegmentHolder::acquire_updates_lock` to cover the proxy install window. `CollectionUpdater::update` already takes this lock before processing any shard update, so concurrent writers wait until proxies are in place — at which point further mutations hit the proxies (recorded as intent and propagated to the merged segment in `finish_optimization`) instead of the originals. The guard is dropped right after proxy install so the slow build phase does not extend it. Tests: - Three `SegmentBuilder::update` tests document the precondition the lock now guarantees: with a target schema that adds a named vector the source lacks, update errors with "missing vector name X". Quantized and mixed-source variants exercise the same error path. - `test_optimize_blocks_proxy_install_on_updates_lock` asserts the invariant directly: while the updates lock is held, proxies are not yet installed. Verified to fail when the new guard is removed (otherwise it passes because `finish_optimization` also takes the same lock, so a naive "did optimize finish?" check would not catch a missing proxy-install guard). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(optimize): finish_optimization lock order; drop redundant tests Address review of #9110: 1. `finish_optimization` was acquiring `upgradable_read` before `acquire_updates_lock`, while the new guard at the start of `execute_optimization` acquires them in the reverse order. With two optimizer threads in flight, thread A in `finish_optimization` could hold `upgradable_read` and wait on `updates_lock` while thread B at the top of `execute_optimization` held `updates_lock` and waited on `upgradable_read` (parking_lot allows only one upgradable reader), deadlocking. Swap `finish_optimization` to take `updates_lock` first so both halves agree. 2. Drop the quantized and mixed-source variants of the inverted `SegmentBuilder` unit test — all three asserted the same error path (the mismatch check fires before quantization training or per-source branching), so only one is useful as documentation of the precondition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: reject source-superset schema mismatch at SegmentBuilder Drop the lock approach (deadlocked test_continuous_snapshot) and fix the bug at the merge layer instead. Snapshot's proxy_all_segments_and_apply acquires the segment_holder upgradable_read first and then takes acquire_updates_lock tactically inside the snapshot operation. The previous commits' lock-extension acquired updates_lock before upgradable_read, so a snapshot in flight and an optimization just entering execute_optimization could deadlock holding each other's required next lock. Snapshot cannot easily reverse its order — that would hold updates_lock for the entire snapshot duration, blocking all writes. Move the fix to where the actual harm happens: SegmentBuilder::update iterates the target's vector_data and silently drops source vectors that aren't in target. That silent drop is what produces the broken merged segment in the CreateVectorName-vs-optimizer race. Add a check that every source vector name is in the target schema; the optimization aborts cleanly on mismatch and the next round (with refreshed config) merges correctly. This is strictly stronger than the lock: the lock only closed the window where V arrived *during* the proxy-install region. The schema check catches both that window and the window where V's apply_segments completed before the optimizer's lock acquisition. Diff is contained to lib/segment; no locking changes, no cross-crate plumbing. test_continuous_snapshot passes again. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(segment_builder): use Cancelled instead of ServiceError for schema mismatch ServiceError flips the shard to RED status (via `report_optimizer_error` → `segments.optimizer_errors`) and stays sticky until the next `recreate_optimizers_blocking` clears it. That's the right shape for hardware/IO failures but wrong for the schema-mismatch case here, which is an expected, recoverable race outcome — the next optimizer round with a refreshed target_config merges the same originals cleanly. `Cancelled` is the variant the optimization worker treats as a recoverable cancellation: logged at debug, tracker marked Cancelled, no `report_optimizer_error` call, no RED status. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(segment_builder): also use Cancelled for the existing target-superset error The existing "missing vector name" check at the start of the merge loop also fires during a race — specifically the optimizer-vs-DeleteVectorName shape, where V is removed from originals before J wraps proxies but J's frozen target_config still has V. Like the new source-superset check, this is an expected, recoverable race outcome, so use Cancelled instead of ServiceError to avoid flipping the shard to RED. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Andrey Vasnetsov
|
75a0a5dd0b |
refactor: move deferred-point ownership into ID tracker (#9062)
* refactor: move deferred-point ownership into the ID tracker Re-implements the idea from #8512 against current `dev`. Deferred-point state (`deferred_internal_id` + `deferred_deleted_count`) moves out of `Segment.deferred_point_status` and the cached `SparseVectorIndex.deferred_internal_id` field into `PointMappings`, exposed through `IdTrackerRead`. The threshold is set once at `MutableIdTracker::open` time; `PointMappings::drop` now maintains the deleted counter inline (with double-delete protection), removing the manual increment in `delete_point_internal` and the `calculate_deleted_deferred_point_count` rescan. Read paths consume the threshold through the id tracker: - The segment read view drops the `deferred_point_status` field and `with_view` no longer threads it in; `read_view/{deferred,info}.rs` call `self.id_tracker.deferred_*()` directly. - `SparseVectorIndex` no longer stores its own copy and its `update_vector` / search debug-assert read from `self.id_tracker.borrow().deferred_internal_id()`. - `VectorQueryContext.deferred_internal_id` and the `SegmentQueryContext::get_vector_context` parameter are gone; the three downstream readers (`plain_vector_index`, sparse search, sparse `update_vector`) consult their own id tracker. `PointMappingsRefEnum` centralises the dispatch: - `iter_internal_with_behavior(DeferredBehavior)` replaces ad-hoc branches in `iter_filtered_points` impls. - `external_iter_cutoff(DeferredBehavior)` covers iterators sourced outside the mapping (field-index outputs in `struct_payload_index::iter_filtered_points`). - The internal `deferred_internal_id()` accessor is private; the raw threshold no longer leaks to consumers. - `iter_from_visible` / `iter_random_visible` read the mapping's own threshold; callers that previously passed `DeferredBehavior::apply(...)` now branch on `deferred_behavior.include_all_points()` (scroll / order_by) or simply drop the argument (sampling / facet). `PayloadIndexRead::query_points` drops the now-redundant `deferred_internal_id` parameter; `iter_filtered_points` takes `DeferredBehavior` directly so HNSW build/search can request `IncludeAll` while normal reads request `Exclude`. RocksDB-related parts of the original PR are skipped — that tracker is already gone from `dev`. Tests adapted: sites that mutated `segment.deferred_point_status` directly now construct a parallel non-deferred segment via `create_deferred_segment(..., 0)` for comparison; `test_deleted_deferred_point_count` reads counters through the id tracker. See `docs/plans/deferred-points-owned-by-id-tracker.md` for the design write-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(benches): drop stale deferred_internal_id arg from query_points calls The boolean / range / conditional bench files weren't built by `cargo test -p segment`, so they slipped through. `cargo clippy --workspace --all-targets` catches them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: drop id_tracker / point_mappings args from iter_filtered_points Both impls already hold an id tracker on `self`: - `StructPayloadIndexReadView` carries `id_tracker: &'a I`, so `self.id_tracker.point_mappings()` borrows from `'a` and the lazy iterator chain keeps working unchanged. - `PlainPayloadIndex` carries `id_tracker: Arc<AtomicRefCell<...>>`, where the mapping borrow is local; collect into a `Vec` and return `into_iter()`. PlainPayloadIndex::iter_filtered_points has no direct callers — only `query_points` was using it — so eager collection is a non-issue. While here, take `self` by value on `iter_internal_visible`, `iter_from_visible`, `iter_random_visible`, `iter_internal_with_behavior`, and `external_iter_cutoff`. `PointMappingsRefEnum` is `Copy`; this matches the existing `iter_internal` / `iter_from` / `iter_random` shape and lets the iterator outlive a local `let point_mappings = ...;` binding. The HNSW `condition_points` helper drops its now-unused `id_tracker` parameter. All callers (sampling, scroll, order_by, facet ×2, hnsw build/search) just drop the two arguments. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: ignore /docs/plans/ and untrack the previously-committed plan `docs/plans/` is a scratch directory for per-feature planning notes — not something we want under source control. Add it to `.gitignore` and drop the deferred-points plan that slipped into history; the design is captured in the PR description. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: replace external_iter_cutoff with filter_deferred iterator wrapper Instead of exposing a raw `Option<PointOffsetType>` cutoff that every caller has to apply with their own `.filter(...)`, give `PointMappingsRefEnum` an iterator wrapper: fn filter_deferred<I: Iterator<Item = PointOffsetType>>( self, iter: I, deferred_behavior: DeferredBehavior, ) -> impl Iterator<Item = PointOffsetType> It returns the iterator unchanged for `IncludeAll` (or when the mapping has no threshold) and otherwise wraps it in a cutoff `.filter`, dispatched via `itertools::Either` so the no-cutoff path stays allocation-free. The struct payload index's `iter_filtered_points` swaps its open-coded filter for a single `point_mappings.filter_deferred(...)` call. The deferred threshold no longer leaks out of `PointMappingsRefEnum`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: move deferred wrapping out of peek_top_all, gate it as test-only `BatchFilteredSearcher::peek_top_all` baked the deferred cutoff into its iterator construction, which was the last place outside `PointMappingsRefEnum` that knew about the threshold. Split the deleted-iteration concern out into a new accessor: fn iter_not_deleted(&self) -> impl Iterator<Item = PointOffsetType> + 'a It borrows `&'a BitSlice` directly (not via `&self`), so callers can chain `filter_deferred` and then move `self` into `peek_top_iter` without lifetime conflicts. Sparse + plain vector index call sites now do: let iter = id_tracker .point_mappings() .filter_deferred(searcher.iter_not_deleted(), DeferredBehavior::Exclude); searcher.peek_top_iter(iter, &is_stopped) leaving `BatchFilteredSearcher` completely ignorant of deferred state. With deferred handling lifted out, `peek_top_all` itself is now used only by tests (3 inline `#[cfg(test)] mod tests`, 1 integration test, 1 bench) — gate it under `#[cfg(feature = "testing")]` to match `new_for_test`. Production code goes through the `iter_not_deleted` + `filter_deferred` + `peek_top_iter` composition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: optimize Segment::retrieve and thread user data through read_vectors Two interlocking changes that together collapse the per-point lookups and intermediate allocations in `Segment::retrieve` down to one external-to-internal pass. ## `IdTrackerRead::resolve_external_ids` (new default trait method) Single-pass translation of a `&[PointIdType]` slice into two parallel vectors `(Vec<PointIdType>, Vec<PointOffsetType>)`. Folds deferred filtering (compare offset against the threshold inline — no separate `point_is_deferred` lookup) and missing-id errors (eager `PointIdError`) into resolution. Lives on the trait so the deferred threshold never leaks out of the id tracker; the parallel-vector shape lets a future batched payload / vector fetcher consume `&offsets` straight without unzipping. The `appendable_flag` guard previously in `point_is_deferred` is gone: non-appendable trackers always carry `deferred_internal_id() == None` (set only via `MutableIdTracker::open`, guarded by the segment constructor), so the check was load-bearing nowhere. ## User-data threading through `read_vectors` `VectorStorageRead::read_vectors` now takes `IntoIterator<Item = (U, PointOffsetType)>` and yields `(U, PointOffsetType, CowVector)`. The user-data tag rides alongside each offset all the way through, so callers can map results back into a parallel array without keeping a separate `offset → ...` lookup table. - Default trait impl: one-line per-key loop. - Dense impl: `unzip()` into parallel `(Vec<U>, Vec<PointOffsetType>)` in a single pass — same allocation count as before, just U riding alongside. - Enum delegations (`VectorStorageEnum`, `VectorStorageReadEnum`) forward unchanged. - `for_each_in_batch` and below stay untouched. `SegmentReadView::vectors_by_offsets<U: Copy>` becomes a lazy filter chain — no parallel `Vec<(orig_idx, offset)>` allocation. The dead `SegmentReadView::read_vectors` helper is removed. ## `Segment::retrieve` end-to-end Per N points / V vectors / payload: | Operation | Before | After | |----------------------------|---------------------|-------| | `id_tracker.internal_id` | N × (1 + V + 1) | N | | `id_tracker.external_id` | N × V | 0 | | `point_is_deferred` | N (when applicable) | 0 | | `offset_to_id` HashMap | N entries | none | | `Vec` in `vectors_by_offsets` | 1 | 0 | The vectors stage passes the external id as `read_vectors`'s user data — the callback gets `id` directly without any index lookup. The payload stage uses `payload_by_offset` against the already-resolved offsets. The shape is also batch-friendly: swapping in a future `IdTrackerRead::batch_internal_id` or `payload_index.batch_get_payload` needs no changes outside the two call sites. ## Behavioural notes - Missing-id now errors eagerly inside resolution, instead of in the vectors stage (`WithVector::Bool(true)` / `Selector`) or payload stage (`with_payload.enable`). The previous `WithVector::Bool(false)` + no-payload path silently inserted an empty record; that is now also an error. None of the existing callers (search post-processing, external retrieve API, the deferred-points test on tests/mod.rs:1179) pass non-existent ids. - Added a per-payload `check_stopped`; the vectors stage already had `stop_if` on its iterator chain. - `vector_by_offset` (the single-element helper) passes `()` as the no-op user data. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: apply rustfmt to optimised retrieve / read_vectors paths Pre-push hook failure on the previous commit was rustfmt. Same content, formatted. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * do not error out on missing points in retrieve --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
xzfc
|
bb8cdf6e5e | Remove unused InvertedIndexMmap / InvertedIndexImmutableRam (#9074) | ||
|
Luis Cossío
|
fe36998514 |
Propagate S to ImmutableIdTracker (#9047)
|
||
|
Andrey Vasnetsov
|
8de9c11a5d |
refactor(index): drop PayloadIndex: PayloadIndexRead super-trait (#8969)
* refactor(index): drop PayloadIndex: PayloadIndexRead super-trait `PayloadIndex` now only declares the mutating surface; reads live on the sibling `PayloadIndexRead` trait. The previous super-trait relationship forced any type that implemented `PayloadIndex` to also implement `PayloadIndexRead`, blocking a future `PayloadIndexRead`-only view that doesn't (and shouldn't) own the writable index machinery. No behavioural change. Audit before committing showed no generic bound site on `PayloadIndex` exists in the workspace, and every caller that uses read methods already imports `PayloadIndexRead` explicitly (the trait was already used as a generic bound on `SegmentReadView`'s `TPayloadIndex` parameter and on `iter_filtered_points`). The full workspace builds clean and all segment / storage / collection tests pass without any consumer update. Doc comment on `PayloadIndex` updated to point readers at `PayloadIndexRead` for the read surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(index): introduce StructPayloadIndexReadView<P, I, V> (#8970) Move the read surface of `StructPayloadIndex` onto a new borrowed view struct generic over `<P: PayloadStorageRead, I: IdTrackerRead, V: VectorStorageRead>`. The view holds exactly the fields that `PayloadIndexRead` requires -- no more, no less: pub struct StructPayloadIndexReadView<'a, P, I, V> { payload: &'a Arc<AtomicRefCell<P>>, id_tracker: &'a I, vector_storages: &'a HashMap<VectorNameBuf, Arc<AtomicRefCell<V>>>, field_indexes: &'a IndexesMap, config: &'a PayloadConfig, visited_pool: &'a VisitedPool, } `StructPayloadIndex` now exposes a `with_view(|v| ...)` accessor that borrows `id_tracker` once at the top and constructs the view for the closure scope. All read-method bodies move onto the view, which is the sole `PayloadIndexRead` implementor for this index. Why three generics ================== - `P: PayloadStorageRead` -- already generic via PR #8968. - `I: IdTrackerRead` -- direct method calls; held as `&I` (not `&Arc<AtomicRefCell<I>>`) because the cell is collapsed at the `with_view` boundary, saving a per-method `borrow()` atomic op. `dyn IdTrackerRead` does not satisfy `I: IdTrackerRead` bounds in Rust without an explicit blanket impl, so generic is the only consistent option here. - `V: VectorStorageRead` -- the only access site is `available_vector_count()` for the `HasVector` cardinality branch (`condition_cardinality` in `read_view/filtering.rs`). Why `payload` keeps the `Arc` ============================= `PayloadProvider<P>::new(...)` (introduced in PR #8968) takes `Arc<AtomicRefCell<P>>` so that the returned `FormulaScorer<'q>` / `Box<dyn FilterContext + 'a>` can outlive the caller frame. The view therefore holds `&'a Arc<AtomicRefCell<P>>` (asymmetric vs the bare `&I` for `id_tracker`). Switching to a borrow-based provider would require reworking `formula_scorer` / `filter_context` to callback style; deferred to a follow-up if needed. What does NOT move ================== - `build_field_indexes` and `clear_index_for_point` stay on `StructPayloadIndex`. `build_field_indexes` is read-shaped but only has write-side callers, and pulls in the `selector` machinery which uses `path` + `storage_type`. Keeping it on the writable struct means `path` and `is_appendable` do not need to leak into the view. - The `selector` / `selector_with_type` helpers stay on the writable struct for the same reason. - The free helpers in `query_optimization/condition_converter.rs` (range / geo / null / is-empty checkers) stay where they are; their visibility is bumped from `fn` to `pub(in crate::index)` so the view can still call them. Module layout ============= lib/segment/src/index/struct_payload_index/ mod.rs # owning struct + with_view build.rs # write-side build coordination payload_index.rs # impl PayloadIndex (mutating only) tests.rs read_view/ mod.rs # view struct + module wiring payload_index_read.rs # impl PayloadIndexRead for view filtering.rs # struct_filtered_context, condition_cardinality, query_field, estimate_field_condition condition_converter.rs # impl block from query_optimization/ optimizer.rs # impl block from query_optimization/ value_retriever.rs # impl block from query_optimization/ tests.rs # smoke test that builds the view directly Consumer migration ================== - `Segment::with_view` nests the new `StructPayloadIndex::with_view` inside it; `SegmentReadViewFor<'s>` uses the view as its `TPayloadIndex` parameter. - HNSW (`hnsw.rs`), sparse (`sparse_vector_index.rs`), plain (`plain_vector_index.rs`) call sites wrap their read-method calls in `payload_index.borrow().with_view(|v| ...)`. - `Segment::get_indexed_fields`, `update_all_field_indices`, and `SegmentBuilder::build` switch to `with_view` for `indexed_fields()` / `get_payload_sequential()`. - Integration tests and benches similarly migrate. - `set_payload` (still on `PayloadIndex` write impl) inlines its former `self.get_payload(...)` call as `self.payload.borrow().get(...)` to avoid going through `with_view` from a `&mut self` write path. Smoke test (`read_view/tests.rs`) constructs the view directly over `InMemoryPayloadStorage` + `InMemoryIdTracker` + an empty vector-storage map, and exercises `indexed_fields()`, `query_points()`, and `available_point_count()` -- proving the view is genuinely decoupled from `StructPayloadIndex`. This is the abstraction PR 4 will use to wire a read-only segment. Verified ======== - `cargo build --workspace --tests --benches` -- green - `cargo test -p segment --lib` -- 666 passed (665 + 1 new smoke test), 0 failed - `cargo test -p segment --tests` -- 120 integration tests pass - `cargo test -p storage --lib` -- 44 passed - `cargo test -p collection --lib` -- 197 passed - `cargo clippy -p segment --tests --benches` -- clean Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Daniel Boros
|
76031bd261 |
feat: split read only payload index (#8858)
* feat: split read only payload index * fix: imports * fix: linter |
||
|
Andrey Vasnetsov
|
144528da31 |
split read only vector store (#8852)
* [AI] split trait for vector store into read only * fmt * fix: trait import --------- Co-authored-by: Daniel Boros <dancixx@gmail.com> |
||
|
Andrey Vasnetsov
|
c1597ac57e |
Split IdTracker trait into IdTrackerRead and IdTracker (#8826)
Read-only methods now live on a separate IdTrackerRead trait, with the mutating IdTracker trait extending it. This lets read-only call sites depend only on the read API. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Andrey Vasnetsov
|
f321c9fe37 |
Low Memory mode (#8714)
* [AI] implement parameter + cover populate + cover quantized vectors * telemetry OpenAPI schema * [AI] hook immutable payload indexes * fmt * do not populate payload index if we fallback to mmap * Reformat * Also suppress universal IO disk cache population --------- Co-authored-by: timvisee <tim@visee.me> |
||
|
Andrey Vasnetsov
|
acfb6503b1 |
crud named vectors (#8605)
* Add empty placeholder vector storage types for named vector CRUD Introduce EmptyDenseVectorStorage and EmptySparseVectorStorage as placeholder storages for newly created named vectors on immutable segments. These report all vectors as deleted, consume no disk space, and are reconstructed from segment config on load via the new VectorStorageType::Empty and SparseVectorStorageType::Empty variants. Key design decisions: - is_on_disk is derived from original user config, not hardcoded - MultiVectorConfig is preserved for multi-vector support - Config mismatch optimizer skips Empty storage to avoid false rebuilds - Quantization delegates normally (handles 0 vectors gracefully) - get_vector includes debug_assert to catch unexpected access Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [AI] segment-level operations for creating and deleting anmed vectors * [AI] implement named vector creation and deleting in proxy segment * [AI] Step 3: Proxy Segment Handling for Named Vector Operations * [AI] implement for Edge * [AI] implement consensus operations for named vector operations * [AI] refactor VectorNameConfig, remove VectorNameConfigInternal * [AI] handle vector schema inconsistency in raft snapshot recovery * [AI] rest + grpc API * [AI] clippy * [AI] generate openAPI schema * fmt * ci fixes * [AI] fix jwt access test * [AI] nop operation for awaiting of consensus-commited update ops * [AI] move vector name operations into points service * [AI] implement internal api for vector name operations * [AI] change collection-level config along with segment level operation * [AI] vector schema reconceliation instead of error * fmt * missing compile-time option * [AI] integration test * [AI] fix missing JWT tests * [AI] remove NOP * [AI] openapi test * [AI] fix initialization of mutable segment * [AI] more simple integration tests * fmt * [AI] make cluster test a bit harder * [AI] make test less flacky * [AI] rabbit comments * [AI] check params compatibility before writing vector config * [AI] make sure to register vector storages in structure payload index * [AI] vector name validation * lower vector length validation to 200 chars to account for prefix in filename * [AI] proxy segment: prevent stale data leak through optimization * fmt * [AI] filter out removed vectors from proxy response * [AI] handle vector name in proxy * fmt * adjust proxy info based on dropped vectors * [AI] proxy segment: update filters to correct has_vector condition * fmt * clippy * Fix consensus snapshot applicaiton for vector schema --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Tim Visée
|
74e51f7339 |
Claude: simplify codebase (#8627)
* [ai] Replace manual into mappings with Into::into * Reformat * [ai] Use implicit .iter * Don't iterate over keys too * [ai] Replace unwrap_or * Reformat * [ai] Use as_deref and then_some * [ai] Use more to_string * [ai] Use explicitly typed into conversions * Reformat * [ai] More explicit into conversions * Reformat |
||
|
Kyamran Shakhaev
|
ee7baa668f |
Use OperationError funcs (#8587)
* Use OperationError funcs * Inline variable * Inline variable --------- Co-authored-by: Tim Visée <tim+github@visee.me> |
||
|
Andrey Vasnetsov
|
f5474ef248 |
Propagate HardwareCounterCell through MmapMapIndex::get_values (#8574)
* Propagate HardwareCounterCell through MmapMapIndex::get_values Previously, `MmapMapIndex::get_values` used `ConditionedCounter::never()`, which silently skipped all hardware IO counter tracking for mmap map index value reads. This propagates a real `HardwareCounterCell` through the full call chain so that disk IO from `values_iter` is properly measured. Changes: - `MmapMapIndex::get_values` now accepts `&HardwareCounterCell`, creates a `ConditionedCounter` via `make_conditioned_counter`, and measures the `deleted` bitvec access (matching `check_values_any` behavior). - `MapIndex::get_values` forwards the counter to the `Mmap` variant. - `FacetIndex::get_point_values` trait method now accepts `&HardwareCounterCell`, propagated through `FacetIndexEnum` and both impls (MapIndex, BoolIndex). - `indexed_variable_retriever` accepts and forwards the counter to `IntMapIndex`, `KeywordIndex`, and `UuidMapIndex` calls. - `SegmentBuilder::update` and `_get_ordering_value` accept and propagate the counter instead of creating disposable instances. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Daniel Boros <dancixx@gmail.com> |
||
|
xzfc
|
0b0df145b3 |
Drop RocksDB (#8529)
* Skip broken tests
* Add rocksdb dropper
This is a temporary tool. It will be removed later in this PR.
* [automated] Drop rocksdb
This commit is made by running tools/rocksdb/drop.sh
* Touch-up after ast-grep
The previous automated commit removed items, but not their comments.
Also, some blocks are left with only one item.
Also, ast-grep can't handle macros like `vec![]`.
This commit completes the job.
* Remove RocksDB dropper
* Remove mentions of rocksdb feature in CI
* Fix clippy warnings
These `FIXME` comments added previously in this PR by ast-grep by
"peeling" cfg_attr like this:
-#[cfg_attr(not(feature = "rocksdb"), expect(...))]
+#[expect(...)] // FIXME(rocksdb): ...
This commit removes these allow/expect attributes and fixes clippy
lints.
* Remove leftover rocksdb-related code
Removed:
- Cargo.toml: rocksdb cargo feature flag and dependencies.
- code: rocksdb-related items that was not under feature flag.
- flags.rs: rocksdb-related qdrant feature flags.
Disabled:
- Some benchmarks because these do not compile now.
* Print warning if rocksdb leftovers found in snapshots
* Remove Clone from tokenizer
* Regenerate openapi.json
|
||
|
Jojii
|
6fc6bcc5b3 |
Don't insert deferred points into sparse index (#8435)
* Don't insert deferred points into sparse index # Conflicts: # lib/segment/src/segment/tests.rs # lib/segment/src/segment_constructor/segment_constructor_base.rs # Conflicts: # lib/segment/src/segment_constructor/segment_constructor_base.rs * Clippy * Assert consistency of deferred_internal_id var * Return before SparseVector conversion in case of deferred point * Use debug_assert instead |
||
|
Ivan Boldyrev
|
2a0b95b08d |
Further split segment traits (#8434)
* Refactor: `SearchSegmentEntry` for search ops * Move `has_deferrend_points_method` to `SearchSegmentEntry` * Reorg imports * Renamings per review * fixup! Renamings per review * `StorageSegmentEntry` trait It contains methods dealing with storage and syncronization * Move index methods from `ReadSegmentEntry` to `SegmentEntry` * Add `_concurrent` suffix to some methods * move field index methods into NonAppendableSegmentEntry --------- Co-authored-by: generall <andrey@vasnetsov.com> |
||
|
Roman Titov
|
04e9bbc4b2 |
Dense vector storage based on universal I/O IoUringFile (#8452)
|
||
|
xzfc
|
7bd5759105 |
Touch-up UIO traits (#8457)
* UIO traits: require Sized I don't think we ever going to add unsized implementation. So, lets, require it on the whole trait rather than on individual methods. * Move `AccessPattern` to the `common` package Was: segment::vector_storage::vector_storage_base::AccessPattern Now: common::generic_consts::AccessPattern * UIO traits: replace `bool` with `AccessPattern` |
||
|
Ivan Boldyrev
|
9146dc4bfe |
Explicit point_mappings guard (#8261)
Instead of `IdTracker` to have `iter_*` methods, it now has a `point_mappings()` method, returning a guard to `PointMapping` value. It prepares for moving the point_mappings under a lock to allow parallel searches and deletions. Also, an incorrect unsafe in `Segment::iter_points` is replaced by a safe version with `self_cell`. |
||
|
Jojii
|
356040312b |
Correct calculation of deferred point counts (#8366)
* Don't account for deferred points in some places # Conflicts: # lib/collection/src/shards/local_shard/scroll.rs * Add in QueryContext * Cover more places * Coderabbit review remarks * Properly count amount of deleted deferred points (#8386) * Properly count amount of deleted deferred points * Prevent double-counting of the same point * Remove hints to estimations * Properly handle counts in ProxySegment * Add tests and fix deleted point count issue * Adjusts tests + fix issues * Separte fields for deferred points in telemetry (SegmentInfo) * Remove deferred_points_count() * openapi * Fix test by manually calculating visible points * Adjust ProxySegment test to revertion of SegmentInfo * Throw error if collection was not found in telemetry (e2e Test) * Update lib/segment/src/segment/segment_ops.rs Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com> * Make new fields in API optional * Don't take range if no deferred point exist * Review remarks --------- Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com> |
||
|
qdrant-cloud-bot
|
6ddb60e5b5 |
Log individual payload index load time per field (#8390)
* Log individual payload index load time per field Add per-field timing to `load_all_fields()` in `StructPayloadIndex`, using the existing `LOAD_TIMING_LOG_TARGET` infrastructure. Each payload field index that takes >= 5ms to load now gets its own log line, making it easy to identify slow-loading fields. Made-with: Cursor * Move `log_load_timing` to common module and use everywhere Move the `log_load_timing` helper from `segment_constructor_base` into `common::defaults` so it can be reused across crates. Convert all raw `log::debug!(target: LOAD_TIMING_LOG_TARGET, ...)` call sites to use the shared function, giving consistent formatting and min-duration suppression everywhere. Made-with: Cursor --------- Co-authored-by: Cursor Agent <agent@cursor.com> |
||
|
qdrant-cloud-bot
|
2406dd9562 |
Suppress noisy zero-duration load timing logs (#8388)
Add a `log_load_timing` helper that checks elapsed time against a 5ms
threshold before logging. Sub-component loads faster than this round to
"0.00s" in the {:.2} format and are pure noise. The per-segment "total
loaded" line remains unconditional.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
|
||
|
qdrant-cloud-bot
|
f90be3fc10 |
Add optional extended load timing logging for storage components (#8363)
* Add debug-level load timing logs for storage components
Add log::debug! calls with target "qdrant::load_timing" that report
how long each storage component takes to load during startup:
- Total shard load time (including WAL replay)
- Total segment load time + load_state
- Per-segment: payload_storage, id_tracker, payload_index
- Per-vector: vector_storage (dense/sparse), quantized_vectors,
vector_index (dense/sparse)
These logs are at DEBUG level so they are silent by default. Enable
selectively via log_level config or env, for example:
log_level: "INFO,qdrant::load_timing=debug"
No config fields, no global flags, no function signature changes.
Made-with: Cursor
* Use seconds with 10ms resolution for load timing logs, extract log target constant
Switch from `{:.2}ms` with `as_secs_f64() * 1000.0` to `{:.2}s` with
`as_secs_f64()` for cleaner output. Move the "qdrant::load_timing"
log target string to a LOAD_TIMING_LOG_TARGET constant in each file.
Made-with: Cursor
* Move LOAD_TIMING_LOG_TARGET to common::defaults
Single definition shared by segment and collection crates, avoiding
duplication of the log target string.
Made-with: Cursor
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
|
||
|
Ivan Pleshkov
|
4b9b39c596 |
Use predefined deferred ID (#8329)
* Adjust points selection for deferred points update * adjust proxy segment implementation * simplify * use simpler proxy impl * stick to Entry API * renaming to stay closer to the original * two passes and simpler impl. * fmt * fmt * use predefined deferred internal id * calculate deferred point id * move deferred check to the entry * fix after rebase * fmt * fix tests * review remarks * fix tests * codespell fix * are you happy clippy --------- Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com> |
||
|
qdrant-cloud-bot
|
7e15d343c2 |
Introduce EdgeShardConfig for edge shard (#8322)
* Introduce EdgeShardConfig for edge shard
- Add EdgeShardConfig and EdgeOptimizersConfig in lib/edge/src/config.rs
- Segment config (vector_data, sparse_vector_data, payload_storage_type)
- Global hnsw_config and per-vector HNSW in segment config
- Optimizer params: deleted_threshold, vacuum_min_vector_number,
default_segment_number, max_segment_size, indexing_threshold,
prevent_unoptimized (excludes memmap_threshold, flush_interval_sec,
max_optimization_threads)
- Persist/load as edge_config.json in shard path
- EdgeShard uses RwLock<EdgeShardConfig>; load() accepts Option<EdgeShardConfig>,
falls back to file or infer from segments; compatibility checked on load
- load_with_segment_config() for backward compatibility (SegmentConfig -> EdgeShardConfig)
- optimize() uses EdgeShardConfig for hnsw and optimizer thresholds
- Public methods: set_hnsw_config(), set_vector_hnsw_config(), set_optimizers_config()
(update and persist)
- Python and examples use load_with_segment_config with existing config API
Made-with: Cursor
* Refactor EdgeShardConfig: user-facing params only, config module
- Replace SegmentConfig inside EdgeShardConfig with user-facing fields:
- on_disk_payload (bool) instead of payload_storage_type
- vectors: HashMap<VectorNameBuf, EdgeVectorParams> with on_disk per vector,
no per-vector quantization; global quantization_config only
- sparse_vectors: HashMap<VectorNameBuf, EdgeSparseVectorParams> with on_disk
- EdgeVectorParams / EdgeSparseVectorParams use on_disk (bool) instead of
storage_type; conversion to VectorDataConfig/SparseVectorDataConfig in
to_segment_config()
- Add config module: mod.rs, optimizers.rs, vectors.rs, shard.rs
- from_segment_config(&SegmentConfig) fills all inferrable params
- to_segment_config() builds SegmentConfig for segments and optimize()
- load_with_segment_config takes Option<SegmentConfig>, uses from_segment_config
Made-with: Cursor
* Move optimizer threshold helpers to shard crate
- Add get_number_segments, get_indexing_threshold_kb, get_max_segment_size_kb,
get_deferred_points_threshold_bytes in shard::optimizers::config
- Collection OptimizersConfig and edge EdgeOptimizersConfig delegate to these
- Single place for threshold logic; collection and edge use shard helpers
Made-with: Cursor
* Use destructuring in config conversions to avoid missing new fields
- EdgeVectorParams: destructure VectorDataConfig in from_*, destructure self in to_vector_data_config
- EdgeSparseVectorParams: destructure SparseVectorDataConfig and SparseIndexConfig in from_*, destructure self in to_sparse_vector_data_config
- EdgeShardConfig: destructure SegmentConfig in from_segment_config, destructure self in to_segment_config
Adding new fields to source structs will now cause compile errors until conversions are updated.
Made-with: Cursor
* refactor: centralize on_disk_payload→payload_storage_type, on_disk→storage_type, and appendable quantization logic
- PayloadStorageType::from_on_disk_payload(bool) in segment (Mmap/InRamMmap)
- VectorStorageType::from_on_disk(bool) in segment (ChunkedMmap/InRamChunkedMmap)
- QuantizationConfig::for_appendable_segment(Option<&Self>) in segment (feature flag + supports_appendable)
- collection: use from_on_disk_payload in non-rocksdb branch
- edge shard/vectors: use new helpers; remove duplicated conditionals
- shard optimizers: use from_on_disk and for_appendable_segment
Made-with: Cursor
* refactor(edge): use EdgeShardConfig directly, drop segment_config
- Add plain_segment_config() for create_appendable_segment (no HNSW)
- Add segment_optimizer_config() built from EdgeShardConfig for blocking optimizers
- Add vector_data_config(name) for query/MMR
- build_blocking_optimizers: use segment_optimizer_config() instead of SegmentConfig
- create_appendable_segment: use plain_segment_config()
- search/query: use config().vectors and vector_data_config() instead of segment_config()
- Remove segment_config() from EdgeShardConfig and EdgeShard
- Add to_plain_vector_data_config on EdgeVectorParams
Made-with: Cursor
* [manual] review changes
* refactor(edge-py): wrap EdgeShardConfig, add EdgeVectorParams/EdgeSparseVectorParams
- PyEdgeConfig now wraps EdgeShardConfig (vectors, sparse_vectors, on_disk_payload, etc.)
- PyEdgeVectorParams / PyEdgeSparseVectorParams wrap edge config types
- PyEdgeOptimizersConfig for optional optimizer settings
- EdgeShard.load() uses EdgeShardConfig; edge::config made pub for Python crate
- cargo fmt + clippy (remove map_identity)
Made-with: Cursor
* refactor(edge-py): simplify config API, remove unused Py* types, add EdgeConfig
- Remove unused PyPayloadStorageType, PyVectorDataConfig, PyVectorStorageType,
PySparseVectorDataConfig, PySparseVectorStorageType from Python bindings
- Move PyEdgeOptimizersConfig to lib/edge/python/src/config/optimizers.rs
- Update qdrant_edge.pyi: EdgeConfig with vectors/sparse_vectors,
EdgeVectorParams, EdgeSparseVectorParams, EdgeOptimizersConfig
- Update examples (common.py, repr.py) to use new config API
- Run cargo fmt
Made-with: Cursor
* [manual] review changes
* [manual] review changes
* [manual] fix test
* Address CodeRabbit review comments for PR 8322 (#8324)
* Address CodeRabbit review comments for PR 8322
- Python examples: explicit imports (repr.py, common.py) and new EdgeConfig API
- HnswIndexConfig: add max_indexing_threads param and property in .pyi and Rust bindings
- EdgeConfig: make vectors optional for sparse-only configs; validate at least one of vectors/sparse_vectors
- EdgeShardConfig::load: use try_exists(), propagate I/O errors
- from_segment_config: infer hnsw_config from per-vector HNSW when all agree
- EdgeShard setters: atomic clone-mutate-save-then-replace; persist config save errors
- Segment compat: prefix vector name in error messages; resolve None datatype to Float32
- max_indexing_threads: preserve 0 (auto) sentinel in trait default; remove per-optimizer overrides
- SegmentOptimizerConfig:🆕 build plain and optimizer maps in single pass
- config_mismatch_optimizer tests: use VectorNameBuf::from() instead of .into()
- vectors.rs: doc updates for per-vector quantization
Made-with: Cursor
* Address @generall review: SaveOnDisk for config, resolve num_rayon_threads in optimizer
- Use SaveOnDisk<EdgeShardConfig> for EdgeShard config (generall: 'We have SaveOnDisk struct for this')
- Create via SaveOnDisk::new() after resolving config; setters use .write() for atomic persist
- set_vector_hnsw_config: clone then mutate then write (fallible setter)
- max_indexing_threads: resolve 0 (auto) via num_rayon_threads inside impl (generall: 'proper solution would be to resolve num_rayon_threads inside the optimizer impl')
- max_indexing_threads_sentinel_aware() now returns Some(num_rayon_threads(raw)) so callers get actual thread count
Made-with: Cursor
* [manual] reorganize num_rayon_threads -> get_num_indexing_threads to better account per-vector configuration
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* update docstring and pyi
* fmt
* fmt
* clipy
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
|
||
|
Luis Cossío
|
c7eadf49de |
Refactor PointToValues to use UniversalRead (#8298)
* refactor PointToValues to use UniversalRead It includes a significant change to `MmapValue` trait to be able to handle Cow reads, instead of just references. * fix str parsing * fix incorrect path * use fallible casting * clippy * No eager allocation this also makes it so that borrowed strs can keep happening :heart-eyes: * Lifetime cleanup --------- Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> |
||
|
Ivan Pleshkov
|
30cf43382b |
Deferred threshold integration (#8246)
* Deferred threshold integration * update deferred id * apply update_deferred_internal_id * fix segment inspector * use avaliable bytes count * renamings * review remarks * review remarks * move has_deferred_points * remove todo * update comments |
||
|
Luis Cossío
|
e28543a604 |
use UniversalRead in ImmutableDenseVectors (#8210)
* use UniversalRead in ImmutableDenseVectors ...renamed from MmapDenseVectors * remove madvise arg * async_raw_scorer |
||
|
Ivan Boldyrev
|
9bf8369a6a |
Use IdTrackerEnum type instead of dyn IdTracker (#8168)
* Use `IdTrackerEnum` type instead of `dyn IdTracker` It would allow to be more flexible on the IdTracker trait, making it dyn-incompatible eventually. Coauthored with Claude Code. * Review fixes |
||
|
xzfc
|
4cabb7fd8e |
Merge io and memory into common (#8155)
* Unify parking_lot/arc_lock feature
* Move lib/common/{io,memory}/* -> lib/common/common/*
- Mmap-related items are grouped into `common::mmap` sub-module:
- `memory/src/chunked_utils.rs` -> `common/src/mmap/chunked.rs`
- `memory/src/madvise.rs` -> `common/src/mmap/advice.rs`
- `memory/src/mmap_ops.rs` -> `common/src/mmap/ops.rs`
- `memory/src/mmap_type_readonly.rs` -> `common/src/mmap/mmap_readonly.rs`
- `memory/src/mmap_type.rs` -> `common/src/mmap/mmap_rw.rs`
- Filesystem-related items are grouped into `common::fs` sub-module:
- `common/src/fs.rs` -> `common/src/fs/sync.rs`
- `io/src/file_operations.rs` -> `common/src/fs/ops.rs`
- `io/src/move_files.rs` -> `common/src/fs/move.rs`
- `io/src/safe_delete.rs` -> `common/src/fs/safe_delete.rs`
- `memory/src/checkfs.rs` -> `common/src/fs/check.rs`
- `memory/src/fadvise.rs` -> `common/src/fs/fadvise.rs`
- Rest is moved straight into `common`:
- `io/src/storage_version.rs` -> `common/src/storage_version.rs`
The old `io` and `memory` are now hollow crates that re-export items
from `common`. These hollow crates will be removed in next commits.
* Replace uses of `io` and `memory` with new paths in `common`
Since `io` and `memory` are just re-exports of `common`, these
replacements are no-op.
* Remove `io` and `memory` crates
|