* Strict mode: add `max_disk_usage_percent`
Mirrors `max_resident_memory_percent`: rejects disk-consuming update ops
(upsert, set/overwrite payload, update vectors) when the filesystem hosting
Qdrant storage is filled above the configured percentage. Delete-style ops
remain allowed so callers can free disk.
Disk usage is sampled via `statvfs` and TTL-cached for 5s (same cadence as
the resident-memory reader) so high-RPS request paths don't hammer the
syscall. Reader is keyed by path in `common::disk_usage` and returns `None`
on stat failure — callers (the strict-mode check) treat `None` as "skip",
matching the memory-check behaviour.
Plumbing follows the existing pattern: field on `StrictModeConfig` (+
output/diff/Hash), gRPC proto field `22`, validation 1..=100, REST/proto
conversions, and the hook into `check_strict_mode_toc_batch` alongside the
memory check (both guarded by `any_consumes_memory`).
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix CI: Windows disk_usage test + e2e WAL config
- `missing_path_returns_none` panicked on Windows because
`GetDiskFreeSpaceEx` succeeds for non-existent paths (it resolves up to
the containing drive). Relax the assertion to "must not panic; if a
value is returned it must be well-formed". The contract we care about
(None on failure) is platform-defined, not something we can portably
force.
- e2e test failed at batch 0 with "WAL buffer size exceeds available disk
space": Qdrant's existing per-shard `DiskUsageWatcher` enforces
`free >= 2 * wal_capacity_mb` and the default WAL didn't fit in the
50 MB tmpfs. Bump tmpfs to 200 MB and shrink `wal_capacity_mb` to 1 MB
(same pattern as `test_low_disk.py`) so our strict-mode gate is the
one that fires, not the WAL pre-check. Raise the gate threshold to
50% to match the larger headroom.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Migrate InvertedIndexCompressedMmap to UniversalRead
* Rehaul search_scratch.rs (was scores_memory_pool.rs)
- Use `typed_arena::Arena` instead of `bumpalo::Bump`.
Reason: bump don't drop.
Drawback: `Arena::new()` is not free, and `Arena::clear()` doesn't
exist; but this commit has workarounds.
- Use names that make more sense.
* Misc fixes
* InvertedIndexCompressedMmap: explicit S type parameter
* load with any universal io
* Don't populate with sequential advice
* use appropriate fs for `exists`
* use `read_whole_via` more
* TODO
* clear ram cache after read_whole
* [ai] TQDT in the API
* [ai] unify new sparse error for TQDT
* Rename to `turbo4`
* Add `Turbo4` to comments and doc strings.
* Also validate named sparse vector creation
* ci(windows): skip IO-heavy tests that aren't OS-specific
On the Windows CI runner, several tests are 3-25x slower than on Ubuntu
purely due to slow filesystem IO. These tests exercise platform-agnostic
logic (optimizer, snapshot, WAL recovery, dedup, deferred points) and
are fully covered by the Linux and macOS jobs.
Mark them with `#[cfg_attr(target_os = "windows", ignore = "...")]` so:
- Windows CI skips them and finishes faster.
- They're still listed and runnable via `cargo test -- --ignored`
on Windows for local debugging.
Based on JUnit timings from CI run 26462785436 (PR #8827), this should
save ~5 minutes wall-clock on the Windows job, taking it closer to the
~13min Ubuntu and ~9min macOS jobs (currently 20m24s).
Tests affected:
- lib/wal: check_wal, check_last_index, check_clear, check_reopen,
check_truncate, check_prefix_truncate, test_prefix_truncate_parametric
- lib/edge/optimize: full tests module
- lib/segment deferred-point tests: read_operations,
dense_segment_combinations, sparse, facets
- lib/collection: snapshot_test, points_dedup, wal_recovery,
collection_test::test_ordered_read_api, snapshot_recovery_test
Co-authored-by: Cursor <cursoragent@cursor.com>
* revert(ci/windows): keep WAL and WAL-recovery tests on Windows
Reviewer correctly pointed out that WAL is mmap-backed and has
substantial Windows-specific code paths:
- Different segment allocation (fs4 vs rustix::ftruncate)
- Windows-specific delete_windows() with mmap-drop + retry loop
- Windows-specific sync_all() because directory fsync is unavailable
- Windows-specific lock proxy file (directories aren't lockable)
So those tests genuinely need Windows coverage. Reverted skips for:
- lib/wal/src/lib.rs: all check_* tests and test_prefix_truncate_parametric
- lib/collection/src/tests/wal_recovery_test.rs: all three tests
Still skipped on Windows (no OS-specific code in their production paths):
- lib/edge/optimize.rs (no cfg(windows) in source)
- lib/segment deferred-point tests (segment/ has no cfg(windows))
- lib/collection snapshot/dedup tests (collection/ has no cfg(windows))
- lib/collection integration snapshot_recovery + ordered_read_api
Co-authored-by: Cursor <cursoragent@cursor.com>
* revert(ci/windows): keep collection integration and snapshot_test
Per reviewer request, keep running these on Windows:
- lib/collection/tests/integration/* (snapshot_recovery_test,
collection_test::test_ordered_read_api)
- lib/collection/src/tests/snapshot_test.rs
These exercise higher-level collection/snapshot behavior that benefits
from cross-platform validation.
Remaining Windows skips (production code has no cfg(windows) branches):
- lib/edge/src/optimize.rs: 14 optimizer tests
- lib/segment/src/segment/tests/mod.rs: 4 deferred-point tests
- lib/collection/src/tests/points_dedup.rs: 2 dedup tests
Co-authored-by: Cursor <cursoragent@cursor.com>
* ci(windows): also skip HNSW/quantization integration tests
Per reviewer, also skip these segment integration test modules on Windows:
- hnsw_quantized_search_test::* (25 tests)
- multivector_filtrable_hnsw_test::* (rstest cases)
- multivector_quantization_test::* (rstest cases)
- byte_storage_quantization_test::* (rstest cases)
- payload_index_test::test_struct_payload_index_nested_fields
These exercise pure HNSW/quantization correctness on top of standard
segment IO that is already covered by tests we keep running on Windows.
Adds ~930s of sequential time to the Windows skip list, bringing the
expected wall-clock saving from ~3 min to ~8-10 min.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: add io_bridge
* fix: cached dispatcher
* feat: add open_with_handle
* fix: naming
* fix: wording
* fix: rebase io_bridge onto split BorrowedReadPipeline/OwnedReadPipeline
* fix: linter
* feat: add S3 backend
* chore: remove io_bridge
* fix: linter
* fix: linter
* fix: read handle
* chore: simplify S3Source
* fix: s3 test
* feat: support multi runtime
* fix: clippy errors
* fix: review comments
* feat: add io design
* feat: add S3 backend
* chore: fix docs
* fix: dev changes
* chore: add some docs
* chore: remove explicit type
* feat: add new methods
* [WIP] review refactor
* fmt
* fix: bytes alignment
* fix: linter
* feat: remove Bytes
* fix: ci/cd
* fix: tests
* fix: tests
* refactor: simplify io_bridge pipeline to Handle-based dispatch
Replace the BridgeRuntime worker thread + request channel + boxed
BridgeRequest with direct tokio Handle usage:
- BridgeRuntime is now just an Arc<Runtime>; schedule() spawns the read
future via Handle::spawn instead of routing it through a dispatcher
thread. Removes BridgeRequest and the now-unreachable S3RuntimeShutDown
error variant.
- Guard against a panicking read task hanging wait() forever: the spawned
task catches unwinds and converts them into a TaskPanicked error reply,
so every scheduled slot is always answered.
- Encapsulate slot bookkeeping in PendingSlots, exposing only the needed
operations instead of a public map + counter.
- Split the grown pipeline.rs into a pipeline/ module (slots / inner /
borrowed / owned), de-duplicating the shared read-future construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: move pipeline buffer ownership into the read future
Instead of the pipeline owning the destination Vec<T> in a slot map and
the future writing through a SendBytePtr raw pointer, let the future
allocate the buffer itself and return it through BridgeResponse. The
buffer crosses the worker-thread boundary as a normal move via the reply
channel, wrapped in a SendableVec<T> newtype that asserts Send for
T: bytemuck::Pod only.
This removes the entire unsafe SendBytePtr apparatus from the pipeline:
no raw pointer, no unsafe fn, no per-call-site unsafe blocks, no
heap-stability invariants. The only remaining unsafe in the crate is one
bounded `unsafe impl<T: Pod> Send for SendableVec<T>` with a trivially
true invariant (Pod types are plain bytes).
PendingSlots collapses back to PendingSlots<U>: slots no longer carry
buffers. AlignedBufWriter::from_raw_bytes (used only by the SendBytePtr
path) and its test are removed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: drop SendableVec wrapper now that Item: Send
UniversalRead's element type is now bound to `Item` (`Pod + Send`),
so the io_bridge pipeline no longer needs a hand-rolled `Send` wrapper
around `Vec<T>` to ship buffers through the reply channel. Replace
`SendableVec<T>` with `Vec<T>` end-to-end and tighten the local impls
to `T: Item`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: implement UniversalRead::reopen for BlobFile
BlobFile has no cached file metadata or mapping — `len()` queries the
object store fresh on each call — so reopen is a no-op, matching the
io_uring impl.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add new fs impl for Blob
* fix: is_in_ram_or_mmap for S3
---------
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [UIO] Split UniversalReadFileOps into filesystem + file traits
`UniversalReadFileOps` is now an instance-based trait describing a
filesystem handle (list/exists with `&self`, plus `from_context`). A new
`UniversalReadFs: UniversalReadFileOps` subtrait adds the
`open(&self, path, options) -> Self::File` capability with `type File:
UniversalRead`. `UniversalRead` no longer extends `UniversalReadFileOps`
and is purely a file-handle trait.
This separates "filesystem instance" from "file handle". Backends that
need per-instance configuration (S3 bucket name + credentials, mmap
default advice, io_uring runtime, block-cache controller `Arc`) gain a
typed home in `Self::ContextConfig`, and `list_files`/`exists`/`open`
become `&self` methods on the filesystem handle.
Concrete filesystem handles introduced for the three existing backends:
- `MmapFs` — unit struct; `ContextConfig = ()`; produces `MmapFile`.
- `IoUringFs` — carries `prevent_caching`; `ContextConfig =
IoUringConfigContext`; produces `IoUringFile`. `IoUringConfigContext`
becomes the construction input rather than a per-open argument.
- `BlockCacheFs` — carries `Arc<CacheController>`; `ContextConfig =
BlockCacheConfigContext`; produces `CachedSlice`.
`TConfigContext` (universal builder methods) is kept so generic-over-`Fs`
code can still set cross-backend knobs via
`Fs::ContextConfig::default().with_prevent_caching(true)`.
Wrappers (`ReadOnly<S>`, `TypedStorage<S, T>`, `StoredStruct<S, T>`),
higher-level storages (`StoredBitSlice<S>`, `UniversalHashMap<K, V, S>`)
keep `<S: UniversalRead>` parameterization but their `open(...)`
constructors now grow a `fs: &Fs` argument bound by
`Fs: UniversalReadFs<File = S>`. `read_json_via` becomes
`read_json_via(fs: &Fs, path)`.
All test code and benches in `common` updated to construct
`MmapFs`/`IoUringFs` inline as needed. `common` compiles cleanly with
tests and benches. `gridstore`, `segment`, and `tonic` caller updates
are in flight in subsequent commits.
* WIP: gridstore + segment caller sweep (partial)
Threads `fs: &Fs` through gridstore's `BitmaskGaps`, `Bitmask`, `Pages`,
and `Gridstore::new`/`open`/`create_new_page`. Most segment callers
have `OpenOptions { extra: ... }` removed and `S::open(path, opts, ctx)`
sites updated mechanically but the trait change is not yet propagated.
Does NOT compile yet. Tracker still has static `S::open` calls, segment
generic constructors (`MmapInvertedIndex<S>`, `UniversalMapIndex`,
`StoredGeoMapIndex`, etc.) still call `S::open`/`S::list_files`/`S::exists`
statically — they need an `fs: &Fs` parameter added. Tonic API
`StorageReadService<S>` also unconverted.
Committed as branch checkpoint; cascade continues in subsequent work.
* gridstore: thread `fs: &Fs` through Bitmask, BitmaskGaps, Pages, Tracker
Per the new `UniversalReadFs` shape, every constructor/method that opens
files takes an `fs: &Fs` parameter. Gridstore is currently mmap-only,
so the top-level `Gridstore` / `GridstoreReader::open` callers in the
crate pass `&MmapFs` inline. Tests do the same.
gridstore lib + tests now compile cleanly. Segment + tonic cascade
still pending.
* WIP: segment caller sweep — dynamic_stored_flags first
* WIP: segment cascade - id_tracker partial
* common benches: update to new UniversalReadFs::open shape (clippy clean)
* segment flags: thread Fs through BufferedDynamicFlags / Bitvec / Roaring
DynamicStoredFlags::set_len now takes `fs: &Fs`. BufferedDynamicFlags
stores an `Arc<Fs>` so the flusher closure can call `set_len` on resize.
BitvecFlags and RoaringFlags expose a new `Fs` type parameter and the
flag tests now pass `Fs::default()` (MmapFs/IoUringFs via duplicate_item).
Concrete consumers (bool/null index, mmap dense/multi/sparse storages)
pin `Fs = MmapFs` and pass `&MmapFs` to inner opens.
* segment: thread Fs through field-index lifecycle methods
Apply the new UniversalReadFs::open shape across:
- full_text_index (MmapInvertedIndex, MmapFullTextIndex, UniversalPostings)
- geo_index (StoredGeoMapIndex build/open + tests + builders)
- numeric_index lifecycle (UniversalNumericIndex build/open)
- map_index lifecycle (UniversalMapIndex build/open)
- stored_point_to_values (open / from_iter)
Concrete consumers pin Fs = MmapFs and pass &MmapFs inline; generic
open paths thread `fs: &Fs` where Fs: UniversalReadFs<File = S>.
* segment: thread Fs through chunked vectors and id-tracker callers
- ChunkedVectors gains `Fs` generic so add_chunk can call create_chunk
after open. ChunkedVectorsRead/load_config and chunks::{read_chunks,
create_chunk} take `fs: &Fs`. Concrete callers (dense / multi-dense /
sparse / quantized) pass MmapFs inline.
- DenseVectorStorageImpl stores `fs: Fs` so `update_from` can reopen
ImmutableDenseVectors. ImmutableDenseVectors::open takes `fs: &Fs`.
- VectorStorageEnum DenseUring* variants thread IoUringFs alongside
IoUringFile.
- segment_builder + segment_constructor_base pass MmapFs to
ImmutableIdTracker::{new, open}.
- QuantizedStorage::from_file takes `fs: &Fs`; quantized_vectors callers
pass &MmapFs.
* segment: apply nightly rustfmt after Fs refactor
cargo +nightly fmt --all over the segment crate after the
UniversalReadFs cascade. No semantic changes.
* segment: thread Fs through benches and id-tracker tests
Update the dynamic-mmap-flags and buffered-update-bitslice benches to
the new UniversalReadFs::open shape (pass `&MmapFs`). Update the
immutable-id-tracker test suite to forward `&MmapFs` to
`from_in_memory_tracker` / `open`.
* uio: pin Fs via UniversalRead::Fs assoc type; per-call OpenExtra
Two design changes that fall out of the per-instance Fs refactor:
1. Bidirectional Fs ↔ File pinning. `UniversalRead::Fs:
UniversalReadFs<File = Self>` lets generic-over-`S` code refer to
`S::Fs` directly instead of carrying an extra `<Fs: UniversalReadFs<File = S>>`
generic param. `ReadOnly<S>` wraps a file but has no natural
filesystem; a phantom `ReadOnlyFs<S::Fs>` satisfies the constraint
while inherent `ReadOnly::open` keeps taking `&S::Fs` directly.
2. `prevent_caching` moves from filesystem-instance state to per-call
`UniversalReadFs::OpenExtra: Default`. Was previously a knob on
`IoUringConfigContext` / `IoUringFs`, conflating "how this fs is
built" with "how this file is opened." Now `IoUringFs::OpenExtra =
IoUringOpenExtra { prevent_caching }`; mmap and block-cache use `()`.
`IoUringConfigContext` is gone, `TConfigContext` slims to a `Default`
marker.
Tonic StorageReadService holds `Arc<S::Fs>` (was `PhantomData<S>`); its
`new()` builds via `S::Fs::from_context(default)` and the spawn_blocking
closures clone the Arc to call instance methods.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: cfg-gate IoUringOpenExtra import for non-linux builds
The IoUringOpenExtra reexport from `universal_io` is gated on
`target_os = "linux"`. The previous commit left an unconditional import
in `persisted_hashmap/tests.rs`, breaking macOS/Windows CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: migrate simple_disk_cache to per-instance Fs API
PR #9097 (merged into dev concurrently with this branch) introduced a
`DiskCache<R: UniversalRead>` using the pre-refactor trait shape:
file-handle-as-filesystem (`R::open`, `R::list_files`), an
`OpenOptionsExtra` field on `OpenOptions`, and trait methods without
`&self`. The Fs-instance refactor on this branch removed all three.
Reshape `simple_disk_cache` to match the new design without breaking the
lazy mirror semantics:
- New `DiskCacheFs<R>` is the filesystem handle. Holds a clone of the
remote `R::Fs`; `list_files`/`exists` delegate; `from_context`
forwards to the inner Fs context. `open` constructs a `DiskCache<R>`
via the global `DiskCacheConfig`.
- `DiskCache<R>` now stores `remote_fs: R::Fs` + `remote_extra:
<R::Fs as UniversalReadFs>::OpenExtra`, so lazy remote opens go
through `self.remote_fs.open(path, options, extra)` instead of the
removed `R::open`. `open_with_config` takes the remote Fs + extra
explicitly (no more hard-coded `prevent_caching: true`; callers pass
the appropriate `OpenExtra`).
- `UniversalRead for DiskCache<R>` now declares `type Fs =
DiskCacheFs<R>` (no more `open` method on the file trait).
- Propagate the necessary bounds (`R::Fs: Clone`,
`<R::Fs as UniversalReadFs>::OpenExtra: Clone`,
`R::OwnedReadPipeline<u8, Range<u32>>: Send`) through `pipeline.rs`
free functions and impl blocks that reach into `DiskCache::remote` /
`local_state`.
- Drop the now-removed `extra: _` destructure in `LocalState::new`.
- Tests construct the remote Fs via `R::Fs::from_context(Default::default())`
and exercise `DiskCache::open_with_config`. 17 simple_disk_cache
tests pass; the 3 `empty_read_does_not_materialize_local_file`
failures pre-exist on dev (verified) and are unrelated.
`mold -run cargo clippy --all-targets` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt on simple_disk_cache migration
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt for local_state imports after rebase
Co-authored-by: Cursor <cursoragent@cursor.com>
* uio: split DiskCacheFs into its own module
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* uio: replace TConfigContext with OpenExtra trait; move DiskCacheConfig onto DiskCacheFs
- Drop the empty TConfigContext marker; ContextConfig is now unconstrained
so backends can require explicit construction.
- Add OpenExtra trait with with_prevent_caching for backend-agnostic
per-call knobs; impl for () (no-op) and IoUringOpenExtra.
- DiskCacheFs now carries Arc<DiskCacheConfig> via the new
DiskCacheFsContext<C>; the prefill flow moves from the deleted
open_with_config into DiskCacheFs::open so Populate::Blocking /
PreferBackground work through the trait API.
- Remove the DiskCacheConfig global; callers must construct the context
explicitly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: nightly fmt after OpenExtra refactor
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: fix spelling — Implementors → Implementers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: possible panic insetad of error propagation
* fix: missing cfg annotation
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Daniel Boros <dancixx@gmail.com>
* refactor: pass point_id into handle_point_version_and_failure
Drop the external_id reverse lookup used for error_status correlation
and take the external point_id as an explicit argument instead. All
callers already have it in scope, and decoupling it from op_point_offset
keeps error correlation correct in flows where the old internal_id is
tombstoned before the recovery check runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: filter_deferred_and_deleted also consults the deleted bitslice
Field-index primary-clause iterators (and the analogous plain/sparse
vector-index paths) are routed through PointMappingsRefEnum to apply the
deferred-threshold cutoff. The old `filter_deferred` only applied that
threshold, so any soft-deleted internal id sitting below the cutoff
slipped through whenever its field-index posting was still live. This
was fine while the only source of mid-range tombstones was the
deferred-tail design, but it breaks the moment a tombstone can land
anywhere in the id range.
Add an unconditional deleted-bitslice check (single bit test per
element) and rename to `filter_deferred_and_deleted` so the contract is
visible at the call site. The Either split is preserved so the
no-threshold path still avoids the cutoff comparison.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: cargo +nightly fmt
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* style: collapse error-status recovery match arm
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a `pub trait Item: bytemuck::Pod + Send` marker (mirroring the
existing `UserData` pattern) and use it in place of `T: bytemuck::Pod`
on the read side of `UniversalRead`, its pipeline traits, and all
impls/wrappers. Some implementations buffer `Vec<MaybeUninit<T>>` that
may be transferred across threads in future backends; tightening the
bound makes that explicit at the trait level instead of leaving callers
to add `+ Send` ad-hoc. Write paths keep the looser `bytemuck::Pod`
bound since they only borrow `&[T]`.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The segment builder evicts the page cache of vector storage, quantized
vectors, payload, payload index and the vector index after a build to
avoid cache pollution, but the id tracker was never cleared. Its on-disk
files (mappings, versions, deleted bitslice) are written during the build
and stay resident in the page cache, so after each optimization the id
tracker files linger as cache even though they are meant to be on-disk
only (expected_cache_bytes == 0).
Add `IdTracker::clear_cache` (default no-op) plus a `clear_cache_if_on_disk`
policy wrapper, and implement the eviction for the mutable and immutable
trackers:
- ImmutableIdTracker pages out its two mmap-backed storages via madvise
and drops the RAM-loaded mappings file via fadvise(DONTNEED).
- MutableIdTracker drops its append-only log files via fadvise(DONTNEED).
The builder calls `clear_cache_if_on_disk`, mirroring the payload index.
The id tracker has no on-disk mode yet, so this always clears for now;
a TODO marks where to gate it once an on-disk mode exists.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: serialize optimization proxy install against shard updates
execute_optimization captures `target_config` from the optimizer's frozen
config and then wraps source segments in proxies. Between those two points,
`CollectionUpdater::update` can apply a `CreateVectorName(V)` to the source
segments via `apply_segments`, leaving the optimizer with sources that have
V but a target_config that does not. The optimization then produces a
merged segment without V, and a follow-up optimization (running with the
refreshed config that includes V) fails to use that segment as a source:
"Cannot update from other segment because it is missing vector name X".
Close the race by extending the scope of the existing
`LockedSegmentHolder::acquire_updates_lock` to cover the proxy install
window. `CollectionUpdater::update` already takes this lock before
processing any shard update, so concurrent writers wait until proxies are
in place — at which point further mutations hit the proxies (recorded as
intent and propagated to the merged segment in `finish_optimization`)
instead of the originals. The guard is dropped right after proxy install so
the slow build phase does not extend it.
Tests:
- Three `SegmentBuilder::update` tests document the precondition the lock
now guarantees: with a target schema that adds a named vector the source
lacks, update errors with "missing vector name X". Quantized and
mixed-source variants exercise the same error path.
- `test_optimize_blocks_proxy_install_on_updates_lock` asserts the
invariant directly: while the updates lock is held, proxies are not yet
installed. Verified to fail when the new guard is removed (otherwise it
passes because `finish_optimization` also takes the same lock, so a
naive "did optimize finish?" check would not catch a missing proxy-install
guard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(optimize): finish_optimization lock order; drop redundant tests
Address review of #9110:
1. `finish_optimization` was acquiring `upgradable_read` before
`acquire_updates_lock`, while the new guard at the start of
`execute_optimization` acquires them in the reverse order. With two
optimizer threads in flight, thread A in `finish_optimization` could
hold `upgradable_read` and wait on `updates_lock` while thread B at the
top of `execute_optimization` held `updates_lock` and waited on
`upgradable_read` (parking_lot allows only one upgradable reader),
deadlocking. Swap `finish_optimization` to take `updates_lock` first so
both halves agree.
2. Drop the quantized and mixed-source variants of the inverted
`SegmentBuilder` unit test — all three asserted the same error path
(the mismatch check fires before quantization training or per-source
branching), so only one is useful as documentation of the precondition.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: reject source-superset schema mismatch at SegmentBuilder
Drop the lock approach (deadlocked test_continuous_snapshot) and fix the bug
at the merge layer instead.
Snapshot's proxy_all_segments_and_apply acquires the segment_holder
upgradable_read first and then takes acquire_updates_lock tactically inside
the snapshot operation. The previous commits' lock-extension acquired
updates_lock before upgradable_read, so a snapshot in flight and an
optimization just entering execute_optimization could deadlock holding each
other's required next lock. Snapshot cannot easily reverse its order — that
would hold updates_lock for the entire snapshot duration, blocking all
writes.
Move the fix to where the actual harm happens: SegmentBuilder::update
iterates the target's vector_data and silently drops source vectors that
aren't in target. That silent drop is what produces the broken merged
segment in the CreateVectorName-vs-optimizer race. Add a check that every
source vector name is in the target schema; the optimization aborts cleanly
on mismatch and the next round (with refreshed config) merges correctly.
This is strictly stronger than the lock: the lock only closed the window
where V arrived *during* the proxy-install region. The schema check catches
both that window and the window where V's apply_segments completed before
the optimizer's lock acquisition.
Diff is contained to lib/segment; no locking changes, no cross-crate
plumbing. test_continuous_snapshot passes again.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(segment_builder): use Cancelled instead of ServiceError for schema mismatch
ServiceError flips the shard to RED status (via `report_optimizer_error` →
`segments.optimizer_errors`) and stays sticky until the next
`recreate_optimizers_blocking` clears it. That's the right shape for
hardware/IO failures but wrong for the schema-mismatch case here, which is
an expected, recoverable race outcome — the next optimizer round with a
refreshed target_config merges the same originals cleanly.
`Cancelled` is the variant the optimization worker treats as a recoverable
cancellation: logged at debug, tracker marked Cancelled, no
`report_optimizer_error` call, no RED status.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(segment_builder): also use Cancelled for the existing target-superset error
The existing "missing vector name" check at the start of the merge loop
also fires during a race — specifically the optimizer-vs-DeleteVectorName
shape, where V is removed from originals before J wraps proxies but J's
frozen target_config still has V. Like the new source-superset check, this
is an expected, recoverable race outcome, so use Cancelled instead of
ServiceError to avoid flipping the shard to RED.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [AI + manual] initial impl
[AI] read_batch which actually batches
manual nits
[AI] better handling of local and remote paths
manual refactor, respect open options
don't delete local file
dumbify read_batch
we want to refactor it anyway
simplify
rename to `DiskCache` in `simple_disk_cache` module
* refactor to use always use ReadPipeline
pass meta to remote pipeline
* nits
* run tests for more Remotes
* fix no more <T> in UniversalRead
* fmt
* chore(deps): unify roaring as workspace dep, move duplicate to dev-deps
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: move deferred-point ownership into the ID tracker
Re-implements the idea from #8512 against current `dev`.
Deferred-point state (`deferred_internal_id` + `deferred_deleted_count`)
moves out of `Segment.deferred_point_status` and the cached
`SparseVectorIndex.deferred_internal_id` field into `PointMappings`,
exposed through `IdTrackerRead`. The threshold is set once at
`MutableIdTracker::open` time; `PointMappings::drop` now maintains the
deleted counter inline (with double-delete protection), removing the
manual increment in `delete_point_internal` and the
`calculate_deleted_deferred_point_count` rescan.
Read paths consume the threshold through the id tracker:
- The segment read view drops the `deferred_point_status` field and
`with_view` no longer threads it in; `read_view/{deferred,info}.rs`
call `self.id_tracker.deferred_*()` directly.
- `SparseVectorIndex` no longer stores its own copy and its
`update_vector` / search debug-assert read from
`self.id_tracker.borrow().deferred_internal_id()`.
- `VectorQueryContext.deferred_internal_id` and the
`SegmentQueryContext::get_vector_context` parameter are gone; the
three downstream readers (`plain_vector_index`, sparse search, sparse
`update_vector`) consult their own id tracker.
`PointMappingsRefEnum` centralises the dispatch:
- `iter_internal_with_behavior(DeferredBehavior)` replaces ad-hoc
branches in `iter_filtered_points` impls.
- `external_iter_cutoff(DeferredBehavior)` covers iterators sourced
outside the mapping (field-index outputs in
`struct_payload_index::iter_filtered_points`).
- The internal `deferred_internal_id()` accessor is private; the raw
threshold no longer leaks to consumers.
- `iter_from_visible` / `iter_random_visible` read the mapping's own
threshold; callers that previously passed
`DeferredBehavior::apply(...)` now branch on
`deferred_behavior.include_all_points()` (scroll / order_by) or simply
drop the argument (sampling / facet).
`PayloadIndexRead::query_points` drops the now-redundant
`deferred_internal_id` parameter; `iter_filtered_points` takes
`DeferredBehavior` directly so HNSW build/search can request
`IncludeAll` while normal reads request `Exclude`.
RocksDB-related parts of the original PR are skipped — that tracker is
already gone from `dev`.
Tests adapted: sites that mutated `segment.deferred_point_status`
directly now construct a parallel non-deferred segment via
`create_deferred_segment(..., 0)` for comparison;
`test_deleted_deferred_point_count` reads counters through the id
tracker.
See `docs/plans/deferred-points-owned-by-id-tracker.md` for the design
write-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(benches): drop stale deferred_internal_id arg from query_points calls
The boolean / range / conditional bench files weren't built by
`cargo test -p segment`, so they slipped through. `cargo clippy
--workspace --all-targets` catches them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: drop id_tracker / point_mappings args from iter_filtered_points
Both impls already hold an id tracker on `self`:
- `StructPayloadIndexReadView` carries `id_tracker: &'a I`, so
`self.id_tracker.point_mappings()` borrows from `'a` and the lazy
iterator chain keeps working unchanged.
- `PlainPayloadIndex` carries `id_tracker: Arc<AtomicRefCell<...>>`,
where the mapping borrow is local; collect into a `Vec` and return
`into_iter()`. PlainPayloadIndex::iter_filtered_points has no direct
callers — only `query_points` was using it — so eager collection is a
non-issue.
While here, take `self` by value on `iter_internal_visible`,
`iter_from_visible`, `iter_random_visible`,
`iter_internal_with_behavior`, and `external_iter_cutoff`.
`PointMappingsRefEnum` is `Copy`; this matches the existing
`iter_internal` / `iter_from` / `iter_random` shape and lets the
iterator outlive a local `let point_mappings = ...;` binding.
The HNSW `condition_points` helper drops its now-unused `id_tracker`
parameter.
All callers (sampling, scroll, order_by, facet ×2, hnsw build/search)
just drop the two arguments.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: ignore /docs/plans/ and untrack the previously-committed plan
`docs/plans/` is a scratch directory for per-feature planning notes —
not something we want under source control. Add it to `.gitignore` and
drop the deferred-points plan that slipped into history; the design is
captured in the PR description.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: replace external_iter_cutoff with filter_deferred iterator wrapper
Instead of exposing a raw `Option<PointOffsetType>` cutoff that every
caller has to apply with their own `.filter(...)`, give
`PointMappingsRefEnum` an iterator wrapper:
fn filter_deferred<I: Iterator<Item = PointOffsetType>>(
self,
iter: I,
deferred_behavior: DeferredBehavior,
) -> impl Iterator<Item = PointOffsetType>
It returns the iterator unchanged for `IncludeAll` (or when the mapping
has no threshold) and otherwise wraps it in a cutoff `.filter`,
dispatched via `itertools::Either` so the no-cutoff path stays
allocation-free.
The struct payload index's `iter_filtered_points` swaps its open-coded
filter for a single `point_mappings.filter_deferred(...)` call. The
deferred threshold no longer leaks out of `PointMappingsRefEnum`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: move deferred wrapping out of peek_top_all, gate it as test-only
`BatchFilteredSearcher::peek_top_all` baked the deferred cutoff into
its iterator construction, which was the last place outside
`PointMappingsRefEnum` that knew about the threshold. Split the
deleted-iteration concern out into a new accessor:
fn iter_not_deleted(&self) -> impl Iterator<Item = PointOffsetType> + 'a
It borrows `&'a BitSlice` directly (not via `&self`), so callers can
chain `filter_deferred` and then move `self` into `peek_top_iter`
without lifetime conflicts.
Sparse + plain vector index call sites now do:
let iter = id_tracker
.point_mappings()
.filter_deferred(searcher.iter_not_deleted(), DeferredBehavior::Exclude);
searcher.peek_top_iter(iter, &is_stopped)
leaving `BatchFilteredSearcher` completely ignorant of deferred state.
With deferred handling lifted out, `peek_top_all` itself is now used
only by tests (3 inline `#[cfg(test)] mod tests`, 1 integration test,
1 bench) — gate it under `#[cfg(feature = "testing")]` to match
`new_for_test`. Production code goes through the
`iter_not_deleted` + `filter_deferred` + `peek_top_iter` composition.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: optimize Segment::retrieve and thread user data through read_vectors
Two interlocking changes that together collapse the per-point lookups
and intermediate allocations in `Segment::retrieve` down to one
external-to-internal pass.
## `IdTrackerRead::resolve_external_ids` (new default trait method)
Single-pass translation of a `&[PointIdType]` slice into two parallel
vectors `(Vec<PointIdType>, Vec<PointOffsetType>)`. Folds deferred
filtering (compare offset against the threshold inline — no separate
`point_is_deferred` lookup) and missing-id errors (eager
`PointIdError`) into resolution.
Lives on the trait so the deferred threshold never leaks out of the id
tracker; the parallel-vector shape lets a future batched payload /
vector fetcher consume `&offsets` straight without unzipping. The
`appendable_flag` guard previously in `point_is_deferred` is gone:
non-appendable trackers always carry `deferred_internal_id() == None`
(set only via `MutableIdTracker::open`, guarded by the segment
constructor), so the check was load-bearing nowhere.
## User-data threading through `read_vectors`
`VectorStorageRead::read_vectors` now takes
`IntoIterator<Item = (U, PointOffsetType)>` and yields
`(U, PointOffsetType, CowVector)`. The user-data tag rides alongside
each offset all the way through, so callers can map results back into a
parallel array without keeping a separate `offset → ...` lookup table.
- Default trait impl: one-line per-key loop.
- Dense impl: `unzip()` into parallel `(Vec<U>, Vec<PointOffsetType>)`
in a single pass — same allocation count as before, just U riding
alongside.
- Enum delegations (`VectorStorageEnum`, `VectorStorageReadEnum`)
forward unchanged.
- `for_each_in_batch` and below stay untouched.
`SegmentReadView::vectors_by_offsets<U: Copy>` becomes a lazy filter
chain — no parallel `Vec<(orig_idx, offset)>` allocation. The dead
`SegmentReadView::read_vectors` helper is removed.
## `Segment::retrieve` end-to-end
Per N points / V vectors / payload:
| Operation | Before | After |
|----------------------------|---------------------|-------|
| `id_tracker.internal_id` | N × (1 + V + 1) | N |
| `id_tracker.external_id` | N × V | 0 |
| `point_is_deferred` | N (when applicable) | 0 |
| `offset_to_id` HashMap | N entries | none |
| `Vec` in `vectors_by_offsets` | 1 | 0 |
The vectors stage passes the external id as `read_vectors`'s user
data — the callback gets `id` directly without any index lookup.
The payload stage uses `payload_by_offset` against the already-resolved
offsets. The shape is also batch-friendly: swapping in a future
`IdTrackerRead::batch_internal_id` or `payload_index.batch_get_payload`
needs no changes outside the two call sites.
## Behavioural notes
- Missing-id now errors eagerly inside resolution, instead of in the
vectors stage (`WithVector::Bool(true)` / `Selector`) or payload
stage (`with_payload.enable`). The previous `WithVector::Bool(false)`
+ no-payload path silently inserted an empty record; that is now also
an error. None of the existing callers (search post-processing,
external retrieve API, the deferred-points test on tests/mod.rs:1179)
pass non-existent ids.
- Added a per-payload `check_stopped`; the vectors stage already had
`stop_if` on its iterator chain.
- `vector_by_offset` (the single-element helper) passes `()` as the
no-op user data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: apply rustfmt to optimised retrieve / read_vectors paths
Pre-push hook failure on the previous commit was rustfmt. Same content,
formatted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* do not error out on missing points in retrieve
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Empty commit to open a PR
* Refactor `MultivectorOffsetsStorageMmap` to use `MmapFile` instead of `MmapSlice`
* simpler result types
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* use generic in QuantizedMmapStorage
* rename to QuantizedStorage
* be explicit about S
* rename builder to `QuantizedStorageBuilder`
* rename file to `quantized_storage.rs`
* Add integration test for `match: {except: []}` with integer index
Regression test for https://github.com/qdrant/qdrant/issues/9050
An empty `except` list (NOT IN []) should always match all points that
have the field, both with and without a payload index. Currently the
integer (and keyword) indexed path incorrectly returns zero results
because:
1. serde deserializes `except: []` as `AnyVariants::Strings([])` (first
variant of the untagged enum)
2. The map index filter_impl returns `iter::empty()` for empty
cross-type variant, instead of matching everything
The test covers:
- except: [] without any index (baseline, currently works)
- except: [] with an integer index (currently broken)
- except: [] with a keyword index (currently broken)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix `match: {except: []}` returning zero results with payload index
Fixes https://github.com/qdrant/qdrant/issues/9050
Root cause: `except: []` deserializes as `AnyVariants::Strings([])`
due to the untagged serde enum trying `Strings` first. On an integer
index, the filter_impl matched `Except + Strings(empty)` and returned
`iter::empty()` (zero results). The same issue existed symmetrically
on keyword indexes with `Integers(empty)` and on UUID indexes with
`Integers(empty)`.
The fix: when the cross-type variant reaches the Except branch (e.g.
Strings on an integer index, Integers on a keyword/UUID index), return
`None` unconditionally — regardless of whether the set is empty. This
delegates to the fallback condition checker, which already handles
`except: []` correctly by matching all values.
The `estimate_cardinality_impl` functions had the same bug (returning
`CardinalityEstimation::exact(0)` for the empty cross-type case) and
are fixed the same way.
Affected index types: integer, keyword (str), UUID.
Bool index is NOT affected — it already returns `None` for all
Any/Except conditions.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Expand match-except test to cover all index types and cross-type filters
Extends the regression test for #9050 to comprehensively cover:
- Integer, keyword, and UUID field indexes
- Empty except list (the original bug) for each index type
- Non-empty except list with matching types (normal filtering)
- Cross-type except values (e.g. strings on integer index, integers on
keyword/UUID index) — type mismatch should exclude nothing → all match
- Exclude-all: listing every value should return zero results
- All scenarios tested both with and without a payload index
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: add failing test for values_per_hash drift on duplicate geo point removal
Reproduces the bug where `remove_point` only calls `decrement_hash_value_counts`
once per unique geohash, while `add_many_geo_points` increments it once per value.
When a point has duplicate geo coordinates (same geohash), the counters drift
upward permanently after removal.
Ref: https://github.com/qdrant/qdrant/pull/9033#discussion_r3241154045
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: decrement values_per_hash once per value on geo point removal
`add_many_geo_points` increments `values_per_hash` once per value, but
`remove_point` deduplicated geohashes with a HashSet + `continue` that
also skipped the per-value decrement. A point with duplicate geo
coordinates (same geohash) therefore left the counters drifted upward
permanently after removal.
Move `decrement_hash_value_counts` above the dedup guard so it runs once
per value, matching the increment side. `points_map` and
`points_per_hash` track points, not values, so they stay deduplicated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: read only full text
* fix: linter
* refactor: follow new file structure
* chore: remove storage enums
* fix: linter
* fix: FullTextReadIndex methods
* chore: remove duplicated impls
* fix: format
* refactor: unify FullTextIndex telemetry via trait default method
Add a `get_telemetry_data` default method to `FullTextIndexRead` built
from the existing telemetry methods, so the `ReadOnlyFieldIndex` match
arm is consistent with every other variant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: move full-text lifecycle methods off FullTextIndexRead trait
`populate`, `clear_cache`, `files` and `immutable_files` are lifecycle
concerns, not part of the read surface. Move them from the
`FullTextIndexRead` trait to inherent methods alongside the rest of each
index's lifecycle code (open / wipe / flusher). Also split the
`mutable_text_index` tests into their own `tests.rs`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: restructure full_text_index module layout
Move the `FullTextIndex` enum into `full_text_index/mod.rs` and split its
impls following the inner-module convention: lifecycle (constructors,
builders, ValueIndexer, PayloadFieldIndex) into `lifecycle.rs`, read-path
impls and shared free functions into `read_ops.rs`. Extract the
`FullTextIndexRead` trait into its own `full_text_index_read.rs`.
Drops the inherent `FullTextIndex::get_telemetry_data`, which duplicated
the `FullTextIndexRead::get_telemetry_data` default method.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: read-only counterpart of MutableFullTextIndex
Extract the shared in-memory state of `MutableFullTextIndex` into
`MutableFullTextIndexInner` (inverted index, config, tokenizer), which
implements `FullTextIndexRead`. `MutableFullTextIndex` now wraps it
alongside a writable `Gridstore`.
Add `ReadOnlyAppendableFullTextIndex<S>`, wrapping the same inner state
with a `GridstoreReader` over generic `UniversalRead`. Mirrors the
`MutableMapIndex` / `ReadOnlyAppendableMapIndex` split; loading and
lifecycle land in a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: turn ReadOnlyFullTextIndex into a two-variant enum
Rename the `read_only_text_index` module to `read_only` and reshape
`ReadOnlyFullTextIndex` from a thin `MmapFullTextIndex` wrapper into an
`Appendable` / `Immutable` enum, mirroring `ReadOnlyMapIndex`. Its
`FullTextIndexRead` impl is now a dispatcher forwarding to the active
variant.
Drop the no-op inherent `populate` (mutable, immutable) and
`immutable_files` (mutable) methods that tripped `clippy::unused_self`
and `clippy::unnecessary_wraps` after the earlier trait-to-inherent
move; `FullTextIndex::populate` inlines the in-RAM no-op directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: split numeric index variants into dedicated modules
Move MutableNumericIndex, ImmutableNumericIndex, and MmapNumericIndex
into their own directories, each split into mod.rs (struct definitions),
lifecycle.rs (open/build/wipe/mutations), and read_ops.rs (accessors),
mirroring the map_index layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: drop single-variant Storage enums in mutable/immutable numeric index
Replace `Storage<T>` wrappers around the only backing store with the
store types directly: `Gridstore<Vec<T>>` for `MutableNumericIndex` and
`Box<MmapNumericIndex<T>>` for `ImmutableNumericIndex`. Collapses the
trivial single-arm matches into direct method calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: introduce NumericIndexRead trait
Mirror `MapIndexRead` from the map_index refactor: define a
`NumericIndexRead<T>` trait in `numeric_index/read_ops.rs` and implement
it on each of the three storage variants
(`MutableNumericIndex`, `ImmutableNumericIndex`, `MmapNumericIndex`).
Trait signatures are unified across variants — in-memory variants accept
and ignore the `hw_counter` argument that the mmap-backed variant uses
for IO tracking, and `total_unique_values_count`, `values_range`, and
`orderable_values_range` return `OperationResult` everywhere so the
dispatcher in `NumericIndexInner` can call them generically.
Variant-specific helpers that don't fit the shared shape stay as
inherent methods: `MutableNumericIndex::map()`,
`ImmutableNumericIndex::values_range_size()`, and
`MmapNumericIndex::{values_range_size, is_on_disk}`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: add ReadOnlyAppendableNumericIndex
Counterpart to `MutableNumericIndex`, mirroring `ReadOnlyAppendableMapIndex`
from the map_index refactor. It reuses the shared `InMemoryNumericIndex`
in-memory state but is backed by a `GridstoreReader` over generic
`UniversalRead` instead of a writable `Gridstore`, and implements
`NumericIndexRead` by forwarding to the in-memory index — no mutation
surface.
Loading / lifecycle (constructor, files, populate, clear_cache) will
follow in a separate change; the storage field is held only to pin the
on-disk layout for now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: rename MmapNumericIndex to UniversalNumericIndex, expose UniversalRead param
Mirror `UniversalMapIndex`: the type is now generic over `S: UniversalRead`
with a `MmapFile` default, so the index can be served from any
`UniversalRead` backend (io_uring, disk-cache wrappers, …) rather than
the hard-coded `MmapFile`.
The `NumericIndexRead` impl and read-side helpers are generic over `S`;
`build` / `open` and the other lifecycle methods stay `MmapFile`-only
since they construct mmap-backed storage from a path. The
`NumericIndexInner::Mmap` enum variant keeps its name and uses the
default `S = MmapFile`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: split numeric_index/storage/mod.rs into lifecycle and read_ops
`storage/mod.rs` now holds only the `NumericIndexInner` enum and module
wiring. The variant-dispatch impls are split into sibling modules
matching the layout of the individual storage variants:
- `lifecycle.rs`: construction, persistence, file listing, cache
control, and `remove_point`.
- `read_ops.rs`: read-path forwarding — value lookups, telemetry, RAM
accounting, `is_on_disk`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: implement NumericIndexRead for NumericIndexInner
The enum-level read dispatch was a set of inherent methods scattered
across storage/read_ops.rs and storage/statistics.rs with signatures
that drifted from the variant trait (`values_count` returned `usize`,
`max_values_per_point` vs `get_max_values_per_point`, a hand-rolled
`get_telemetry_data`). Make `NumericIndexInner` implement
`NumericIndexRead` directly so it shares one interface with the three
storage variants.
- All 12 trait methods are forwarded via match dispatch in
storage/read_ops.rs; `values_range` / `orderable_values_range` box
the per-variant iterators.
- `get_histogram`, `get_points_count`, `total_unique_values_count` move
out of statistics.rs into the trait impl; `values_is_empty` and
`get_telemetry_data` now come from the trait defaults.
- `point_ids_by_value` and `is_on_disk` stay as enum-only inherent
helpers (not part of the shared trait).
- Callers updated: `NumericIndex::values_count` unwraps the now
`Option`-returning trait method; `filter` boxes `point_ids_by_value`;
`field_index.rs` and `numeric_field_index.rs` import the trait.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: add ReadOnlyNumericIndexInner
Read-only counterpart to `NumericIndexInner`, mirroring `ReadOnlyMapIndex`
from the map_index refactor. Lives under
`numeric_index/storage/read_only` and selects across the two read-only
storage backends:
- `Appendable(ReadOnlyAppendableNumericIndex<T, S>)` — loaded into RAM
from the appendable Gridstore format.
- `Immutable(UniversalNumericIndex<T, S>)` — served directly from the
immutable stored format.
Implements `NumericIndexRead` by forwarding each method to the active
variant; `values_is_empty` / `get_telemetry_data` come from the trait
defaults.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: rename numeric_index read_ops to numeric_index_read, split mod.rs
Two changes:
- Rename `numeric_index/read_ops.rs` (the `NumericIndexRead` trait
definition) to `numeric_index_read.rs`, freeing the `read_ops` name.
- Split the leftover content of `numeric_index/mod.rs` into sibling
modules, matching the per-variant layout:
- `lifecycle.rs`: the `Encodable` key-format trait + impls and the
`HISTOGRAM_*` construction constants.
- `read_ops.rs`: the `StreamRange` trait and the `Range` →
index-key-bounds conversion.
`mod.rs` now only wires modules and re-exports. `Encodable` and
`StreamRange` keep their public paths via re-export; `tests.rs` gains
explicit imports for the symbols it previously picked up through the
`mod.rs` glob.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: dissolve numeric_index/index.rs into mod, lifecycle, read_ops
`index.rs` only held the `NumericIndex` wrapper and its two impl blocks;
spread them to match the per-module layout used elsewhere:
- `NumericIndex` struct + `NumericIndexIntoInnerValue` trait → `mod.rs`
(type definitions live with the module wiring).
- The inherent `impl NumericIndex` (open / build / cache control /
storage introspection) → `lifecycle.rs`, alongside the `HISTOGRAM_*`
seed constants.
- The `PayloadFieldIndexRead` impl → `read_ops.rs`.
Also move the `Encodable` key-format trait out of `lifecycle.rs` into
its own `encodable.rs`. `mod.rs` keeps re-exporting `Encodable` so its
public path is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add ReadOnlyNumericIndex with NumericIndexRead + PayloadFieldIndexRead
Read-only counterpart to `NumericIndex`, wrapping `ReadOnlyNumericIndexInner`
plus the payload value type parameter `P`. Implements both `NumericIndexRead`
and `PayloadFieldIndexRead` by forwarding to the inner storage-variant enum.
To support `PayloadFieldIndexRead` without duplicating the query logic, the
cardinality/filter/payload-block/condition-checker code is extracted into a
new `query` module of generic free functions over `NumericIndexRead<T>`.
`ReadOnlyNumericIndexInner` implements `PayloadFieldIndexRead` by plugging
into those helpers; `ReadOnlyNumericIndex` delegates to its inner.
The writable `NumericIndexInner` path is left untouched — its existing
variant-specialized `estimate_points` heuristic stays in `storage`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: make query.rs the single source of truth for numeric index queries
The generic `query` helpers and the per-`NumericIndexInner` impls in
`storage/{trait_impls,statistics}.rs` had duplicated cardinality /
filter / payload-block / condition-checker logic. Collapse them onto
the shared `query` helpers:
- `storage/trait_impls.rs`: `PayloadFieldIndexRead for NumericIndexInner`
now forwards each method to `query::*` instead of carrying its own copy.
- `storage/statistics.rs`: deleted — `range_cardinality` and
`estimate_points` were duplicates of the `query` versions.
- `estimate_points` needs a range size; add `values_range_size` to the
`NumericIndexRead` trait with a default that counts `values_range`,
overridden by the `Immutable` / `Mmap` variants with their `O(log n)`
boundary search. The `MutableNumericIndex::map()` accessor (its only
caller was the old `estimate_points`) is removed.
- `values_range_size` takes `hw_counter` and threads it into
`values_range` rather than fabricating a disposable counter.
`tests.rs` calls `query::range_cardinality` directly now that the
inherent method is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: integrate read-only numeric index into ReadOnlyFieldIndex
Wire the four numeric variants (`IntIndex`, `DatetimeIndex`,
`FloatIndex`, `UuidIndex`) into `ReadOnlyFieldIndex`, mirroring
`FieldIndex`:
- `PayloadFieldIndexRead` / `FieldIndexRead` dispatch covers the new
variants — telemetry, value counts, value retrievers, `as_numeric`.
- `ReadOnlyNumericFieldIndex` is the read-only counterpart of
`NumericFieldIndex` (Int/Float order-by erasure over
`ReadOnlyNumericIndexInner`); `as_numeric` returns it for the
Int/Datetime/Float variants (UUIDs aren't numerically order-by-able,
matching `FieldIndex`).
- `ReadOnlyNumericIndex` gains per-`(T, P)` `value_retriever` methods
(in `read_only/value_retriever.rs`) and an `inner()` accessor.
`StreamRange` is now backed by a shared generic `query::stream_range`
helper over `NumericIndexRead`, implemented for both `NumericIndexInner`
and `ReadOnlyNumericIndexInner` — replacing the bespoke `EitherVariant`
dispatch in `storage/trait_impls.rs`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: collapse ReadOnlyNumericIndex value retrievers onto one generic method
The four per-`(T, P)` `value_retriever` methods were identical except
for the per-value `T -> Value` conversion. Extract that conversion into
a `NumericValueToJson` trait (one tiny impl per `(T, P)`) and keep a
single generic `value_retriever` that builds the retriever closure once.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fmt
* refactor: dedup NumericFieldIndex / ReadOnlyNumericFieldIndex
The two enums were structurally identical — same `StreamRange`,
`get_ordering_values`, and `NumericFieldIndexRead` bodies — differing
only in the backing storage type. Collapse them onto one generic
`NumericFieldIndexView<'a, I, F>` with a single set of impls (over
`I: NumericIndexRead<i64> + StreamRange<i64>` and the `f64` counterpart).
`NumericFieldIndex` and `ReadOnlyNumericFieldIndex` are now type aliases
of the generic view, so every existing call site is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
