mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-03 00:20:57 -05:00
5436eb3bde
* refactor: split numeric index variants into dedicated modules Move MutableNumericIndex, ImmutableNumericIndex, and MmapNumericIndex into their own directories, each split into mod.rs (struct definitions), lifecycle.rs (open/build/wipe/mutations), and read_ops.rs (accessors), mirroring the map_index layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: drop single-variant Storage enums in mutable/immutable numeric index Replace `Storage<T>` wrappers around the only backing store with the store types directly: `Gridstore<Vec<T>>` for `MutableNumericIndex` and `Box<MmapNumericIndex<T>>` for `ImmutableNumericIndex`. Collapses the trivial single-arm matches into direct method calls. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: introduce NumericIndexRead trait Mirror `MapIndexRead` from the map_index refactor: define a `NumericIndexRead<T>` trait in `numeric_index/read_ops.rs` and implement it on each of the three storage variants (`MutableNumericIndex`, `ImmutableNumericIndex`, `MmapNumericIndex`). Trait signatures are unified across variants — in-memory variants accept and ignore the `hw_counter` argument that the mmap-backed variant uses for IO tracking, and `total_unique_values_count`, `values_range`, and `orderable_values_range` return `OperationResult` everywhere so the dispatcher in `NumericIndexInner` can call them generically. Variant-specific helpers that don't fit the shared shape stay as inherent methods: `MutableNumericIndex::map()`, `ImmutableNumericIndex::values_range_size()`, and `MmapNumericIndex::{values_range_size, is_on_disk}`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: add ReadOnlyAppendableNumericIndex Counterpart to `MutableNumericIndex`, mirroring `ReadOnlyAppendableMapIndex` from the map_index refactor. It reuses the shared `InMemoryNumericIndex` in-memory state but is backed by a `GridstoreReader` over generic `UniversalRead` instead of a writable `Gridstore`, and implements `NumericIndexRead` by forwarding to the in-memory index — no mutation surface. Loading / lifecycle (constructor, files, populate, clear_cache) will follow in a separate change; the storage field is held only to pin the on-disk layout for now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: rename MmapNumericIndex to UniversalNumericIndex, expose UniversalRead param Mirror `UniversalMapIndex`: the type is now generic over `S: UniversalRead` with a `MmapFile` default, so the index can be served from any `UniversalRead` backend (io_uring, disk-cache wrappers, …) rather than the hard-coded `MmapFile`. The `NumericIndexRead` impl and read-side helpers are generic over `S`; `build` / `open` and the other lifecycle methods stay `MmapFile`-only since they construct mmap-backed storage from a path. The `NumericIndexInner::Mmap` enum variant keeps its name and uses the default `S = MmapFile`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: split numeric_index/storage/mod.rs into lifecycle and read_ops `storage/mod.rs` now holds only the `NumericIndexInner` enum and module wiring. The variant-dispatch impls are split into sibling modules matching the layout of the individual storage variants: - `lifecycle.rs`: construction, persistence, file listing, cache control, and `remove_point`. - `read_ops.rs`: read-path forwarding — value lookups, telemetry, RAM accounting, `is_on_disk`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: implement NumericIndexRead for NumericIndexInner The enum-level read dispatch was a set of inherent methods scattered across storage/read_ops.rs and storage/statistics.rs with signatures that drifted from the variant trait (`values_count` returned `usize`, `max_values_per_point` vs `get_max_values_per_point`, a hand-rolled `get_telemetry_data`). Make `NumericIndexInner` implement `NumericIndexRead` directly so it shares one interface with the three storage variants. - All 12 trait methods are forwarded via match dispatch in storage/read_ops.rs; `values_range` / `orderable_values_range` box the per-variant iterators. - `get_histogram`, `get_points_count`, `total_unique_values_count` move out of statistics.rs into the trait impl; `values_is_empty` and `get_telemetry_data` now come from the trait defaults. - `point_ids_by_value` and `is_on_disk` stay as enum-only inherent helpers (not part of the shared trait). - Callers updated: `NumericIndex::values_count` unwraps the now `Option`-returning trait method; `filter` boxes `point_ids_by_value`; `field_index.rs` and `numeric_field_index.rs` import the trait. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: add ReadOnlyNumericIndexInner Read-only counterpart to `NumericIndexInner`, mirroring `ReadOnlyMapIndex` from the map_index refactor. Lives under `numeric_index/storage/read_only` and selects across the two read-only storage backends: - `Appendable(ReadOnlyAppendableNumericIndex<T, S>)` — loaded into RAM from the appendable Gridstore format. - `Immutable(UniversalNumericIndex<T, S>)` — served directly from the immutable stored format. Implements `NumericIndexRead` by forwarding each method to the active variant; `values_is_empty` / `get_telemetry_data` come from the trait defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: rename numeric_index read_ops to numeric_index_read, split mod.rs Two changes: - Rename `numeric_index/read_ops.rs` (the `NumericIndexRead` trait definition) to `numeric_index_read.rs`, freeing the `read_ops` name. - Split the leftover content of `numeric_index/mod.rs` into sibling modules, matching the per-variant layout: - `lifecycle.rs`: the `Encodable` key-format trait + impls and the `HISTOGRAM_*` construction constants. - `read_ops.rs`: the `StreamRange` trait and the `Range` → index-key-bounds conversion. `mod.rs` now only wires modules and re-exports. `Encodable` and `StreamRange` keep their public paths via re-export; `tests.rs` gains explicit imports for the symbols it previously picked up through the `mod.rs` glob. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: dissolve numeric_index/index.rs into mod, lifecycle, read_ops `index.rs` only held the `NumericIndex` wrapper and its two impl blocks; spread them to match the per-module layout used elsewhere: - `NumericIndex` struct + `NumericIndexIntoInnerValue` trait → `mod.rs` (type definitions live with the module wiring). - The inherent `impl NumericIndex` (open / build / cache control / storage introspection) → `lifecycle.rs`, alongside the `HISTOGRAM_*` seed constants. - The `PayloadFieldIndexRead` impl → `read_ops.rs`. Also move the `Encodable` key-format trait out of `lifecycle.rs` into its own `encodable.rs`. `mod.rs` keeps re-exporting `Encodable` so its public path is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add ReadOnlyNumericIndex with NumericIndexRead + PayloadFieldIndexRead Read-only counterpart to `NumericIndex`, wrapping `ReadOnlyNumericIndexInner` plus the payload value type parameter `P`. Implements both `NumericIndexRead` and `PayloadFieldIndexRead` by forwarding to the inner storage-variant enum. To support `PayloadFieldIndexRead` without duplicating the query logic, the cardinality/filter/payload-block/condition-checker code is extracted into a new `query` module of generic free functions over `NumericIndexRead<T>`. `ReadOnlyNumericIndexInner` implements `PayloadFieldIndexRead` by plugging into those helpers; `ReadOnlyNumericIndex` delegates to its inner. The writable `NumericIndexInner` path is left untouched — its existing variant-specialized `estimate_points` heuristic stays in `storage`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: make query.rs the single source of truth for numeric index queries The generic `query` helpers and the per-`NumericIndexInner` impls in `storage/{trait_impls,statistics}.rs` had duplicated cardinality / filter / payload-block / condition-checker logic. Collapse them onto the shared `query` helpers: - `storage/trait_impls.rs`: `PayloadFieldIndexRead for NumericIndexInner` now forwards each method to `query::*` instead of carrying its own copy. - `storage/statistics.rs`: deleted — `range_cardinality` and `estimate_points` were duplicates of the `query` versions. - `estimate_points` needs a range size; add `values_range_size` to the `NumericIndexRead` trait with a default that counts `values_range`, overridden by the `Immutable` / `Mmap` variants with their `O(log n)` boundary search. The `MutableNumericIndex::map()` accessor (its only caller was the old `estimate_points`) is removed. - `values_range_size` takes `hw_counter` and threads it into `values_range` rather than fabricating a disposable counter. `tests.rs` calls `query::range_cardinality` directly now that the inherent method is gone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: integrate read-only numeric index into ReadOnlyFieldIndex Wire the four numeric variants (`IntIndex`, `DatetimeIndex`, `FloatIndex`, `UuidIndex`) into `ReadOnlyFieldIndex`, mirroring `FieldIndex`: - `PayloadFieldIndexRead` / `FieldIndexRead` dispatch covers the new variants — telemetry, value counts, value retrievers, `as_numeric`. - `ReadOnlyNumericFieldIndex` is the read-only counterpart of `NumericFieldIndex` (Int/Float order-by erasure over `ReadOnlyNumericIndexInner`); `as_numeric` returns it for the Int/Datetime/Float variants (UUIDs aren't numerically order-by-able, matching `FieldIndex`). - `ReadOnlyNumericIndex` gains per-`(T, P)` `value_retriever` methods (in `read_only/value_retriever.rs`) and an `inner()` accessor. `StreamRange` is now backed by a shared generic `query::stream_range` helper over `NumericIndexRead`, implemented for both `NumericIndexInner` and `ReadOnlyNumericIndexInner` — replacing the bespoke `EitherVariant` dispatch in `storage/trait_impls.rs`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: collapse ReadOnlyNumericIndex value retrievers onto one generic method The four per-`(T, P)` `value_retriever` methods were identical except for the per-value `T -> Value` conversion. Extract that conversion into a `NumericValueToJson` trait (one tiny impl per `(T, P)`) and keep a single generic `value_retriever` that builds the retriever closure once. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fmt * refactor: dedup NumericFieldIndex / ReadOnlyNumericFieldIndex The two enums were structurally identical — same `StreamRange`, `get_ordering_values`, and `NumericFieldIndexRead` bodies — differing only in the backing storage type. Collapse them onto one generic `NumericFieldIndexView<'a, I, F>` with a single set of impls (over `I: NumericIndexRead<i64> + StreamRange<i64>` and the `f64` counterpart). `NumericFieldIndex` and `ReadOnlyNumericFieldIndex` are now type aliases of the generic view, so every existing call site is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>