* Fix phrase matching crossing string-array element boundaries
When indexing a string array like ["quick", "brown"], the index
concatenated tokens from all elements into a single flat document.
A phrase query for "quick brown" found the tokens consecutively,
producing false matches.
Insert a sentinel token ("\x00") between array elements during
indexing. It is registered as a normal vocab token that occupies a
position in the document, so phrase windows cannot span it. No
tokenizer ever produces this string, so it never appears in a query.
Closes#8937
Co-authored-by: Cursor <cursoragent@cursor.com>
* Strip null bytes from tokenizer output
Ensure no tokenizer can produce tokens containing '\0', which is
reserved as the array-boundary sentinel for phrase matching.
Covers all tokenizer paths:
- process_token_cow (used by Word, Whitespace, Prefix, Multilingual)
- Japanese tokenizer (has its own inline processing)
Co-authored-by: Cursor <cursoragent@cursor.com>
* Revert "Strip null bytes from tokenizer output"
This reverts commit fe6ea7a955.
* filter sentinel token from query
* Preallocate vector
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Tim Visée <tim+github@visee.me>
* genericize ChunkedVectors.status, remove `Sized` bound
* check exists with `UniversalReadFileOps`
* Inline UioChunkedVectors bound, drop the alias (#8952)
The empty trait + blanket impl was a stable-Rust trait-alias workaround
that hid a fairly short bound (UniversalWrite<T> + UniversalWrite<Status>
+ Send + 'static) at the cost of an indirection readers had to mentally
unwind. Spelling it out at the three sites that need it is shorter overall
and immediately tells the reader what is required.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add `EncodedStorage::for_each_in_batch` method
* Implement `for_each_in_batch` method for `QuantizedChunkedMmapStorage`
* Add `EncodedVectors::for_each_in_batch` method
* Add `EncodedVectors::score` method
* Implement `score_stored_batch` for `QuantizedQueryScorer` and `Quanti…
* Remove `TElement` and `TMetric` type parameters from `QuantizedMultiQ…
* Use `QuantizedMultiQueryScorer` when building `raw_internal_scorer`...
`special_check_condition` in `FieldIndex` did not handle
`Match::TextAny`, so nested-condition evaluation fell through to the
`ValueChecker` fallback which uses naive `String::contains` instead of
proper full-text tokenization. This caused "good" to match "goodness"
inside nested filters.
Add `check_payload_match_any` to `FullTextIndex` and wire it into
`special_check_condition`. Also replace the catch-all `_ => None` with
explicit match arms for all `Match` variants.
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* propagate to BufferedDynamicFlags
* use duplicate for tests
* propagate to Bitvec/Roaring flags
* propagate to RoaringFlags
* fixup! propagate to BufferedDynamicFlags
* propagate to BitvecFlags
Extract the read-side API into ChunkedVectorsRead<T, S: UniversalRead<T>>
and rebuild the existing ChunkedVectors<T, S: UniversalWrite<T>> on top of
it via composition + Deref. Lets read-only consumers use the storage
without pulling in UniversalWrite, and avoids method duplication.
Reorganizes the file into a chunked_vectors/ module: config (constants,
ChunkedVectorsConfig, Status), chunks (read_chunks/create_chunk helpers),
read (ChunkedVectorsRead), write (ChunkedVectors wrapper). read_chunks
now takes a writeable flag so both open paths share it.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(segment): immutable map index skips values with no live points on load
When MmapMapIndex::open ORs the id-tracker's runtime deletion bitvec
into the on-disk one at open time, ImmutableMapIndex::open_mmap could
insert a zero-count entry into value_to_points for any value whose live
points were all deleted, then immediately trip its own post-build sort
assert. Skip such values, mirroring the runtime invariant maintained by
remove_idx_from_value_list which already removes entries when their
count drops to zero.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(segment): file-system immutability test for payload indices
Builds an immutable segment with all 8 PayloadSchemaType variants
indexed, snapshots every byte under payload_index/, then asserts
byte-for-byte equality plus per-field query correctness after
delete_point, flush, drop+reload, and a second deletes+flush on the
reloaded segment. Each query exercises a different read path of an
immutable index variant: map exact-match (keyword/uuid/integer),
numeric range (float), datetime range, geo bounding box, full-text
token match, bool match. Reproduces the regression fixed in the
previous commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the layout of mutable_id_tracker: storage helpers for the
mappings, versions, and deleted bitslice files live in their own
submodules, leaving mod.rs focused on the ImmutableIdTracker type and
its trait impls.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a read-only payload storage that wraps `GridstoreReader<Payload>`
and implements `PayloadStorageRead`. Also rename the parameter on
`PayloadStorageRead::get`/`get_sequential` from `point_id` to `point_offset`
for consistency with the underlying storage API.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(index): extend VectorIndexRead and PayloadIndexRead for telemetry/info
Two trait extensions (no defaults — every implementor must opt in):
* \`VectorIndexRead::is_index\` — distinguishes a real index from a plain
full-scan one. Used by reporting code. Moved out of inherent
\`VectorIndexEnum::is_index\` into the trait. Explicit impls on Plain
(false), HNSW (true), Sparse* (true).
* \`PayloadIndexRead::get_telemetry_data\` — per-field-index telemetry.
Moved out of inherent \`StructPayloadIndex::get_telemetry_data\` into
the trait impl. \`PlainPayloadIndex\` returns an empty Vec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(segment): migrate info/size_info/telemetry to SegmentReadView
Step 11 of the SegmentReadView migration — the final logical step.
New \`read_view/info.rs\` exposes builder methods:
* \`build_size_info(uuid, segment_type, is_appendable)\`
* \`build_info(uuid, segment_type, is_appendable)\` — same plus
\`index_schema\`
* \`build_telemetry(uuid, segment_type, is_appendable, config, detail)\`
The trivial segment-level fields (\`uuid\`, \`segment_type\`,
\`is_appendable\`, \`config\`) are passed in by the caller — they stay
direct on each segment-type rather than going through the view.
Everything else (vector data breakdown, payloads size, deferred
counts, vector-index telemetry, payload-field telemetry, …) is
computed once inside the view through the read traits.
\`Segment::size_info\`, \`info\`, and \`get_telemetry_data\` collapse to
one-line \`with_view\` delegators that pass in the trivial fields.
Cleanup: \`Segment::deferred_deleted_count\` is now unused (the view
has its own equivalent helper); deleted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Leftovers from moving \`StructPayloadIndex::formula_scorer\` out of this
file. Caught by clippy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
\`FormulaScorer<'a>\` is already self-contained — it holds the parsed
formula, prefetch scores, retrievers and condition checkers, all owned
or borrowed independently of any payload index. Wrapping it in a trait
adds nothing (a future ReadOnlySegment can construct one too).
* Remove the \`FormulaScorerRead\` trait. \`score(point_id)\` goes back
to being an inherent method on \`FormulaScorer\`.
* \`PayloadIndexRead::formula_scorer\` returns
\`OperationResult<FormulaScorer<'q>>\` directly.
* \`StructPayloadIndex\` and \`PlainPayloadIndex\` updated to match.
* \`PlainPayloadIndex\` no longer needs the turbofish placeholder —
\`Err(...)\` is enough.
* View's \`formula_rescore.rs\` drops the \`FormulaScorerRead\` import.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 10 of the SegmentReadView migration.
Move \`segment/formula_rescore.rs\` → \`read_view/formula_rescore.rs\`:
* \`do_rescore_with_formula\` (private helper).
* \`rescore_with_formula\` (\`ReadSegmentEntry\` orchestrator).
Both now use the trait-method \`PayloadIndexRead::formula_scorer\`
(added in the prior commit) and \`IdTrackerRead::internal_id\` instead
of inherent calls.
\`Segment::rescore_with_formula\` collapses to a single
\`with_view(|v| v.rescore_with_formula(...))\` delegator. The legacy
\`segment/formula_rescore.rs\` is deleted; \`mod formula_rescore;\`
removed from \`segment/mod.rs\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A read-only payload index implementation will have its own concrete
formula scorer type, so PayloadIndexRead can't return the appendable
\`FormulaScorer<'a>\` directly.
* New \`FormulaScorerRead\` trait next to \`FormulaScorer\` exposing only
what the rescore code path consumes (\`score(point_id)\`). Implemented
for \`FormulaScorer<'_>\` by moving its inherent \`score\` into the
trait impl.
* \`PayloadIndexRead::formula_scorer\` returns
\`OperationResult<impl FormulaScorerRead + 'q>\` (RPITIT).
* The inherent \`StructPayloadIndex::formula_scorer\` (which lived in
\`formula_scorer.rs\`) is moved into the trait impl block in
\`struct_payload_index.rs\`, with the body delegating to a new
\`FormulaScorer::new\` constructor (fields stay private).
* \`PlainPayloadIndex\` always returns
\`Err::<FormulaScorer<'q>, _>(...)\` — formula scoring is not
supported there. The turbofish supplies the placeholder type tag.
* Re-export \`FormulaScorer\` and \`FormulaScorerRead\` from
\`rescore_formula::mod\`. \`retrievers_map\` bumped from \`pub(super)\`
to \`pub(crate)\` so the trait impl can call it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A new index implementation should never silently skip IDF wiring. Make
\`VectorIndexRead::fill_idf_statistics\` a required method (no default)
and provide explicit no-op impls for the dense indexes:
* \`PlainVectorIndex\` — no-op.
* \`HNSWIndex\` — no-op.
\`SparseVectorIndex\` keeps the real implementation: moved from a
sibling \`pub fn\` (inherent) into the \`VectorIndexRead\` trait impl
block, so there is now exactly one definition.
\`VectorIndexEnum\` already overrides per-variant from the previous
commit; nothing changes there.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 9 of the SegmentReadView migration.
New \`read_view/search.rs\` hosts:
* \`retrieve\` — full body (deferred filtering, vector enrichment, payload
attachment).
* \`process_search_result\` — converts internal \`ScoredPointOffset\`s into
user-facing \`ScoredPoint\`s; calls the new \`view.retrieve\`.
* \`search_batch\` — \`ReadSegmentEntry\` orchestrator.
* \`fill_query_context\` — uses the trait \`VectorIndexRead::fill_idf_statistics\`
(added in the prior commit) and the existing \`indexed_vector_count\`.
The \`Segment\`-side trait method bodies for \`search_batch\`, \`retrieve\`,
and \`fill_query_context\` collapse to single \`with_view\` delegators.
\`rescore_with_formula\` (whose body still uses Segment-side
\`do_rescore_with_formula\`, migrated in step 11) routes its
\`process_search_result\` call through \`with_view\`.
\`segment/search.rs\` is trimmed to just the
\`#[cfg(feature = "testing")] pub fn search\` helper, which calls
\`search_batch\` (now a \`with_view\` delegator) — no migration needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds \`fill_idf_statistics\` to the \`VectorIndexRead\` trait with a
no-op default (only sparse-vector indexes track IDF). \`VectorIndexEnum\`
overrides it with the per-variant dispatch that previously lived as an
inherent method.
The inherent \`indexed_vectors\` is removed: it returned the same value
as the trait method \`indexed_vector_count\` for every variant
(sparse's \`indexed_vector_count\` is itself
\`inverted_index.vector_count()\`). The remaining caller in
\`fill_query_context\` will use \`indexed_vector_count\` after the next
commit moves it to the view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The introduced \`FacetIndexRead\` trait was almost identical to the
existing \`FacetIndex\` trait — the only difference was the augmented
\`for_each_value\` signature on \`FacetIndexEnum\` that took
\`hw_counter\` and \`deferred_internal_id\` to skip values whose only
points are deferred.
* Move that "skip deferred" logic onto \`FacetIndex\` itself as a default
method \`for_each_visible_value\` — implemented in terms of the
existing \`for_each_value\` and \`for_each_value_map\`. All
\`FacetIndex\` impls (MapIndex, BoolIndex, FacetIndexEnum) get it for
free.
* Make \`FacetIndexEnum\` impl \`FacetIndex\` directly (delegating to the
inner index's \`FacetIndex\` impl). The augmented inherent
\`for_each_value\` is gone.
* \`PayloadIndexRead::facet_index_for\` now returns
\`Option<impl FacetIndex + '_>\` (no separate \`FacetIndexRead\` trait).
* View facet code uses \`for_each_visible_value\` instead of the old
augmented \`for_each_value\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 8 of the SegmentReadView migration.
Move \`segment/facet.rs\` → \`read_view/facet.rs\`:
* \`approximate_facet\`
* \`facet_values\`
Both now go through the trait-method \`facet_index_for\` (added in the
prior commit) and the existing trait-method \`filter_context\` instead
of the previous inherent \`get_facet_index\` and
\`struct_filtered_context\` calls.
\`Segment::unique_values\` and \`Segment::facet\` (the \`ReadSegmentEntry\`
trait orchestrators) collapse to single \`with_view\` delegators. The
legacy \`segment/facet.rs\` is deleted; \`mod facet;\` removed from
\`segment/mod.rs\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same shape as the numeric_index_for change in the previous PR: a
read-only segment will have its own per-key facet-index type, so
PayloadIndexRead can't return the concrete \`FacetIndexEnum<'a>\`.
* New \`FacetIndexRead\` trait in \`field_index/facet_index.rs\` mirrors
the inherent methods of \`FacetIndexEnum\` (\`for_points_values\`,
\`for_each_value\`, \`for_each_value_map\`, \`for_each_count_per_value\`).
* Implemented for \`FacetIndexEnum<'a>\` by delegating to the inherent
methods.
* \`PayloadIndexRead::facet_index_for(key) -> Option<impl FacetIndexRead + '_>\`
added (RPITIT, no boxing). Each impl picks its own concrete return
type:
- \`StructPayloadIndex\` returns \`Option<FacetIndexEnum<'_>>\`.
- \`PlainPayloadIndex\` always returns \`None\`; the explicit
\`None::<FacetIndexEnum<'_>>\` turbofish supplies a placeholder
type tag.
* \`FacetIndexRead\` re-exported from \`field_index/mod.rs\` (the
\`facet_index\` submodule itself stays \`pub(super)\`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 7 of the SegmentReadView migration.
Move \`segment/sampling.rs\` → \`read_view/sampling.rs\`:
* \`read_by_random_id\`
* \`filtered_read_by_index_shuffled\`
* \`filtered_read_by_random_stream\`
* \`read_random_filtered\` orchestrator
\`Segment::read_random_filtered\` collapses to a single
\`with_view(|v| v.read_random_filtered(...))\` delegator. The legacy
\`segment/sampling.rs\` is deleted; \`mod sampling;\` removed from
\`segment/mod.rs\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the bespoke 4-arm \`NumericRangeIter\` enum introduced in the prior
commit. The existing \`common::either_variant::EitherVariant\` is the
same shape and already has \`Iterator\` plus all the standard adapter
specializations.
Adds a \`DoubleEndedIterator\` impl to \`EitherVariant\` (\`next_back\`,
\`nth_back\`, \`rfold\`, \`rfind\`) so it can be used in this position.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both \`StreamRange<T>::stream_range\` and the new
\`NumericFieldIndexRead\` trait now return \`impl Iterator\` /
\`impl DoubleEndedIterator\` directly. No more boxed trait-object
allocations in the order-by hot path.
Mechanism:
* \`NumericIndexInner::stream_range\` has 4-way branching
(empty / mutable / immutable / mmap), all with different concrete
iterator types. Unified via a new 4-arm \`NumericRangeIter\` enum that
delegates \`Iterator\` and \`DoubleEndedIterator\` to its variants.
* \`NumericFieldIndex::stream_range\` and \`get_ordering_values\` have
2-way branching (Int / Float). Unified via \`itertools::Either\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A read-only payload index implementation will have its own concrete
numeric-index type that doesn't fit the appendable
\`NumericFieldIndex<'a>\` enum. So PayloadIndexRead can't return that
concrete type directly.
Introduce \`NumericFieldIndexRead\` trait exposing only what ordered
reads consume (\`get_ordering_values\`, \`stream_range\`). Implement it
for the existing \`NumericFieldIndex<'a>\` (delegates to the inherent
methods + \`StreamRange\`).
\`PayloadIndexRead::numeric_index_for\` now returns
\`Option<impl NumericFieldIndexRead + '_>\` (RPITIT). Each
implementation picks its own concrete return type:
- \`StructPayloadIndex\` returns \`Option<NumericFieldIndex<'_>>\`.
- \`PlainPayloadIndex\` always returns \`None\` (it has no field
indexes); the explicit \`None::<NumericFieldIndex<'_>>\` turbofish
just supplies a placeholder type tag — no value is constructed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 5 leftover: with PayloadIndexRead.iter_filtered_points now on the
trait, filtered_read_by_index can move to the view alongside the other
three scroll helpers.
* `read_view/scroll.rs` gains `filtered_read_by_index` and the
`read_filtered` orchestrator.
* `segment/scroll.rs` is deleted entirely; `mod scroll;` removed from
`segment/mod.rs`.
* `Segment::read_filtered` collapses to a single
`with_view(|v| v.read_filtered(...))` delegator.
* `scroll_filtering_test.rs` integration test calls
`filtered_read_by_index` via `with_view`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 6 of the SegmentReadView migration.
Move `lib/segment/src/segment/order_by.rs` into
`read_view/order_by.rs`:
* `filtered_read_by_index_ordered` — uses
`payload_index.iter_filtered_points` (now on trait, see prior commit)
and `numeric_index_for` (also now on trait).
* `filtered_read_by_value_stream` — uses `numeric_index_for` and
`filter_context` (already on trait).
* `read_ordered_filtered` — `ReadSegmentEntry` orchestrator, also moved
to the view.
`Segment::read_ordered_filtered` collapses to a single
`with_view(|v| v.read_ordered_filtered(...))` delegator. The legacy
`segment/order_by.rs` file is deleted entirely; `mod order_by;` removed
from `segment/mod.rs`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Prerequisite for migrating order_by, sampling, facet, and scroll's
`filtered_read_by_index` to `SegmentReadView`. Both methods existed only
as inherent methods / public field access on `StructPayloadIndex`; the
view holds a generic `&TPI: PayloadIndexRead` reference and so can
only call trait methods.
* `iter_filtered_points` is now a trait method, generic over
`I: IdTrackerRead` for the id_tracker parameter and using `impl Iterator`
in return position (RPITIT). Each impl keeps its own zero-cost concrete
iterator chain — no boxed dyn-iterator allocation. The trait becomes
non-object-safe, which is fine because nothing currently uses
`dyn PayloadIndexRead`.
* `numeric_index_for(key) -> Option<NumericFieldIndex<'_>>` exposes the
per-key numeric-index lookup that ordered reads rely on, replacing
direct access to `StructPayloadIndex::field_indexes`.
* Implementations on `StructPayloadIndex` (real) and `PlainPayloadIndex`
(the latter has no numeric indexes, so `numeric_index_for` returns
`None`).
* Existing internal callers (hnsw, scroll, sampling, order_by, facet)
now pass `&*id_tracker` to fully deref `AtomicRef<IdTrackerEnum>` to
`&IdTrackerEnum` for the generic-param inference.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 5 of the SegmentReadView migration.
New `read_view/scroll.rs` module hosts:
* `should_pre_filter` — payload-index cardinality estimation,
used by all three scroll-shaped trait methods (read_filtered,
read_ordered_filtered, read_random_filtered).
* `read_by_id_stream` — streamed enumeration of visible points.
* `filtered_read_by_id_stream` — streamed enumeration with a
payload-filter context applied.
`Segment`-side `read_filtered` collapses to a `with_view` orchestrator
(except for the `filtered_read_by_index` branch, see below).
`read_ordered_filtered` and `read_random_filtered` now route their
`should_pre_filter` calls through `with_view` (their own bodies migrate
in steps 6 and 7).
`filtered_read_by_index` stays on `Segment` for now: it depends on
`StructPayloadIndex::iter_filtered_points`, which is an inherent method
that takes the concrete `&IdTrackerEnum` and returns `impl Iterator`.
Migrating it cleanly requires extending `PayloadIndexRead` with a
trait-object-friendly version of that method, which is the same
prerequisite Steps 6/7/8 will need (sampling, order_by, facet all use
`iter_filtered_points`). I will do that as a focused pre-step before
Step 6.
`deferred_internal_id` / `deferred_deleted_count` view helpers bumped
from private to `pub(super)` so the new `scroll` module can call them.
`scroll_filtering_test.rs` integration test updated to call
`filtered_read_by_id_stream` via `with_view`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 4 of the SegmentReadView migration.
New `read_view/payload.rs` module hosts:
* `payload_by_offset` (was `pub(super)` on Segment in `segment_ops.rs`)
* `payload` (`ReadSegmentEntry` trait method, real composition:
`lookup_internal_id` + `payload_by_offset`)
* `estimate_point_count` (`ReadSegmentEntry` trait method, composes
filter handling, payload-index cardinality and deferred-point
adjustment)
The corresponding Segment-side trait methods become `with_view`
delegators, and `Segment::payload_by_offset` is deleted.
`Segment::lookup_internal_id` (a transitional `with_view` delegator
introduced in step 1) is now unused after `vector` and `payload`
moved off of it; deleted as well. Tests that called it now go
through `with_view`.
`get_indexed_fields` stays direct on Segment — single-line
`payload_index.borrow().indexed_fields()`, no shared logic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 3 of the SegmentReadView migration.
Move the vector-access logic onto the view in a new
`read_view/vectors.rs` module:
* `vector_by_offset`, `vectors_by_offsets` (was `pub(super)` on Segment)
* `read_vectors` (was `pub(crate)` on Segment in the now-deleted
`segment/vectors.rs`)
* `vector` and `available_vectors_size_in_bytes` (`ReadSegmentEntry`
trait methods)
`Segment::vector` and `Segment::available_vectors_size_in_bytes`
collapse to `with_view` delegators. The `retrieve` trait method now
calls `view.read_vectors(...)` inside per-name `with_view` closures.
Removed entirely:
* `segment/vectors.rs` (its only function moved to the view)
* `Segment::vector_by_offset` and `Segment::vectors_by_offsets`
in `segment_ops.rs`
Kept on Segment:
* `vector_names` — trivial `vector_data.keys().cloned().collect()`,
no shared logic to factor out.
* `all_vectors` — invariant `NamedVectors<'_>` lifetime fights the
view borrow, and the body is a 6-line iteration over `self.vector`
(which is itself now a view delegator). ReadOnlySegment will write
the same trivial loop.
Tests in `segment/tests/mod.rs` that exercised
`segment.vector_by_offset(...)` now go through
`segment.with_view(|v| v.vector_by_offset(...))`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the six deferred-point methods (deferred_internal_id /
deferred_deleted_count helpers + deferred_point_count, has_deferred_points,
point_is_deferred, deferred_point_ids, available_point_count_without_deferred)
out of read_view/segment_ops.rs into their own read_view/deferred.rs module.
Keeps each read_view file focused on a single concern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 2 of the SegmentReadView migration.
Move the five composition-style deferred-point methods onto the view:
`deferred_point_count`, `has_deferred_points`, `point_is_deferred`,
`deferred_point_ids`, `available_point_count_without_deferred`. Each
combines the deferred-point status with id-tracker reads, so they are
real shared logic — not trivial getters.
The Segment-side trait method bodies in `entry.rs` collapse to
`self.with_view(|v| v.foo(...))` delegators.
`deferred_internal_id` and `deferred_deleted_count` are one-line
field accessors on `DeferredPointStatus`, so they stay direct on
`Segment`. The view has its own private helpers with the same
one-liner shape — duplicated only at the access-path level.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 1 of the SegmentReadView migration.
* `SegmentReadView` is expanded to its final field set, parameterised over
four read-only traits (`IdTrackerRead`, `PayloadIndexRead`,
`PayloadStorageRead`, `VectorDataRead`). A `SegmentReadViewFor<'s>` type
alias hides the verbosity at call sites for `Segment`. Steps 2-11 only
add `impl` methods; the struct shape is now frozen.
* `Segment::with_view` borrows the three `AtomicRefCell`-wrapped storages
up front and populates every view field; closure bound relaxed
`Fn` -> `FnOnce`.
* `lookup_internal_id` (Option -> PointIdError conversion) and
`read_range` (iterator + take_while) are migrated to the view. The
previous Segment-side helpers become thin `with_view` delegators where
external callers still rely on them.
* Trivial id-tracker getters (`has_point`, `is_empty`,
`available_point_count`, `deleted_point_count`, `total_point_count`)
stay implemented directly on `Segment` -- one-liners that
`ReadOnlySegment` will reimplement just as cheaply, no shared logic to
factor out.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
