* Add `FlagsMode::from_feature_flags`, the mode for newly created flags
Compact in serverless-compatible deployments, dynamic otherwise. Only
creation consults it; opening existing flags detects their mode from
disk.
* Support the compact mode in the read-only flags types
Add `ReadOnlyFlags`, the mode-dispatching union of the two read-only
counterparts, serving the shared `RoaringFlagsRead` surface. Teach
`InMemoryBitvecFlags` to detect the mode it opens; its compact
`reload_appended` decodes the whole (small) file, as the format has no
random access.
* Create flags through mode selection in storages and indexes
Vector storage deleted flags and the bool/null indexes now open through
`open_or_create` with the mode from the feature flags: serverless
deployments create compact flags, dedicated ones keep creating dynamic
flags, and existing flags are opened in their detected mode either way.
* Read flags in either mode in the read-only bool and null indexes
`ReadOnlyFlags` shares the `RoaringFlagsRead` surface and the lifecycle
signatures of the roaring type it replaces, so the swap is a type
rename.
* Add TODO to not lock bitmask structure during flush
* `MutableStoredBitmask::save` returns the number of bytes written
Zero when the skip-clean save wrote nothing. Lets wrappers charge the
actual write to a hardware counter.
* Refuse to open compact flags in a dynamic-mode directory
Creating the compact file next to dynamic files would leave a directory
of both modes behind, which every later open rejects — refuse up front
instead. Both production callers already rule the case out through
`FlagsMode::detect`, so this only removes a foot-gun for future callers.
The open-or-eagerly-create logic moves into
`open_or_create_compact_mask`, shared with the update-only writer next.
* Rewrite `UpdateOnlyStoredFlags` onto the compact bitmask
The update-only flags writer now writes the compact mode — a single
roaring-encoded `compact_flags.dat` through `MutableStoredBitmask` —
instead of rewriting the whole padded dynamic file pair every batch. A
flush with no effective changes now writes nothing at all, where the old
writer rewrote the full mask on any `set`.
This also fixes opening serverless-created segments: the old open
eagerly wrote a `status.dat` into directories the writable side had
created in the compact mode, leaving files of both modes behind and
poisoning the directory for every later open.
A directory already holding dynamic-mode flags is refused loudly rather
than kept current or migrated; rebuild the segment to migrate its flags.
Migration may come later.
Drops the now-dead `InMemoryBitvecFlags::into_bitvec` and
`DynamicFlagsStatus::new`, and demotes `file_size_for` to private.
* Run edge tests with serverless feature flags
The edge fixtures ran with default feature flags, building leader shards
with dynamic-mode flags — a configuration edge never serves in
production, and one the update-only flags writer now refuses. It also
hid that the writer poisoned compact directories: no test exercised
update-only writes over a serverless-created shard.
Feature flags are process-global and first-init-wins, so every fixture
in the binary initializes the same serverless set; the manifest test
folds into it, since serverless implies `write_segment_manifest`.
* Don't use sequencial mode for one shot reads
* Add `CompactStoredFlags`, segment wrapper over the mutable bitmask
RAM-resident flags with a Flusher (skips the write when clean, cancels after drop) and files lister, backed by one compact stored-bitmask file rewritten whole on flush. Not integrated yet.
* Add `FlagsMode`, detecting the storage mode of a flags directory
`Dynamic` is the existing mmap stack for dedicated deployments, `Compact`
the compact stored-bitmask file for serverless ones; detection probes
which files are present. Also add the clippy allow the compact flags
tests were missing.
* Support the compact storage mode in `BitvecFlags` and `RoaringFlags`
The wrappers keep their in-memory read state in both modes; the new
`FlagsStorage` dispatches the write side between `BufferedDynamicFlags`
and `CompactStoredFlags`. `open_or_create` opens existing flags in their
detected mode and only applies `mode_if_create` to fresh ones — existing
call sites keep constructing the dynamic stack through `new`.
* Add `ReadOnlyCompactFlags`, read-only counterpart of compact flags
Bound to `UniversalRead`: opens on the bitmask header alone,
materializes the bitmap lazily on first query, and never creates a
missing file. Implements `RoaringFlagsRead` for the shared query
surface; `live_reload` reopens a fresh handle, as flushes replace the
file whole but cached handles keep serving the bytes they were opened
on. Not integrated yet.
* Skip compact live-reload tests on Windows, which forbids the rename
Both tests replace the compact flags file behind a reader whose disk
cache keeps the "remote" file mapped. On Unix the rename-over succeeds
and the mapping serves the old inode — the staleness under test — but
Windows forbids renaming over a mapped file, failing the writer's flush
with access denied. A limitation of the local-mmap remote stand-in, not
of the reload logic, which stays covered on the other targets.
* Don't check legacy flag file
* fix(shard): flush CoW destinations before the payload-index pre-build flush
create_field_index force-flushes each segment before building an index on it
(flush-before-build, #9767), one segment at a time, outside flush_all's
all-segment lock capture and copy-on-write dependency ordering. That flush
durably advances a CoW source past the delete halves of its pending moves.
The appendable-first iteration order usually flushes the destination before
the source, but not always: a destination proxy-wrapped by a running
optimization is classified non-appendable and can skip its flush entirely
through the already_indexed short-circuit (the proxy reports the field as
present), and a move landing mid-pass is ordered behind nothing. Once the
source flushes, the move's WAL entry stops being replayable: the pre-image
is durably deleted while the only current copy sits in the unflushed
destination, and a graceful close then loses the point.
This is the root cause of the nightly model-testing reload divergence
(#10095), traced end-to-end in CI runs 31583878492 and 31583871346: cow move
op 5197 into a freshly proxied destination, index op ~5252 flushing every
source past it while skipping the proxy, destination reloading at 5181,
replay declining with 'No point with id'.
The fix mirrors flush_all's invariant at the only per-segment flush site:
before flushing a segment, flush the destinations of its pending
flush_dependency edges (one hop suffices, destinations are appendable and
never CoW sources). Destination guards are taken before the flush lock to
keep the documented [segment locks -> flush lock] ordering.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(shard): regression test for the CoW-destination flush in create_field_index
Reproduces the #10095 loss shape deterministically: a pending copy-on-write
move out of a non-appendable source, a destination whose own pre-build flush
is skipped by the already_indexed short-circuit, then a holder-wide
create_field_index. Verified failing with the dependency-aware flush
neutralized (destination stays behind the move while the source flushes past
it) and passing with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(shard): move the CoW-aware single-segment flush into SegmentHolder
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ensure_appendable_segment` used a shard-root `payload_index.json` that
nothing in edge ever wrote, so a shard loaded with only immutable
segments got a bare appendable segment and the appendable chain stayed
unindexed until a merge happened to include an indexed segment.
Build the segment directly and seed it with the union of
`get_indexed_fields()` over the loaded segments, the same reconciliation
the optimizer performs for its CoW segment. Drops the shard-root file
from edge.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
limits_only_apply_while_the_quota_is_enabled assumed the system temp dir
is on a filesystem at least 1% full, the smallest configurable disk
limit. On a tmpfs /tmp the usage floors to 0% and the limit never trips.
Measure the tempdir's filesystem first and fall back to a tempdir in the
crate directory when it is under 1% used.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Extract stored bitmask encoding into `bitmask_file_bytes`
Also single-source the u32 position-space bound as `MAX_LOGICAL_LEN`.
* Add `MutableStoredBitmask`, collecting bitmask changes in RAM
Materializes via the existing reader without keeping the file handle open, tracks diverged positions, and atomically rewrites the whole file on save - skipping the write when nothing changed.
* Rename payload to bits
* Use changed boolean
* Remove now obsolete test
* Borrow the bitmap in bitmask encoding via Cow, avoiding a clone on save
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The on-disk numeric index ended check_values_any with .unwrap_or(false),
so a real mmap read error was downgraded to "point does not match" and a
filtered search/scroll could silently return an incomplete result set.
Promote NumericIndexRead::check_values_any to OperationResult<bool> and
propagate the error through the dispatcher and the range condition checker,
which already returns OperationResult<bool>. Resolves the two FIXMEs in
numeric_index_read.rs and storage/read_ops.rs.
* [UpdateOnly] Honor upsert update_mode in the batch writer
The writer rejected every `UpsertPointsConditional`. Accept the ones whose
condition is empty — `insert_only` and `update_only` — since existence is
the whole gate they need, and locating a batch's points already answers it.
The gate is evaluated per mutation at its position in the fold, so an
`insert_only` upsert sees a point an earlier operation of the same batch
created, matching a leader that resolves each operation only after the ones
before it were applied. A conditional upsert may therefore not discard the
mutations it follows.
Rejecting an upsert also means never reading the point it would have
overwritten: `needs_stored_point` asks whether the first mutation that
applies to an existing point discards it, so an `insert_only` batch pays
nothing for the ids that are already taken.
A conditional upsert carrying a real filter is still rejected — evaluating
one needs payload indexes the writer never fetches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] State contracts in the update-mode docstrings
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] Trim the update-mode diff
Drop `--update-mode` from edge-shard-update: the modes are covered by unit
and end-to-end tests, and the flag cost a wrapper enum, a conversion and a
parameter threaded through both run paths. The tool still reports rejected
points, which the exhaustive match requires.
Inline `always_applies` into its one caller, drop the two test-batch
wrappers over `conditional_batch`, drop the `update_only` fold test whose
truth table two other tests already assert, and shorten two over-long
comment blocks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Unary inverse hyperbolic cosine, parallel to sqrt/ln/exp/log10, in REST,
gRPC, and edge (FFI + Python) interfaces. Inputs below 1 produce the same
NonFiniteNumber error as an invalid sqrt or ln.
Closes#10186
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`amalgamate.py` builds the generated crate in place, leaving a Cargo target
directory next to the sources. `/examples/target` was ignored but the
publish crate's own was not, so it showed up as ~32k untracked files that a
path-wide `git add` picks up.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Drop the vestigial UniversalWrite bound from the update-only writer
Neither writer kind performs in-place writes: AppendableSegment is built on
UniversalAppend, and DeleteOnlySegment tombstones via whole-mask atomic_save
(UniversalWriteFileOps), which UniversalAppend's supertrait already carries.
The bound is a leftover from the DiskIdTracker-based iterations that mutated
the deleted mask in place.
With it gone, UpdateOnlyEdgeShard::apply_batch is instantiable with the
object-store-appendable CachedBlobFile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* edge-shard-update: --apply writes the batch, over object storage too
Open the object-storage backends through CachedBlobFs/CachedBlobFile
instead of the read-only DiskCacheFs handle, so the shard is appendable in
both modes, and add --apply: generate the same schema-derived batch and run
apply_batch instead of preview_batch. Dry run stays the default and the
generation is shared, so the preview cannot drift from what an apply would
do. AwsConfig::native_append is exposed as --native-append for
AiStor/RustFS-style endpoints; the Cached* types join io_bridge_object_store's
re-export of the io_bridge stack.
Applying to a leader-produced shard currently fails with a clean refusal —
its appendable segment's payload storage was created in mutable mode, which
the append-only writer rejects — the known segment-bootstrap gap, next in
line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* CachedBlobFile: latency tracing for append_bytes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* UpdateOnlyEdgeShard: sequential batches through one writer
Writers open once at shard open, next to the lookup segments they resume
from. apply_batch hands the writer back on success, live-reloading the
lookup half of every segment the batch wrote to (new
LookupSegment::live_reload, mirroring the read-only segment's); on error
the writer is consumed, since its lookups may no longer describe the
durable state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* edge-shard-update: --interactive mode, sequential batches on one writer
After each applied batch, prompt on stdin for the next round's ids and
apply them through the writer apply_batch handed back — no shard
re-open — with op-num (and seed) incremented per round.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* CachedBlobFile: create the missing object on an offset-0 rewrite append
The caller-side rewrite path (part-copy S3 stores below the direct-append
threshold) validated the offset against the mirror length, whose
initialization HEAD-requests the remote and surfaced NotFound for an
object that does not exist yet. Direct-append backends (GCS compose,
native append) already create the object on an offset-0 append; the
rewrite now reads a missing remote as length zero so its whole-object PUT
does the same, and a non-zero offset against a missing object reports an
offset conflict.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* UpdateBatchOutcome: per-point records of retired slots
Each applied point now carries a PointApplyRecord: what happened to it
(stored/deleted/skipped/missing) and which slots it vacated where —
tombstoned per segment, or superseded in place for the old write-target
copy of a stored point. Built in the same loop that decides
tombstone-vs-supersede, so the report cannot drift from the writes.
edge-shard-update logs one line per point after the applied summary,
telling a fresh insert from an overwrite and naming the segments the old
copies were deleted from.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`cargo audit` flags rand 0.7.3 as unsound (RUSTSEC-2026-0097), and
permutation_iterator 0.1.2 is its sole importer. The crate is
unmaintained, so the finding is permanent for as long as we depend on
it.
Everything we used it for is "pick k distinct random indices out of n",
which is exactly `rand::seq::index::sample` from the workspace rand.
Switch the three src call sites and the two benches over, and drop the
dependency. 11 crates leave Cargo.lock.
Also fix a comment in quantile.rs claiming the permutation was
deterministic per count: the old crate keyed itself from thread_rng on
every call, so it never was.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The app telemetry was the only user of the sys-info crate, while segment
already depends on sysinfo for cgroup-aware memory accounting. Read the
distribution id/version via sysinfo statics, and the disk size fallback
via common::disk_usage, so the whole sys-info crate (and its bundled C
sources) drops out of the build. sysinfo is hoisted to a workspace
dependency, shared by the root crate and segment.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add CachedBlobFile: cached reads + write-through appends for object stores
Combine a DiskCache mirror (reads) with a BlobFile remote handle (appends)
into CachedBlobFile/CachedBlobFs, the appendable universal-IO citizen for
object stores. Appends perform the remote mutation inline and are durable
at Ok: a native write-offset append in AppendMode::Native (with a soft
limit on appends per object), or a whole-object rewrite in
AppendMode::Rewrite for stores without native append. After a successful
append the mirror length is advanced without extra IO; appended blocks
fault in from the remote on first read.
The multipart UploadPartCopy rewrite path (prefix >= 5 MiB) and the
rewrite-required error classification are left as todo!() pending the
AsyncRewrite backend capability.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Backend-advertised AppendMethod; reactive appended-block cap recovery
Replace CachedBlobFile's stored AppendMode with AsyncAppend::supported_append:
the backend advertises Native or PartialUpload, and append takes a matching
AppendRequest variant, rejecting the ones it does not support. The multipart
UploadPartCopy todo moves into the S3 backend's PartialUpload arm.
Drop the native_appends soft-limit counter: it is per-handle in-memory state
that resets on every restart, so it can never be the correctness mechanism
and persisting it would not make it authoritative either. The store is the
authority: hitting its appended-block cap now surfaces as the new
UniversalIoError::AppendRewriteRequired (S3 400 TooManyParts), and
CachedBlobFile recovers with a whole-object rewrite. Unrecognized errors
stay hard errors instead of silently triggering rewrites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Per-store append strategies; server-side rewrites for plain S3 and GCS
Replace the single AppendContext struct with an enum of strategy objects,
one per store capability, each owning its append logic:
- NativeAppend: the signed write-offset PutObject (S3 Express, MinIO
AiStor; AwsConfig::native_append declares it for AiStor-like endpoints,
s3_express implies it).
- PartCopyAppend: plain S3 — appends land as one atomic multipart rewrite
whose prefix parts are server-side UploadPartCopy requests; nothing but
the appended data crosses the network. object_store keeps such
provider-specific calls out of its portable surface, so the requests are
hand-signed like the native append.
- ComposeAppend: GCS — the appended data is uploaded as a temporary
neighbor object and composed onto the destination server-side,
conditional on the observed generation (a real compare-and-swap).
AppendMethod is replaced by AppendSupport, which tells the caller the only
thing it needs: when the store takes a direct append. Always (native, and
compose: no part minimums, no block cap), AboveThreshold (part-copy: the
copied prefix lands as non-last multipart parts, >= 5 MiB each), or Never.
CachedBlobFile drops its hardcoded MIN_COPY_PREFIX and rewrites locally
only below the backend-advertised threshold; AppendRequest::Rewrite now
means only "append and rebuild as a single blob" — the appended-block cap
recovery.
The append module is split one file per strategy, with a shared
SignedRequestContext transport and a test-only HTTP stub.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* DiskCache tracks the remote object's etag
Seeded from the new known_etag open extra (OpenExtra::with_known_etag),
refreshed from FileInfo on schedule_reopen, and settable directly for
callers that mutate the remote out of band.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove AppendRequest enum; appended-block cap recovery moves into the backend
AsyncAppend::append takes plain (path, offset, data). A native S3 store
that rejects an append with TooManyParts now falls back to the part-copy
rewrite inside the dispatcher, instead of surfacing AppendRewriteRequired
to CachedBlobFile for a second Rewrite request. The Rewrite variant was
handled identically to Append everywhere except that one native path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Escalate to download+rewrite when the store rejects a part-copy rewrite
The cap-recovery rewrite is chosen by the store's returned error, not a
client-side threshold: a part-copy attempt rejected with EntityTooSmall
(typed as UniversalIoError::AppendEntityTooSmall, parsed from the S3
error <Code>) falls back to downloading the sub-part-minimum prefix and
PUTting the whole object back, guarded by a prefix-length offset check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix S3 Express appends: zonal endpoint + s3express SigV4 service
Hand-issued appends targeted the standard endpoint and signed as "s3",
so every append to a directory bucket got 404 NoSuchBucket, masked as
AppendOffsetConflict by the 404 mapping. Derive the zonal
{bucket}.s3express-{az}.{region} base from the mandatory --{az}--x-s3
bucket suffix (mirroring object_store's private derivation), carry the
SigV4 service name in SignedRequestContext, and treat a 404 as a
conflict only for NoSuchKey or bodiless responses — NoSuchBucket stays
a loud error guarding the endpoint derivation. extract_xml_tag moves up
to the context module and now tolerates tag attributes and
pretty-printed bodies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Server-side etag precondition on appends; BlobFile loses UniversalAppend
AsyncAppend::append carries an expected_etag that S3 part-copy rewrites
attach as x-amz-copy-source-if-match (412 -> AppendEtagMismatch, a new
typed error) and download_rewrite checks against the GET's own etag;
native write-offset PUTs and GCS compose ignore it. BlobFile appends
only through the inherent etag-aware append_bytes now — CachedBlobFile
calls it directly with its DiskCache-tracked etag — and BlobFs's
mutating ops become inherent, delegated from CachedBlobFs, per the
standing TODOs. The append conformance battery runs over the
CachedBlobFs stack, via new direct constructors that share one backend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Drop unfulfilled too_many_arguments expectation
rewrite_parts has exactly seven parameters — at the clippy threshold,
not over it — so the lint never fires and the expect fails CI under
-D unfulfilled-lint-expectations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A raw point can carry its payload as the byte blob it is stored as, mirroring
`PointStructRaw.raw_payload` on the internal gRPC API. The blob travels from the
sending node into the receiving node's WAL untouched, so the sender never parses
the payload it read and neither node builds a protobuf value tree for it.
It is parsed exactly once, where the operation is unpacked for apply
(`process_point_operation`), because that is the first place the parsed form is
actually needed: `set_full_payload` goes through the payload index, which cannot
be updated from bytes. The gRPC boundary therefore only checks the encoding tag
and rejects a point that sets both payload fields, the way the enclosing request
already rejects both `points` and `raw_points`.
Moving the parse onto the apply path makes its error classification load-bearing,
so a malformed blob is reported as `OperationError::MalformedPayloadBlob` — the
payload sibling of `MalformedVectorBlob`, mapped to `CollectionError::BadInput`
for the same reason: a bad blob that reached the WAL has to be skipped on replay
instead of crash-looping recovery.
Three consequences of the blob living that long are handled explicitly rather
than by convention:
- `decode_payload_raw` takes the blob only once it has parsed, so a failure
leaves the point holding it instead of holding neither representation.
- `upsert_points_raw` and `sync_points_raw` refuse a point that still carries a
blob. They read the parsed payload, so such a point would otherwise be stored
with no payload at all, and a `debug_assert!` would not catch it in release.
- `is_equal_to` compares blob to stored blob as bytes. A differing encoding costs
a redundant upsert on sync, never a skipped one.
The `raw_payload_transfer` bench measures the trade, per 100-point batch (one
transfer batch) at payloads of ~200 B / ~700 B / ~7 KB:
- Sender, storage bytes to wire: 16x / 37x / 113x faster. This is where the whole
win is — no parse of the blob that was read, no value tree built.
- WAL encode: 5x / 11x / 25x faster, writing a byte string instead of a map.
- Receiver, wire to applicable point: 1.09x / 1.10x / 1.06x. Near neutral, as it
swaps walking a prost value tree for a JSON parse.
- Wire bytes: ~6% smaller. WAL bytes: 10-32% *larger*, because the blob is JSON
while a parsed payload is written as a compact CBOR map.
The WAL growth is accepted rather than fixed: decoding earlier to win those bytes
back costs a second full deserialization, and would leave the receiving side with
a `payload_raw` that is never populated. Making the blob itself compact belongs in
the payload storage encoding (`RawPayloadEncoding` is the extension point for it),
not here.
Two flags, both off by default and both sender-only (nodes accept raw points and
raw payloads regardless), read where the transfer batch is prepared:
- `transfer_raw_points` transfers every collection as raw points, not only those
whose vector storage would drift in a decode-encode round-trip.
- `transfer_raw_payloads` ships the blob a raw read hands out; without it the
prepared batch decodes it back into the parsed payload, and the wire message is
exactly what it is today.
Neither is enabled by `all`: a node only accepts them once it runs a version that
understands them, so they can only be switched on a release later. Nothing
enforces that yet — the transfer has no peer-version gate.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Move delete-only tombstoning into per-format update-only id trackers
DeleteOnlySegment::tombstone_points wrote the deleted mask file directly,
hard-coding that both immutable id-tracker formats store it the same way.
Give each immutable format (in-RAM and disk-resident) its own update-only
tracker that owns the decision of where its tombstones go, and dispatch
through DeleteOnlyIdTrackerEnum, which lives next to ReadOnlyIdTrackerEnum.
Both formats share one deleted-mask file today, so the trackers delegate to
a shared writer in deleted_storage.rs; a format that diverges later changes
only its own update_only module.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Enforce empty-batch guard in the shared tombstone writer
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test: parameterize collection fixture and share resharding consensus stub
Let integration tests build a collection with a custom optimizers
config, and move NoopReshardingConsensus from the consensus idempotency
test into the shared test module.
* test: reproduce scale-down resharding abort hanging on deferred points
abort_resharding holds the shard holder write lock while
scale_down_cleanup_points deletes migrated points with
WaitUntil::Visible and no timeout. With prevent_unoptimized enabled that
wait only resolves once the optimizer has cleared every deferred point
of the shard, so a stalled optimizer wedges the write lock, and with it
every shard holder reader and - in a cluster - the consensus apply
thread driving the abort (SetShardReplicaState(Dead) on a
ReshardingScaleDown replica, e.g. after that peer is killed).
The test drives such an abort with the optimizer disabled and asserts it
completes. It currently fails by hanging into its 30s timeout, and must
pass once the cleanup delete no longer waits for visibility.
* When cleaning up old points, don't wait until visible
The debug_assert wrote 'page_size_bytes % block_size_bytes * region_size_blocks == 0', which parses as ((page % block) * region) == 0 and is therefore always true. Use is_multiple_of(block * region), matching the sibling assertion in open(). Rebased onto dev after the gridstore crate moved under lib/blobstore.
extract_tracing_id guarded on value.len() (bytes) but truncated with chars().take(MAX_TRACING_ID_LEN) (chars), so a multibyte header could exceed the documented byte cap by up to 4x. Truncate on a UTF-8 char boundary at or below MAX_TRACING_ID_LEN bytes. Adds regression tests.
get_random_layer draws from a half-open [0, 1) uniform, so the sample can be 0.0; ln(0.0) is -inf and the float-to-usize cast saturates the level to usize::MAX, making set_levels allocate unboundedly. Clamp the sample to f64::MIN_POSITIVE. Extracts a small level_from_sample helper and tests it.
load_from_wal sized the WAL-replay handoff from update_sender.capacity() (currently-available slots, which vary at runtime) but treated it as the total queue size, contradicting update_queue_length() in the same file which uses max_capacity() as the total. Use max_capacity(). This also removes the 'update_queue_size as u64 - 1' underflow risk, since max_capacity() is >= 1 while capacity() can be 0.
CompressedPostingBuilder stored the running byte offset with 'data_size as u32', which silently truncates if a posting list's compressed id-data ever exceeds u32::MAX bytes, producing a wrong offset and corrupt reads. Use u32::try_from(..).expect(..) so it fails loudly, matching the guarded sibling in the posting_list crate.
`TableOfContent::new` installs the process-global quota manager, and
`set_global` treated a second install as a startup-order bug worth a
`debug_assert!`. A test binary runs all of its tests in a single process,
and every test that builds a table of contents installs the manager again,
so all but the first one panic.
Building several tables of contents in one process is legitimate for test
harnesses, so the second install is no longer fatal. A node that installs
twice still reports it loudly through the existing error log.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
to_block_range cast block indices with a raw as u32, silently truncating
past 64 TiB and reading the wrong region. Mirror the sibling guard in
disk_cache/cached_slice.rs and fail loudly with u32::try_from(...).expect.
Sparse and MultiDense variants already reject empty vectors via
validate_sparse_vector_impl / validate_multi_vector_by_length, but
Dense always returned Ok(()). Reuse the existing
validate_non_empty_dense helper from the REST validation module for
consistency.
Alternative to #10188. Instead of skipping check_consistency_and_repair's
storage cleanup entirely on append-only segments, and keeping a separate
delete_point_tombstone_only helper that callers must remember to pick, fold
the append-only decision into delete_point_internal itself:
- delete_point_internal now branches on is_append_only_delete() internally:
tombstone-only (id tracker drop only) when true, full payload/field-index
clear + id-tracker drop otherwise.
- delete_point_tombstone_only is removed; both call sites (ordinary
delete_point, and check_consistency_and_repair's cleanup of dangling
versions found by fix_id_tracker_inconsistencies) now just call
delete_point_internal, so the repair path gets correct append-only
behavior for free instead of needing its own explicit check.
- version_tracker.set_payload (payload-storage version, used for partial
snapshots) moves inside delete_point_internal too, gated behind the same
branch: it's only bumped when payload storage is actually touched. Took
the opportunity to also thread it through an explicit op_num: Option<..>
parameter, since check_consistency_and_repair's repair pass has no real
op_num to associate the change with.
This keeps "how to delete a point" a property of segment state rather than
something every caller has to branch on externally, which is what let the
original bug slip through check_consistency_and_repair in the first place.
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
confidence_agresti_coull_interval uses z = 2.0, which is ~95.45% confidence, not the 0.99 the doc comment claimed (0.99 needs z ~ 2.576). Correct the comment; z is left unchanged since raising it would widen every interval and break test_confidence_interval.
* [UpdateOnly] tombstone points in immutable segments via whole-mask rewrite
DeleteOnlySegment::tombstone_points marks the retired slots in the
segment's deleted-points bitmask (id_tracker.deleted, shared by the
immutable and disk-resident tracker formats) and replaces the file
whole via atomic_save — the one mutation that works on backends
without random-offset writes. Both read-only trackers already
live-reload this file by opening a fresh handle and diffing, so the
rewrite needs no read-side changes.
The mutation cycle lives in StoredBitSlice::atomic_update: read the
stored bits (or start from a caller-provided seed), apply the update,
save atomically; a closure error writes nothing. The seed comes from
the read phase by analogy to AppendableIdTrackerState:
LookupSegment::writer_state now returns WriterIdTrackerState, whose
DeleteOnly variant carries the deleted mask when the tracker already
holds it in memory — always for the immutable tracker, only if
materialized for the disk-resident one, which deliberately avoids
loading the full deleted set.
Tombstoning needs no more of the backend than reads plus atomic_save,
so DeleteOnlySegment's bound drops to
UniversalRead<Fs: UniversalWriteFileOps>.
Unlike the writable trackers' drop(), the slot's version is not zeroed
(the versions file is in-place-mutated, which object stores cannot do):
deletion authority in these formats is the bit — every lookup filters
through it — and a stale version on a tombstoned slot is the same state
a crash between drop-bit and drop-version leaves, which
fix_inconsistencies already absorbs as storage cleanup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Close the temp-file handle in tests that atomically replace it
NamedTempFile holds the file open for its lifetime, and Windows refuses
the rename in atomic_save while any handle is open. into_temp_path()
closes the handle and keeps the deletion guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* add `UniversalIoError::UnchangedOpen`
* add & impl `CachedReadFs::reschedule_prefetch`
* clippy
* add `OkUnchanged` helper
* propagate scheduling errors immediately
* drop lock before reacquiring it
* clear prefetched_files on new snapshot
* only avoid prefetch on full FileInfo equality
* `UnchangedOpen` maps to `Cancelled`
* match-all match
* Add edge-tool: CLI for creating, seeding, optimizing, and uploading local edge collections
Mirrors the style of lib/edge/tools/shard_update and shard_query: `create` builds a
minimal EdgeShard on disk (dense/sparse vectors, quantization presets including
turbo4, payload indexes, target segment count), `upsert` seeds it with random points
matching its live schema, `optimize` runs the shard optimizers, and `upload` pushes
the resulting directory to S3/GCS. Useful for quickly spinning up test collections
without a running Qdrant server, then promoting them to object storage.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* edge-tool: initialize feature flags, enable serverless_compatible, fix --sparse ambiguity
Initialize the global feature-flag OnceLock at startup (with serverless_compatible
set, cascading write_segment_manifest/append_only_mutations/compact_bitmask/
append_only_storages) so runs no longer spam "Feature flags not initialized!" and
collections are created in the serverless-compatible format.
Also splits --sparse into a plain boolean flag plus a repeatable --sparse-name:
clap's optional-value parsing for the old `--sparse [NAME]` form silently
swallowed a following positional PATH as the sparse vector's name whenever
--sparse was the last flag before it (e.g. `create --dense 1024 --sparse ./col`).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* edge-tool: fix --sparse=NAME to require_equals instead of a separate flag
The --sparse/--sparse-name split from the previous commit lost the ability to
name a sparse vector with --sparse itself. Restore a single --sparse[=NAME]
flag, but with require_equals(true): clap then only binds a value via
--sparse=NAME, never via a following bare token, so it stays safe next to the
trailing PATH positional in every position (bare --sparse, --sparse=NAME, or
multiple --sparse=NAME occurrences) without reintroducing the ambiguity that
made --sparse swallow PATH as its value.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* edge-tool: remove --segments from create, it has no effect there
EdgeOptimizersConfig::default_segment_number only feeds MergeOptimizer as a
merge-down ceiling (reduce segment count when it exceeds the target); unlike
the main collection's LocalShard::build_local, EdgeShard::new never loops to
pre-create N appendable segments. A freshly created collection always starts
at exactly 1 segment, so passing --segments to `create` was silently a no-op.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* edge-tool: add --indexing-threshold-kb to create
Unlike --segments (removed previously), the indexing threshold is a parameter
IndexingOptimizer actually consults on every optimize() run: segments larger
than it get an HNSW index built. Verified end-to-end (create with a 1KB
threshold, upsert 2000 points, optimize) that it produces an hnsw-indexed
segment where it would otherwise stay plain.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* edge-tool: add --clean to upload, wiping the destination prefix first
Lists every object under DESTINATION and deletes it via ObjectStore::delete_stream
before uploading, so re-uploading a collection recreated with a different shape
(different segment UUIDs) doesn't leave the old segment's files behind.
Verified against the local S3 proxy: uploaded one collection, then a second,
differently-shaped one to the same prefix without --clean (29 objects, stale
leftovers from the first); re-uploading the second with --clean correctly
dropped it back to exactly its own 19 files.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* [UpdateOnly] create Blobstore-backed storages append-only under a feature flag
A new `append_only_storages` feature flag, enabled by `serverless_compatible`,
switches every Blobstore creation site — the payload storage, the appendable
field indexes (numeric, map, geo, full-text) and the sparse vector storage —
to the append-only Logstore mode. One shared helper maps each site's Gridstore
layout to its Logstore counterpart, carrying the page size and compression
over; blocks and regions have no append-only equivalent. Only creation
consults the flag: an existing storage keeps its persisted mode, both modes
are always readable, so flipping the flag never strands data.
Two changes make the flag usable rather than booby-trapped:
`Logstore::delete_value` now succeeds trivially where nothing is stored, as
mutable mode does, and errors only for a stored value. The ordinary write
paths delete defensively — an index clears a slot before filling it, an empty
value is stored as a deletion — and only ever hit occupied slots when
something is genuinely mutated in place.
A segment derives `append_only_storages` from the persisted payload storage
mode when it opens — not from the flags, which may have changed since it was
created — and it forces append-only mutation semantics on itself: every
mutation clones to a fresh slot, and the same-operation slot-reuse shortcut
is disabled, since the second step of a multi-step write would rewrite a
payload row those storages cannot rewrite.
The end-to-end test runs as its own binary (feature flags are process-global)
with `serverless_compatible` on: the segment comes out holding Logstore
storages, and upserts, updates of existing points, multi-step same-operation
writes, deletes, an index build over existing points, a flush and a reload all
run against them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] assume the flag pairing instead of deriving it, trim the docs
Per review: `append_only_storages` without `append_only_mutations` is not a
state to defend against — `init_feature_flags` forces the pairing, and the
same-operation slot-reuse check reads the flag directly. That deletes the
segment-side derivation: the `append_only_storages` segment field, the
persisted-mode read at open, and the `is_append_only` accessor chain through
`Blobstore`, `PayloadStorageImpl` and `PayloadStorageEnum`.
Docstrings and comments trimmed to the guarantees; how the write paths use
them is their own business.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] restructure the creation config around mode-neutral options
`CreateOptions` in the blobstore crate holds what a caller actually decides —
page size, block size, compression — and `into_config(append_only)` turns them
into the config of either mode, each taking the fields it can express. The
segment-side `storage_config` supplies only the mode, from the feature flag.
That removes the misnomer chain the previous cut left behind: nothing named
gridstore returns a config that might not be one, and no call site builds a
`GridstoreConfig` just to have its fields repacked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] keep Logstore strict; fix the callers that deleted nothing
Per review, `Logstore::delete_value` goes back to an unconditional error: a
delete reaching an append-only storage is a caller bug to fix, not a case to
absorb. The callers that issued vacuous deletes are fixed instead:
- The numeric and geo indexes only delete from the storage when their
in-memory index actually held values at the slot — the two are written in
lockstep, so an empty slot has nothing stored either. The map and text
indexes already worked this way.
- The sparse storage skips the delete for keys at or past its end, where
nothing was ever stored.
Each removed call was wasted work in mutable mode too. The e2e test now also
drives a numeric index and the sparse storage against append-only mode, and
asserts that deleting a stored sparse vector fails.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [UpdateOnly] drop the same-op slot reuse; upserts write the whole point at once
The append-only path never needs a multi-step point write: the one real
multi-stepper was the shard's upsert — `upsert_point` followed by a payload
step under one operation number — and it now goes through
`upsert_moved_point`, which writes vectors and payload as one operation and
one slot. With that, the same-operation slot-reuse carve-out in
`handle_point_mutate` has nothing to carry: on an append-only segment every
mutating step clones to a fresh slot, unconditionally, and the
`append_only_storages` special case disappears with it. The version gate
skips only on strictly newer versions, so a caller that still multi-steps
stays correct — it pays a slot per step.
`PointToUpsert` now exposes the point's parts — raw vectors, decoded vectors,
payload — and both write paths are provided from them: `upsert_into` hands
the parts to `upsert_moved_point`, `write_moved` adapts them to the
copy-on-write move callback. The two hand-written `upsert_into` bodies and
the follow-up payload helper are gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Regenerate OpenAPI for the `append_only_storages` feature flag
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* one extra debug assertion
* fmt
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] wire every component into the appendable segment
`AppendableSegment::store_points` sheds its `todo!()`: the id tracker claims a
fresh slot per point, every component writes its data at those slots — each
named vector storage, the payload storage, the payload indexes — and only then
do the versions cover them, the step that makes the points visible to readers.
A crash anywhere in between leaves claimed, unpublished slots, which the next
writer to open the segment retires. Each vector comes from whichever half of
`FullyQualifiedPoint` holds it: the batch's decoded vectors win over the bytes
carried from the point's previous slot, and a name in neither still takes its
slot as a vector the point does not have.
The store components open lazily, on the first `store_points`. A batch that
only deletes writes nothing but the mappings log, so it never pays for those
opens — and it keeps working against segments whose payload storage was created
in mutable mode, which the append-only writers refuse and which is all any
leader builds today.
The writer now also remembers what it stored, so `tombstone_points` skips a
point this very batch wrote instead of retiring its fresh slot; the caller can
hand over every slot a stored point used to occupy without holding that rule.
`UpdateOnlySegmentEnum::open` takes the segment config, which is where the
writer learns which vector storages exist.
The end-to-end edge tests now run stores the whole way through: located and
resolved through the `LookupSegment`s, appended by the writer, and read back
through an ordinary follower — a new point with its payload, a rewrite winning
over the old copy, a replayed batch skipping on the published versions, and a
second writer resuming every component where the first ended. The leader still
writes its payload storage in mutable mode, so the tests recreate it empty in
append-only mode, standing in for segment creation wiring that does not exist
yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] drop the stored-ids guard from `tombstone_points`
The caller already never asks to retire a point its batch stored — it has to
hold that rule regardless, since `preview` mirrors it to count outcomes — so
the writer-side set was redundant state, and it made `tombstone_points`
silently drop requests instead of honoring a stated contract. The contract is
now stated: only points the batch deletes go here, because a delete addresses
the external id and would take a stored point's fresh slot along with the
stale one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] gate the store tests off Windows
The leader's writable storage preallocates chunk files, and the append-only
writer cuts them back to end at the data — its append offset is a
compare-and-swap token, so a file longer than the data would make every append
conflict. That cut replaces the file, which Windows refuses while the writer's
own `LookupSegment`s hold it memory-mapped; on Linux the old inode simply lives
on under the mappings. Nothing to fix in the writer: Windows cannot shrink a
mapped file, and the production target is object storage, where neither
preallocation nor mmap exists.
The delete tests keep running everywhere; the store tests move into a
`#[cfg(not(windows))]` module together with the imports and helpers only they
use, so the Windows build carries no unused-import warnings. Cross-checked with
`--target x86_64-pc-windows-msvc`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] wire the quantized overlay into StoreComponents
Opens UpdateOnlyQuantizedVectors alongside each dense, non-multivector,
non-Turbo4-datatype vector's raw storage, when the segment's quantization
config supports incremental appends (Binary/Turbo). Multivector and Turbo4
combinations are out of scope (see UpdateOnlyQuantizedVectors' own doc
comment) — such a vector simply has no quantized overlay entry and stays
searchable exactly through its raw storage alone, same as before.
store_points keeps the overlay's row count in exact lockstep with the raw
storage: every point takes a row in both, in the same order, at the same
id (start_slot + offset) — a decoded vector encoded for real, a
Raw-bytes-carryover blob decoded back to f32 per its actual storage
datatype (mirroring QuantizedVectors::create_impl's use of
PrimitiveVectorElement::quantization_preprocess for the same purpose on
the non-update-only path), and a Missing vector as an all-zero placeholder.
Skipping a row for the latter two cases would silently misalign every
later quantized lookup — scoring one point's vector against another's
quantized copy — so this mirrors the raw storage's own "every point takes
its slot" rule exactly rather than only handling the common decoded case.
UpdateOnlyQuantizedVectors now retains its resolved QuantizedVectorsConfig
(exposed via quantization_config()/dim()) rather than discarding it after
opening storage, since a reopened overlay's persisted config is the source
of truth for how to decode carried-over bytes — not necessarily identical
to whatever live config the caller has to hand. Its now-unused flusher()
is dropped: like every other update-only storage in this stack, a write is
already durable when append_many/upsert_vector returns.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [UpdateOnly] implement the appendable quantized-vector overlay (dense, Binary/Turbo)
Appendable/plain segments can carry live quantized vectors today: PlainVectorIndex::
update_vector calls quantized_vectors.upsert_vector alongside the raw vector on every
insert (lib/segment/src/index/plain_vector_index/lifecycle.rs), auto-created for a
fresh segment when appendable_quantization is on and the method supports it
(QuantizationConfig::supports_appendable — Binary and Turbo only; Scalar/Product are
policy-gated off regardless of storage backend). The update-only vector-storage stack
(this PR's base) had no equivalent: UpdateOnlyVectorStorage::open never read
quantization_config, and nothing under vector_storage/*/update_only/ mentioned
quantization at all — a segment configured with quantization would silently lose it
end-to-end once written through this path.
This adds UpdateOnlyQuantizedVectors, mirroring QuantizedVectors' auto-create/reopen
behavior but scoped to dense (single-vector) Binary/Turbo — the two methods that
support incremental appends, matching current capability exactly (multivector support
is a follow-up: it needs its own append-only offsets storage, mirroring
MultivectorOffsetsStorageChunked the same way this mirrors QuantizedChunkedStorage).
The only new machinery is UpdateOnlyQuantizedChunkedStorage, an EncodedStorage backed
by UpdateOnlyChunkedVectors (append-only, S: UniversalAppend) instead of
ChunkedVectors' positional writes (S: UniversalWrite) — everything else reuses the
quantization crate's EncodedVectorsBin::encode/load and EncodedVectorsTQ::encode/load
completely unchanged, since both are already generic over the storage backend. It
writes files in the exact layout QuantizedChunkedStorage reads, so a promoted segment's
quantized data reads through the existing, unmodified reader with no new reading code.
UpdateOnlyChunkedVectors gains one addition: a `get` method to read back a single
vector, needed because EncodedVectors::load validates the storage's vector size by
reading vector 0 (skipped when the store is still empty).
Verified: the update-only writer's persisted bytes, read back through the standard
(non-update-only) QuantizedChunkedStorage + EncodedVectorsBin/TQ::load, match a
RAM-backed reference fed the same vectors one at a time through upsert_vector,
byte-for-byte, for both Binary and Turbo.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* [UpdateOnly] fix quantized reopen: resume writing shouldn't validate stored reads
The previous commit made reopening a non-empty quantized overlay panic
(EncodedVectorsBin/TQ::load validates a non-empty store by reading its
vector 0, which UpdateOnlyQuantizedChunkedStorage's write-only design
cannot serve) and worked around it with a redundant pre-check plus a
todo!(), narrowing the tests to single-session-only writes.
Both of those were the wrong fix. A writer resuming appends doesn't need
`load`'s read-and-validate — it only needs the fitted metadata (encoding,
stats) to keep encoding consistently, and that invariant already holds by
construction: every vector this writer ever encodes is sized from the same
`quantized_vector_size` `load` and the new path both read. Added
`EncodedVectorsBin`/`EncodedVectorsTQ::reopen_for_write` to the
quantization crate — identical to `load` minus the validating read — and
switched `open_existing` to it. `UpdateOnlyQuantizedChunkedStorage` stays
write-only as originally designed; no new read capability, no pre-check,
no todo.
Tests restored to the original two-writer split (write half, drop, reopen,
write the rest), now genuinely exercising resume-with-data instead of
avoiding it, and still passing byte-for-byte against the reference.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* [UpdateOnly] split EncodedStorage into EncodedStorageWrite + EncodedStorage
A write-only storage (the update-only quantized overlay) had to fake a
full EncodedStorage impl with unreachable!() read stubs just to satisfy
EncodedVectorsBin/TQ's generic bound. Split the trait so a write-only
backend only needs to implement EncodedStorageWrite; EncodedStorage adds
the read methods on top. The overlay now implements EncodedStorageWrite
alone — no panicking stand-ins for methods that don't exist.
* [UpdateOnly] remove UpdateOnlyQuantizedVectors::create
Nothing in this stack builds the first appendable segment of a
collection yet (that's still a todo!() in edge/src/update_only), so
create() had no real caller and open() had to guess from file absence
whether to invoke it. open() now only reopens an overlay create()
already persisted; the bootstrap logic moved into tests.rs as a
private fixture helper, since tests still need it to build fixtures.
* [UpdateOnly] fix CI: codespell typo and lint dead-code on unwired write path
codespell flagged "implementors" (wants "implementers") in two doc
comments. Separately, CI's lint job runs clippy without --all-targets,
so the update-only quantized write path — genuinely unreachable from
any non-test code until #10152 wires it into a segment — trips
-D warnings dead-code. Scope #![allow(dead_code)] to the two files
that are only exercised by their own tests today, and allow the now
test-only UpdateOnlyQuantizedChunkedStorageBuilder re-export.
* [UpdateOnly] fix ast-grep: use expect(dead_code) instead of allow
* fix CI: remove unused EncodedStorageWrite import in gpu vector storage
Left over from splitting EncodedStorage into EncodedStorageWrite +
EncodedStorage; only caught under --all-features since gpu is gated
behind a feature flag.
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: qdrant-cloud-bot <111755117+qdrant-cloud-bot@users.noreply.github.com>
Luis Cossío