Andrey VasnetsovandClaude Fable 5 5f8cef9ebd [UpdateOnly] Writer over object storage (#10214) ...

* Drop the vestigial UniversalWrite bound from the update-only writer

Neither writer kind performs in-place writes: AppendableSegment is built on
UniversalAppend, and DeleteOnlySegment tombstones via whole-mask atomic_save
(UniversalWriteFileOps), which UniversalAppend's supertrait already carries.
The bound is a leftover from the DiskIdTracker-based iterations that mutated
the deleted mask in place.

With it gone, UpdateOnlyEdgeShard::apply_batch is instantiable with the
object-store-appendable CachedBlobFile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* edge-shard-update: --apply writes the batch, over object storage too

Open the object-storage backends through CachedBlobFs/CachedBlobFile
instead of the read-only DiskCacheFs handle, so the shard is appendable in
both modes, and add --apply: generate the same schema-derived batch and run
apply_batch instead of preview_batch. Dry run stays the default and the
generation is shared, so the preview cannot drift from what an apply would
do. AwsConfig::native_append is exposed as --native-append for
AiStor/RustFS-style endpoints; the Cached* types join io_bridge_object_store's
re-export of the io_bridge stack.

Applying to a leader-produced shard currently fails with a clean refusal —
its appendable segment's payload storage was created in mutable mode, which
the append-only writer rejects — the known segment-bootstrap gap, next in
line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* CachedBlobFile: latency tracing for append_bytes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* UpdateOnlyEdgeShard: sequential batches through one writer

Writers open once at shard open, next to the lookup segments they resume
from. apply_batch hands the writer back on success, live-reloading the
lookup half of every segment the batch wrote to (new
LookupSegment::live_reload, mirroring the read-only segment's); on error
the writer is consumed, since its lookups may no longer describe the
durable state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* edge-shard-update: --interactive mode, sequential batches on one writer

After each applied batch, prompt on stdin for the next round's ids and
apply them through the writer apply_batch handed back — no shard
re-open — with op-num (and seed) incremented per round.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* CachedBlobFile: create the missing object on an offset-0 rewrite append

The caller-side rewrite path (part-copy S3 stores below the direct-append
threshold) validated the offset against the mirror length, whose
initialization HEAD-requests the remote and surfaced NotFound for an
object that does not exist yet. Direct-append backends (GCS compose,
native append) already create the object on an offset-0 append; the
rewrite now reads a missing remote as length zero so its whole-object PUT
does the same, and a non-zero offset against a missing object reports an
offset conflict.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* UpdateBatchOutcome: per-point records of retired slots

Each applied point now carries a PointApplyRecord: what happened to it
(stored/deleted/skipped/missing) and which slots it vacated where —
tombstoned per segment, or superseded in place for the old write-target
copy of a stored point. Built in the same loop that decides
tombstone-vs-supersede, so the report cannot drift from the writes.

edge-shard-update logs one line per point after the applied summary,
telling a fresh insert from an overwrite and naming the segments the old
copies were deleted from.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

2026-08-14 20:06:00 +02:00

..

src

[UpdateOnly] Writer over object storage (#10214)

2026-08-14 20:06:00 +02:00

Cargo.toml

Universal IO: append to file (#9720)

2026-07-15 13:55:08 +02:00