mirror of
https://github.com/qdrant/qdrant.git
synced 2026-09-25 23:47:38 -05:00
Arnaud GourlayandClaude Opus 5
f3ffc65531
* fix(shard): flush CoW destinations before the payload-index pre-build flush create_field_index force-flushes each segment before building an index on it (flush-before-build, #9767), one segment at a time, outside flush_all's all-segment lock capture and copy-on-write dependency ordering. That flush durably advances a CoW source past the delete halves of its pending moves. The appendable-first iteration order usually flushes the destination before the source, but not always: a destination proxy-wrapped by a running optimization is classified non-appendable and can skip its flush entirely through the already_indexed short-circuit (the proxy reports the field as present), and a move landing mid-pass is ordered behind nothing. Once the source flushes, the move's WAL entry stops being replayable: the pre-image is durably deleted while the only current copy sits in the unflushed destination, and a graceful close then loses the point. This is the root cause of the nightly model-testing reload divergence (#10095), traced end-to-end in CI runs 31583878492 and 31583871346: cow move op 5197 into a freshly proxied destination, index op ~5252 flushing every source past it while skipping the proxy, destination reloading at 5181, replay declining with 'No point with id'. The fix mirrors flush_all's invariant at the only per-segment flush site: before flushing a segment, flush the destinations of its pending flush_dependency edges (one hop suffices, destinations are appendable and never CoW sources). Destination guards are taken before the flush lock to keep the documented [segment locks -> flush lock] ordering. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(shard): regression test for the CoW-destination flush in create_field_index Reproduces the #10095 loss shape deterministically: a pending copy-on-write move out of a non-appendable source, a destination whose own pre-build flush is skipped by the already_indexed short-circuit, then a holder-wide create_field_index. Verified failing with the dependency-aware flush neutralized (destination stays behind the move while the source flushes past it) and passing with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(shard): move the CoW-aware single-segment flush into SegmentHolder Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>