mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-02 16:10:58 -05:00
0b642178e2
* Buffer multi-dense offsets store to prevent reload corruption The appendable multi-dense storage flushes its `vectors` and `offsets` chunked stores independently. Each flusher snapshots its `status.len` at creation time but msyncs chunk bytes at execution time. A re-upsert that grows a point past its reserved capacity rewrites the point's `offsets` entry in place to a freshly-appended row region; if that lands in a flush's creation-to-execution window, the relocated entry becomes durable while the `vectors` store's recorded length still predates the rows it references. On reload that point is unreadable and a WAL append reuses the rows, clobbering another point. Wrap the offsets store in a write-back buffer (`BufferedOffsets`) so the durable offsets can never reference rows beyond the durable `vectors` length. Offset writes stage in a pending overlay and only land in the durable store while a flush executes; the flusher snapshots the pending set at creation time, so any write after that stays buffered for the next flush. Both flushers snapshot at the same instant, yielding a consistent durable cut. Rows written after the cut are unreferenced garbage the next append overwrites. This prevents the skew at the source instead of patching it on reload, and also closes the residual offsets-length smear. The buffer follows the Gridstore flusher convention: pending writes live inside the single lock-guarded store, reads consult the overlay then the durable bytes under one lock, and the durable msync runs after releasing the write lock so it never stalls concurrent reads. Enable the "m" multivector in the collection model test now that its reload divergence is resolved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Narrow offsets flush write-lock scope Build and sort the pending snapshot before taking the write lock; only the apply + reconcile need it. Shortens the lock hold so concurrent reads block less. Addresses review feedback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>