mirror of
https://github.com/qdrant/qdrant.git
synced 2026-07-23 11:11:00 -05:00
a6ea303e4b
* Compact stored bitmask for on-disk field index deleted masks Add StoredBitmask: a compact persisted bitmask written and read as a whole. The payload is a roaring bitmap of whichever bit value is the minority (mostly-0 and mostly-1 masks both stay tiny), falling back to raw dense bits when roaring would not be smaller, so the file is never larger than the dense representation. Files are replaced atomically via UniversalWriteFileOps::atomic_save; there is no in-place mutation. Use it for the write-once "no values" masks of the on-disk numeric, geo, map and full-text indexes, replacing the raw dense bitslice files sized at point_count/8 bytes regardless of content. Writing the new format is gated by the compact_bitmask feature flag (default off; enabled by `all` and `serverless_compatible`). Reading is format-agnostic regardless of the flag: the compact deleted_mask.bin is tried first, then the index-specific legacy file, so old segments stay readable and flag-off builds produce byte-identical legacy files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Split save_bitmask into named helpers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Regenerate OpenAPI spec for compact_bitmask feature flag Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review findings on stored bitmask - Avoid u64 overflow in the payload bound check when opening a mask with a corrupted payload_len. - Reject roaring payload positions beyond logical_len at read time, enforcing the BitmaskContent range contract for corrupted files. - Remove the opposite-format mask file after a successful save, so a rebuild with a flipped compact_bitmask flag can't leave a stale compact file shadowing the fresh legacy one (or an orphaned legacy file next to a compact one). - Make the compact-open numeric test tolerate builds that already wrote the compact format. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
