mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-06 10:00:58 -05:00
192ed73385
[audit-K] Make create_shard_key crash-safe with a single commit point create_shard_key persisted shard_key_mapping.json incrementally, one add_shard per placement entry, so the operation was not atomic across a crash. The re-apply gate state.shards_key_mapping.contains_key(&shard_key) becomes true after the first loop iteration, so a crash mid-loop on a multi-shard placement left the peer permanently holding a subset of the key's shards while every other peer had all of them. Write the mapping exactly once instead, after every shard of the key exists on disk, so there is no partial state to observe. SaveOnDisk writes atomically, load_shards derives the shard id list from the mapping alone (unreferenced directories are invisible after restart), create_shard_dir wipes leftovers, and max_shard_id reads only the mapping - so a replay allocates exactly the ids the crashed attempt did, which are the ids every other peer allocated too. The contains_key gate then holds as intended: it fires only for an operation that already completed in full, or for a genuine duplicate. ShardHolder::add_shards registers a batch and persists the mapping once, with add_shard as a one-element wrapper; the mapping write moved ahead of the in-memory updates, being the only fallible step. No caller changes behavior: Collection::new and load_shards already hit the write_optional early return, and start_resharding_unchecked still adds a single id. An empty placement is now rejected rather than silently returning Ok without creating the key. Covered by create_shard_key_test.rs: id allocation, replay before and after the commit, duplicate rejection, and the empty placement guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>