mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-04 00:51:06 -05:00
192ed73385
[audit-K] Make create_shard_key crash-safe with a single commit point create_shard_key persisted shard_key_mapping.json incrementally, one add_shard per placement entry, so the operation was not atomic across a crash. The re-apply gate state.shards_key_mapping.contains_key(&shard_key) becomes true after the first loop iteration, so a crash mid-loop on a multi-shard placement left the peer permanently holding a subset of the key's shards while every other peer had all of them. Write the mapping exactly once instead, after every shard of the key exists on disk, so there is no partial state to observe. SaveOnDisk writes atomically, load_shards derives the shard id list from the mapping alone (unreferenced directories are invisible after restart), create_shard_dir wipes leftovers, and max_shard_id reads only the mapping - so a replay allocates exactly the ids the crashed attempt did, which are the ids every other peer allocated too. The contains_key gate then holds as intended: it fires only for an operation that already completed in full, or for a genuine duplicate. ShardHolder::add_shards registers a batch and persists the mapping once, with add_shard as a one-element wrapper; the mapping write moved ahead of the in-memory updates, being the only fallible step. No caller changes behavior: Collection::new and load_shards already hit the write_optional early return, and start_resharding_unchecked still adds a single id. An empty placement is now rejected rather than silently returning Ok without creating the key. Covered by create_shard_key_test.rs: id allocation, replay before and after the commit, duplicate rejection, and the empty placement guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
14 lines
306 B
Rust
14 lines
306 B
Rust
mod collection_restore_test;
|
|
mod collection_test;
|
|
mod common;
|
|
mod continuous_snapshot_test;
|
|
mod create_shard_key_test;
|
|
mod distance_matrix_test;
|
|
mod grouping_test;
|
|
mod lookup_test;
|
|
mod multi_vec_test;
|
|
mod pagination_test;
|
|
mod snapshot_recovery_test;
|
|
mod sparse_idf_test;
|
|
mod wal_less_snapshot_clocks_test;
|