mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-04 17:10:59 -05:00
835da459cb
* Do not select ourselves when finding auto shard transfer source * Limit automatic shard transfers with basic transfer count check * Extract shard recovery transfer request logic to separate function * Add global shard transfer tracker structure to allow rate limiting * Count and limit incoming and outgoing shard transfers separately * Make automatic shard transfer limit configurable * Move shard transfer tracker from global to collection level * Comment out new config parameters * Fix incorrect comment * Fix missing space in log message * Fix negated condition * Remove logic for requesting shard transfer on replica state change * Check shard transfer limits in consensus sync, use consensus state Instead of bothering with tracking proposed shard transfers, this now purely relies on state that is already in consensus. Each time we sync consensus, we request the maximum number of shard transfers up to the limit at that time. * Remove now obsolete shard transfer tracker * Revert now obsolete changes * Improve transfer IO counting comment, revert now obsolete code * Fix typos * Fix flipped variables, don't take self reference on Copyable type * Update lib/collection/src/collection/shard_transfer.rs Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Handle incoming/outgoing transfer counts separately, don't tuple it * Improve loop for counting incoming/outgoing transfers * Remove unused test function * Add consensus tests for automatic shard transfer limits * Apply suggestions from code review Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> * Remove debug lines from new test * Reorder last test a bit to resolve flakyness * We can have one more transfer for recovery on the other alive node * Attempt to reduce test flakyness, more points and more frequent polling * Update config/config.yaml Co-authored-by: Luis Cossío <luis.cossio@qdrant.com> * Explicitly note default shard transfer limit in configuration * Use default for shard transfer IO everywhere * Rename transfer limit check function to be more explicit --------- Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com> Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>