Files
qdrant/docs
Tim Visée 44fa95f6d5 Shard diff transfer integration (#3509)
* Add first stubs for WAL delta shard transfer method

* Repurpose queue proxy, use it for transferring WAL diff as well

* Integrate WAL delta transfer is transfer selection logic

* Add WalDelta shard transfer type which is not exposed in public API

* Basic implementation of falling back to stream records transfer

* Share await_consensus_sync function

* Ask remote shard for recovery point

* During WAL delta transfer, resolve shard diff locally for recovery point

* Rebase on latest dev, support empty WAL diff

* Rebase on latest dev, support empty WAL diff

* Use partial snapshot state for WAL delta transfer

* Set cutoff point on remote shard after shard WAL delta transfer

* Set cutoff point on remote shard after stream records transfer

* During WAL delta transfer, set s tate from partial snapshot to partial

* Describe WAL delta transfer in a comment

* Allow updating cutoff point in stream records transfer to fail

* Do not set cutoff point on remote shard on WAL delta transfer

* Make await consensus sync logic easier to read and reason about

* Fix fallback to other shard transfer method on WAL delta transfer fail

* Various minor improvements

* Add TODO for just ignoring API unimplemented errors

* Only allow stream records cutoff point error if remote is older version

* Allow switching to partial to fail when falling back

* Only change shard state to partial if not in partial state already

* Change default shard transfer method back to stream records

* Add important TODO back

* Add WAL delta shard transfer method in gRPC

* Prefer configured shard transfer method as default

* Extract shard transfer fallback logic into separate function
2024-02-23 17:32:22 +01:00
..
2022-01-18 15:49:35 +01:00
2022-06-21 17:52:56 +02:00