mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-06 18:10:58 -05:00
2735d40ecc
* Reset first_voter and prune address book on first-peer --reinit
A peer removed from consensus and killed after `RemoveNode(self)` was
committed but before it was applied keeps the old cluster's
`first_voter` and peer addresses in `raft_state.json`. First-peer
`--reinit` reset `conf_state` to a single voter (itself) but left both
untouched (`first_voter` is in fact never reset, even when the removal
is applied).
Both values are served to peers bootstrapping onto the reinitialized
cluster. A joining peer seeds its initial `conf_state` with the
advertised `first_voter`, and Raft conf-changes are deltas on top of
that base - so a stale `first_voter` permanently corrupts the joining
peer's voter set: it ends up with {old first peer, itself}, missing the
actual leader. Its `/readyz` then treats the still-alive old peer as a
cluster member and waits for the old cluster's commit index, which its
own consensus never reaches.
This only manifests when the reinitialized leader replicates its log as
plain entries (nothing applied before the kill, so the log anchor is
index 0). If the leader sends a snapshot instead, the snapshot's full
`conf_state` heals the corrupted seed - which is why
test_reinit_removed_peer only failed sporadically on CI.
Fix first-peer `--reinit` to behave like founding a fresh cluster:
reset `first_voter` to this peer (not `None`, or `recover_first_voter`
would re-derive the old first voter from the retained Raft log) and
prune `peer_address_by_id` to this peer only.
The kill-before-apply state is now injected deterministically into
test_reinit_removed_peer, reproducing the exact CI failure against the
unfixed binary. Since `--reinit` now prunes the address book, the
stale-address injection in
test_reinit_removed_peer_readyz_ignores_old_cluster moved to a restart
without `--reinit`, so it keeps exercising the /readyz `conf_state`
membership filter from #9688.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix test_reinit_consensus expecting stale address book after --reinit
The test waited for cluster size 2 right after starting the reinitialized
first peer, before the second peer was even started. That only passed
because first-peer --reinit used to keep the old cluster's addresses in
the address book - the stale state the previous commit removes. A
reinitialized first peer is a fresh single-member cluster; the other
peers re-join and re-register their new addresses right after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>