* extend WAL retention if some other replicas are dead
* use merged WAL commit
* Handle ManualRecovery state
---------
Co-authored-by: KShivendu <kshivendu1@gmail.com>
* Handle operations when having no shards with custom sharding
* Return error instead of acknowledge
* fmt
* clippy
* shorter msg
* Return collection payload schema if exists even without any shards
* minor clippy fix
* Replace `Option<Segment>` with `enum LoadSegmentOutcome`
* Replace some Path/PathBuf with str/String
* Rename field Segment::{current_path -> segment_path}
* safe_delete
* Add a test for recovery after kill during Partial
* Address AI review
* simplify test
* Merge pull request #7829
* introduce a new state
* switch to ManualRecovery for user-initiated snapshot operations
* fmt
* fix test and regen api
* test fix: instead of force recovery on update failed snapshots now re…
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* move QdrantInternalService to its own file
* implement internal telemetry endpoint
cleaner diff on src/main.rs
upd response in proto
* use try_from conversion
* rename request/response in proto
* Use RwLock for pending changes in MmapBitSliceBufferedUpdateWrapper
* Use RwLock for pending operations in DatabaseColumnScheduledDeleteWrapper
* Use RwLock for pending updates in DatabaseColumnScheduledUpdateWrapper
* Drop alive guard before reconciliation, we don't touch files after
* Remove redundant clone
* Update comments
* in MmapSliceBufferedUpdateWrapper
* in MmapBitsliceBufferedUpdateWrapper
* in MutableIdTracker's versions updates
* in MutableIdTracker's mapping updates
* clone updates only when non-empty
* only lock for reconciling pending changes
* simpler reconciling
* use Mutex as argument to ensure we only lock within reconciliation
* Add test to reproduce broken WAL delta after aborting stream records
* Add staging env var to slow down stream records transfers for test
* Tweak test formatting and utilities a bit
* Add comment to test, link to PR describing bug
* Update test so it still succeeds with patched behavior
* Fix broken WAL delta after stream records abort (#7791)
* Make set_replica_state async
* Add function called when active state of local replica changes
* Add snapshot for newest clocks
* Bump newest clocks snapshot on replica deactivation
* Use newest clocks snapshot during recovery
* Add enum for specifying whether to take or clear clocks snapshot
* Store clock snapshot inside clock map, removing extra file
This greatly simplifies state handling. It also prevent any kind of
desynchronization because all newest clocks are always persisted
atomically.
* Immediately persist clocks after taking snapshot
* Always update snapshot, only take if missing
* Take clock snapshots through each shard flavor, including proxies
* Propagate dedicated functions for taking and clearing clocks snapshot
* Only persist clocks immediately if changed on snapshot/clear
* Simplify recovery point logic, always take clocks snapshot if exists
* Remove unwrap
* Fix typo
* Fix doc comment
* Transfer driver is async, use Tokio sleep
* Reduce visibility
* easy fix with version check
* Link to PR
* Fix get transfers with source/target pairs, also fix dead replica aborts
* Use new source/target functions elsewhere
* Flip shard and peer arguments
* Fix incorrect check, from should be to
* Inline more format arguments
---------
Co-authored-by: timvisee <tim@visee.me>
* Enrich Gridstore model testing
* better
* no clear on Windows
* less windows
* sigh
* change strategy
* never give up
* x100 scale down for Windows
* last touch
* decrease for others too
* remove Iter test - the quadratic cost is not worth it
* Revert "remove Iter test - the quadratic cost is not worth it"
This reverts commit f0e98cb33b.
* try with limited Iter
* less logging as it is hurting crasher
* tracker.mapping_len() is slow
* hard limit good enough for now
