* Use RwLock for pending changes in MmapBitSliceBufferedUpdateWrapper
* Use RwLock for pending operations in DatabaseColumnScheduledDeleteWrapper
* Use RwLock for pending updates in DatabaseColumnScheduledUpdateWrapper
* Drop alive guard before reconciliation, we don't touch files after
* Remove redundant clone
* Update comments
* Add test to reproduce broken WAL delta after aborting stream records
* Add staging env var to slow down stream records transfers for test
* Tweak test formatting and utilities a bit
* Add comment to test, link to PR describing bug
* Update test so it still succeeds with patched behavior
* Fix broken WAL delta after stream records abort (#7791)
* Make set_replica_state async
* Add function called when active state of local replica changes
* Add snapshot for newest clocks
* Bump newest clocks snapshot on replica deactivation
* Use newest clocks snapshot during recovery
* Add enum for specifying whether to take or clear clocks snapshot
* Store clock snapshot inside clock map, removing extra file
This greatly simplifies state handling. It also prevent any kind of
desynchronization because all newest clocks are always persisted
atomically.
* Immediately persist clocks after taking snapshot
* Always update snapshot, only take if missing
* Take clock snapshots through each shard flavor, including proxies
* Propagate dedicated functions for taking and clearing clocks snapshot
* Only persist clocks immediately if changed on snapshot/clear
* Simplify recovery point logic, always take clocks snapshot if exists
* Remove unwrap
* Fix typo
* Fix doc comment
* Transfer driver is async, use Tokio sleep
* Reduce visibility
* Move pyproject.toml to root
* Migrate pyproject.toml from Poetry to uv
* Update GH workflows
* Update test script, doc and nix to use uv
* Use latest uv
* Fix uv.lock
* Cleanup shell.nix
* Cleanup
- Explicit `uv sync` is not required, `uv run` will install deps
automatically.
- We don't provide a python package, so the `[build-system]` section
is not needed.
* Fix UV_VERSION inconsistency
---------
Co-authored-by: tellet-q <elena.dubrovina@qdrant.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* Separate set and unset lists for each Gridstore pointer update
This makes the list of pointer updates much easier to grasp. The
simplification is desired because this exact structure has been a cause
for bugs multiple times now.
* Add more aggressive debug assertions
* Use consistent terminology
* Correct removal of set
* Simplify drain function
* Add safe guard to ignore empty pointer updates
* Rework tracker getter, explicitly branch variants
* Patch transmute, require Sized type
* rename and adjust comments
- set/unset is now current/to_free
- adjusted descriptions and comments for this nomenclature too
* clippy
* clippyyy
not 4, not 2, but 3 spaces
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Fix Gridstore tracker drain and persist not working properly
* Patch existing test
* Add new test to assert buggy scenario we found
* Mention PR in test
* Add lease structure to invalidate pending flushers after wipe/clear
* We don't use bitmask as barrier anymore
* Add missing early return
* Add test
* Replace flush lease with is alive boolean we also use elsewhere
* Reimplement test, now it does fail on the old implementation
* In test, cover both new fixed and old broken flushing
* Remove old test case because it is flaky
* Fix clippy
* Fix outdated comment
* On Windows ARM64 builds, disable usage of neon
* Also disable optimized popcount on Windows ARM64
* fix quantization build
* revert changes in BQ
---------
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* Fix delete_page not dropping sequential mmap
* In Gridstore Page, support using a single mmap
* In Gridstore Page, only open multiple mmaps if supported at runtime
* Support opening single mmap in UniversalMmapChunk
* Rename environment var, log warning when QDRANT_NO_MULTI_MMAP is set
* Support opening single mmap in MmapDenseVectors
* Remove unused result response
* Import LazyLock
* Disable rocksdb compile time feature by default
* Also disable RocksDB feature in segment crate
* Enable RocksDB feature in all CI builds
* Remove extra job for testing non-RocksDB build, it's the default now
* Keep RocksDB structures in generated OpenAPI schema
* Fix obsolete --workspace flag breaking builds with explicit features
* Also build including RocksDB in e2e tests on CI
* After clearing WAL, flush segment
* Add debug log when WAL is cleared
* Clear WAL on consensus snapshot after writing state, truncate on start
* Apply consensus snapshot offset
* Fix off by one error
* Tweak debug assertion message
* Change WAL reconciliation condition, and fully clear WAL in this case
* Add debug assertion to prove Raft index and snapshot index are equal
* Add documentation to resolve bot nit
* Return error on WAL clear failure
* Fix typo
* Remove unused truncate functions
* Wrap the tracker and bitmask in Arc
* Implement deferred flusher for gridstore, defer tracker writes
* Remove clone
* Dynamically adjust list of pending updates to drain what is flushed
* Return proper flusher, defer premature Gridstore flushing errors
* Use deferred Gridstore flusher across storages
* Remove all Arc<RwLock<_>> wrappers around Gridstore in storages
* Flush pages inside closure, also wrap them in a lock
* Remove direct flush function from Gridstore
* Add test to assert behavior of deferred flushing in Gridstore
* Add much more extensive test, including concurrent flushes and deletes
* Add test for draining value pointer, make sure to drop when flushed
* Feature gate RocksDB components
* Prematurely abort blocking task in `spawn_cancel_on_drop` on drop
These tasks are intended to be cancellable. Now we prematurely abort the
task if the future was dropped before the task is executed.
* Prematurely abort blocking task in `spawn_cancel_on_token` on cancel
These tasks are intended to be cancellable. Now we prematurely abort the
task if the cancellation token is triggered before the task is executed.
* Prematurely abort blocking task for fetching telemetry
* Prematurely abort stoppable task on drop, all are safe to abort early
* Make `move_dir` either move everything, or nothing at all
That is with the exception of file IO errors in which case data may be
partially moved.
Before this PR it was possible for the new target directory to be
created without moving all data into it. Now we either do all, or
nothing.
* Prematurely abort task for creating full snapshot
It is fine to either create it, or not at all.
* Prematurely abort blocking task for waiting on consensus leader
* Prematurely abort blocking cardinality estimation and shard info tasks
* Prematurely abort blocking point deduplication task
* Prematurely abort blocking task for checking available disk space
* Prematurely abort blocking shard read operations
All shard read operations, such as retrieve, scroll, facets and more can
be safely aborted prematurely.
Related to: <https://github.com/qdrant/qdrant/pull/7530>
* Prematurely abort blocking task for waiting on replica state
* Prematurely abort blocking task for waiting on transfer replica states
* Prematurely abort blocking task for loading segment
This can safely be aborted before the task is started
* Prematurely abort blocking task waiting for replica states
* Prematurely abort blocking task for creating snapshot file
Safe because it aborts before writing any snapshot files to disk
* Add collections_vector_total metric for backwards compatibility
Only this metric was removed since Qdrant version 1.15.5, and so this
adds it back for backwards compatibility. Though it has been marked as
deprecated now.
* Prefix replica metrics with collection, part of collection namespace
* Count running optimizations per collection
* Remove deprecation flag
* Minor optimization, preallocate vectors
* Correctly rebuild hash rings, don't use default with custom sharding
* Correctly initialize hash ring map with custom sharding
Don't load the non key ring by default
* Validate shard key for resharding more aggressively
When using auto sharding, require no shard key. When using custom
sharding, require a shard key.
* Only pass sharding method into shard holder construction
* Remove now obsolete debug assertion, we use same source config
* Correctly reconstruct hash rings, also add shard IDs in auto mode
* Add test, do not break consensus on invalid resharding down operation
* Add tests for resharding down shard count precondition
* Minor test tweaks
* Improve debug assertion
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* In metrics, aggregate page faults
* Include page fault of active children (recursively)
* type alias Pid: i32
* Code improvements
* Ignore already terminated processes
* Clippy
* Also count faults for joined descendant threads in children
* Import std::cmp::min
* Rework child fault recursion, remove second hash set
* Add simple test for iterating child processes
* Fix typo
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Use limit values directly
* We don't include the parent PID
* Don't allocate hashmap when listing process children
* Extend process child PIDs test, fork recursively
---------
Co-authored-by: jojii <jojii@gmx.net>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Add ReadActive replica state
* Add consensus function to switch from Partial to ReadActive and confirm
* On stream records to a different shard ID, switch to ReadActive first
* Update OpenAPI spec
* In stream records peer sync, require all peers to be up-to-date
* Review fixes
* Rename ReadActive to ActiveRead
* update openapi spec
* explicit state checking functions
* typo
* update fallback routing with usage of a new state
---------
Co-authored-by: KShivendu <kshivendu1@gmail.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* Change page fault metrics from gauges to counters
* Change collection_vectors_total to collection_vectors
* Rename metric for page faults in joined child processes
* Rename some procfs metrics
* Show single limit for number of open files
* For number of file descriptors and mmaps follow convential names
* Rename dead_shards_total to dead_replicas
* Fix metric prefix for gRPC entries, add missing underscore
A underscore was missing if no global prefix was set
* Re-sort segments on flush, don't use proxy but inner segment state
* Also sort segments when proxying all segments for snapshot
The snapshot operation explicitly flushes segments. We therefore must
satisfy flush ordering there too.
* Add some helpful comments
* Add explicit flush ordering enum for segments
* Minor tweaks
* Fix spelling in comment
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Explicitly use C memory representation for Gridstore value pointers
* Inline pure function
* Grow tracker file to be next multiple of two
* In debug builds, panic if getting out of bound point from Gridstore
* Also represent region gaps as C struct
* Remove debug assertion
* Adjust file size test
* Disallow peer to join with URI that is already used
* Add test for rejecting peer join with duplicate URI
* Improve peer rejection logic
* Try to rejoin twice, we expect a consistent result
* Remove deprecated vectors count from collection info
* Remove vectors count from shard info
* Update OpenAPI and gRPC spec
* Remove vectors count from example
* Add unit test with proxy that propagates older delete to wrapped segment
* Link to pull request
* Disable faulty debug assertion, replace with comment noting semantics
* Minor tweaks
* Point version check in optimizer makes sense, assert no double segment
* In proxy segment, don't return deleted point versions
More specifically, don't return a point version from the wrapped segment
if that point is already marked for deletion within the proxy.
* Do get wrapped segment point version if it's newer than soft delete
* Flatten function
* When swapping existing segment, remove first to allow segment ID reuse
* Share original segment ID with proxies, keeps original mapping
* Add replace function, that directly replaces existing segment
* Make numeric index storage non optional
* Make map index storage non optional
* Make full text index storage non optional
* Remove unnecessary option
* Make geo index storage non optional
* Make bool index storage non optional
* Remove unnecessary result
* Make null index storage non optional
* Remove load functions from numeric index
* Remove load functions from map index
* Remove unnecessary result and option
* Remove load functions from full text index
* Remove load functions from geo index
* Remove unnecessary result and option
* Remove load functions from bool index
* Remove load functions from null index
* Don't invoke load when loading payload indices, remove load from trait
* Open and load simple bool index in single stage
* Open and load mutable bool index in single stage
* Open and load mutable null index in single stage
* Refactor index selector, don't return early
* Open and load mutable full text index in single stage
* Open and load immutable full text index in single stage
* Open and load mmap full text index in single stage
* In struct payload index open, use option to specify loaded, set rebuild
* Open and load mutable numeric index in single stage
* Open and load immutable numeric index in single stage
* Open and load mmap numeric index in single stage
* new_null_index always returns null index
Missing refactor for MutableNullIndex. Since we have the
`create_if_missing` flag, we no longer have `open_if_exists` function,
ergo, signature for `new_null_index` should not include an `Option`.
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Deprecate init_from
* Mark memmap_threshold as deprecated
* Mark locks API as deprecated
* Mark RBAC collection access payload filter as deprecated
* Allow deprecations in tests and benchmarks
* Add test to catch bug
Bug: <https://github.com/qdrant/qdrant/issues/6880>
* Fix loading null index from wrong path
* Simplify test
* Also bump total point count when removing a point
* Add debug assertion, ensure we don't create null index in selector
* Update bug link
* Remove unused import
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Tim Visée