* On segment flush, read-lock all segments to prevent CoW between flushes
For example, we have a point on an immutable segment. If we use a
set-payload operation, we do copy-on-write. The point from immutable
segment A is deleted, the updated point is stored on appendable segment
B.
Because of flush ordering segment B (appendable) is flushed before
segment A (not-appendable). If the copy-on-write operation happens in
between, the point is deleted from A but the new point in B is not
persisted. We cannot recover this by replaying the WAL in case of a
crash because the point in A does not exist anymore, making
copy-on-write impossible.
Locking all segments prevents copy-on-write operations from occurring in
between flushes.
* Return proper status on set payload operations in segment
* Disable propagating point deletions to its vectors, it looses vectors
* Update vector tests after disabling point delete propagation to vectors
* Implement retry with exponential backoff for read-locking all segments
* Use try_read_for rather than sleeping after lock attempts
* Simplify locking many, just lock one by one
Refs: <https://github.com/qdrant/qdrant/pull/2527>
Co-authored-by: generall <andrey@vasnetsov.com>
* Comment out now obsolete test
* Error handling in method getting segment locks by ID
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Clean up imports
* Clean up whitespace
* Use drop collection function from helpers
* Use parameterized integration tests to test on_disk variations
* Promote on_disk_vectors parameter into dedicated fixture
* Merge basic retrieve test with parameterization
* Flatten functions
* Split validation test functions
* Don't parameterize on_disk states when testing validation
* Parameterize init_from_collection test sequence
* Parameterize on_disk_payload in tests with fixture
* Parameterize new batch update tests
* Fix integration test collection setup distance after incorrect merge
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
* wip: re-implement http tests using openapi validation library
* collection delete api
* payload operations test
* payload delete operations test
* payload index creation and removing test
* uuid operations test
* schema consistency test
* add tests into pipeline
* chmod +x
* faster local runs with docker cache
* upd instruction
