* nested object filter
* code review
* add support for must_not in nested
* extract functions
* support and test must_not in SimpleConditionChecker
* add index matching unit test (to be continued)
* remove extra clone
* test with should
* WIP: Nested object filter suggestions (#1855)
* switch to bitvec
* fix clippy
* more tests
* fmt
* fix some tests
* add test with text
* support for nested should
* do not rely on indexes for nested queries & fix test
* use index to make index-aware checks in nested payload
* fix value-count tests
* re-fa-cto-ring
* fmt
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Enable alignment assertion on Windows, error on Windows if mmap is empty
* We only need std::io on Unix platforms
* Sync after resizing file
* Don't test empty mmaps on Windows, add test to assert this edge case
* Test emtpy bitslice on non-Windows platforms
* Clean-up
* Prevent divide by zero panics for zero sized mmap types
* Support zero-sized mmap types on all platforms, extend test cases
- fix divide by zero error for zero-sized types
- zero-sized types are now supported
- on Windows, don't error on empty mmap for zero-sized type
- extend tests to cover all cases
Windows does not support mmap on empty files. Instead of returning an
error on Windows for this specific case, we return an empty type or
slice at a dangling pointer that is properly aligned. That makes the
behavior of the typed mmap consistent across all platforms.
* wip: chunked mmap
* Fix typo
* insert and get methods
* dynamic bitvec
* clippy
* wip: vector storage
* wip: fmt
* wip: mmap chunks
* wip: mmap problems
* Share transmuted mutable reference over mmap
* option to enable appendable mmap vectors
* fmt
* rename storage status file
* update tests
* fix get deleted value range
* add recovery to vector storage tests
* add flush to tests
* fix transmute from immutable to mutable
* make transmuted pointer private
* remove unused unsafe functions
* force WAL flush if wait=true
* move wal flush into updater thread
* remove flush from update api
* Minimize pub visibility for specialized/dangerous functions
* Allocate vector with predefined capacity
* Inline format parameters
* Assert we have multiple chunks while testing, test is useless otherwise
* Remove unnecessary scope
* Remove unnecessary dereference
* Random bool has 0.5 as standard distribution, use iter::repeat_with
* Replace RemovableMmap::new with Default derive
* Rename len to num_flags
* Use Option replace as it is convention alongside take
* Add FileId enum to replace error prone manual ID rotating
* Use debug_assert_eq where applicable
* Refactor drop and set to replace
* Change default chunk size for chunked mmap vectors to 32MB
This change is made as per GitHub review, because allocating a few
storages with 128MB would take a significant amount of time and storage.
See: https://github.com/qdrant/qdrant/pull/1838#discussion_r1187215475
* Replace for-loops with iterators
* Draft: add typed mmap to improve code safety (#1860)
* Add typed mmap
* Replace some crude mmap usages with typed mmap
* Use typed mmap for deleted flags
* Simplify dynamic mmap flags a lot with new typed mmap, remove flags option
* Reformat
* Remove old mmap functions that are now unused
* Reimplement mmap locking for mmap_vectors
* Add MmapBitSlice tests
* Replace MmapChunk with new typed mmap
* Update docs
* Clean-up
* Disable alignment assertions on Windows for now
* Rename mmap lock to mlock to prevent confusion with lockable types
* one more small test
* Some review fixes
* Add aliasing note
* Add basic error handling in typed mmap constructors
* Use typed mmap error handling throughout project
* Move mmap type module to common
* Fix transmute functions being unsound
See https://github.com/qdrant/qdrant/pull/1860#discussion_r1188593854
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
* Use custom build profile for docker builds
* trigger ci
* change source code to trigger ci with rebuild one more time
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add caching of docker layers in CI
Build required docker images for CI in a workflow step using buildkit's
gha cache type. This will populate the local layer cache from github
actions' cache. Builds in subsequent CI steps will be nearly instant,
because all layers can be reused.
* add minor change to see if build time is any faster
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Add update/delete REST endpoints
* Add update/delete vector REST endpoints to OpenAPI specification
* Fix update vector with no changes not triggering a validation error
* Update OpenAPI specification for updated request type
* Add segment entry function to update named vectors
* Use already available function to update existing vectors
We already had a segment function to update existing named vectors. This
change ensure we use that instead of separating it separately. As a
bonus, this adds support for setting multiple named vectors at once.
* Update set vectors ourselves, don't drop omitted vectors
* Refactor vector updating functions, separate update and replace
* Add basic vector ops, add update/delete functionality to segment updater
* Add internal and public gRPC types and actions for vectors
* Add gRPC API actions
* Reformat
* Add VectorOperations to vector ops, add basic validation
* Validate gRPC vector types
* Validate vector operation structs
* Construct PointIdsList through From trait
* Update gRPC docs
* Use VectorsSelector for vector deletions in gRPC
* Add support for updating multiple points/vectors in update vectors API
* Update gRPC docs
* Fix incorrect gRPC type numbering
* Return point ID error from vector update/delete functions if not found
* Fix disbalanced vectors test
* Remove unnecessary mut
* Simplify creating option with if
* Simplify some returns
* DRY in vector name checking functions
* Use panic with attribute rather than debug_assert false
* Make update vector function plural because we can update multiple
* WIP: Start working on out-of-RAM errors handling [skip ci]
* Implement basic handling of out-of-RAM errors during Qdrant startup
* Try to fix CI fail by allowing both V1 and V2 cgroups
* Try to fix CI fail by improving cgroups handling
* Fix cgroups path detection/handling (+ some minor stylistic changes)
* fixup! Fix cgroups path detection/handling (+ some minor stylistic changes)
* Add test
* Enable low RAM test
* fixup! Add test
* free memory checks
* rm unused function
* Oom fallback script (#1809)
* add recover mode in qdrant + script for handelling OOM
* fix clippy
* reformat entrypoint.sh
* fix test
* add logging to test
* fix test
* fix test
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Minor collection optimizer cleanup
* Make optimizers better aware of available vs soft deleted points
* Fix incorrect deleted state on proxy segment for double delete
* Rename upsert_vector to upsert_point, because we work with points
* Refactor point methods for more clear and consistent naming
* Replace internal_size in IdTracker with total_point_count
* Keep track of vector deletion count on storage creation
* Add sparse index optimizer, to optimize indexes with high deletion count
* Add minimum vector count threshold to sparse index optimizer
* Add sparse index optimizer test
* Use consistent naming, write vector in full everywhere
* Simplify vacuum optimizer a bit
* Merge sparse index optimizer into vacuum optimizer
* Improve update_from in segment builder by returning early
* More accurately count vectors in segment optimizer
* Remove random from vacuum optimizer tests to make them more reliable
* Don't expose the total points in segment info, use available points
* Process review feedback
* Compare available vectors against indexed ones in vacuum optimizer
This is much better than using the number of soft-deleted vectors when
the segment was created for calculations. Not to mention that value had
other problems as well.
* Remove create_deleted_vector_count field, update vacuum test parameters
* Potentially solve out of bound panic when building index
* Review fixes:
- Propagate deleted flags into payload hnsw building
- Use `total` number of points for building HNSW instead of number of
available points
- minor refactoring of `hnsw_config` copy -> clone
- Better detection of `indexed_points` in HNSW
* fix assert condition
* Optional named vectors optimizer reveiw 2 (#1794)
* review with Ivan
* fmt
* remove available_vector_count from segment entry
* remove total_point_count from segment entry
---------
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* rollback changes in deleted count in proxy segment
* improve vector threshold detection logic in optimized_segment_builder
* style changes
* fix propagate deleted points to vectors
* Fix typo in method name
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* Exclude deleted vectors from HNSW graph building stage
* When estimating query cardinality, use available points as baseline
We should not use the total number of points in a segment, because a
portion of it may be soft deleted. Instead, we use the available
(non-deleted) points as baseline.
* Add plain search check to unfiltered HNSW search due to deleted points
* Cardinality sampling on available points, ignore deleted named vectors
* Estimate available vectors in query planner, now consider deleted points
In the query planner, we want to know the number of available points as
accurately as possible. This isn't possible because we only know the
number of deletions and vectors can be deleted in two places: as point
or as vector. These deletions may overlap. This now estimates the number
of deleted vectors based on the segment state. It assumes that point and
vector deletions have an overlap of 20%. This is an arbitrary
percentage, but reflects an almost-worst scenario.
This improves because the number of deleted points wasn't considered at
all before.
* Remove unused function from trait
* Fix bench compilation error
* Fix typo in docs
* Base whether to do plain search in HNSW upon full scan threshold
* Remove index threshold from HNSW config, only use full scan threshold
* Simplify timer aggregator assignment in HNSW search
* Remove vector storage type from cardinality function parameters
* Propagate point deletes to all its vectors
* Check for deleted vectors first, this makes early return possible
Since point deletes are now propagated to vectors, deleted points are
included in vector deletions. Because of that we can check if the vector
is deleted first so we can return early and skip the point deletion
check.
For integrity we also check if the point is deleted, if the vector was
not. That is because it may happen that point deletions are not properly
propagated to vectors.
* Don't use arbitrary vector count estimation, use vector count directly
Before we had to estimate the number of vectors (for a named vector)
because vectors could be deleted as point or vector. Point deletes are
now propagated to vector deletes, that means we can simply use the
deleted vector count which is now much more accurate.
* When sampling IDs, check deleted vecs before deleted points
* On segment consistency check, delete vectors for deleted points
* Fix vector delete state not being kept when updating storage from other
* Fix segment builder skipping deleted vectors breaking offsets
* update segment to handle optional vectors + add test (#1781)
* update segment to handle optional vectors + add test
* Only update stored record when deleting if it wasn't deleted already
* Reformat comment
---------
Co-authored-by: timvisee <tim@visee.me>
* Fix missed vector name test, these are now marked as deleted
* upd test
* upd test
* Update consensus test
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Use resize rather than while-push loop
* Add deleted flags to simple vector storage
* Add deleted flag to memmap vector storage
* Map BitSlice on mmap file for deleted flags
* Use vector specific deletion BitSlice in RawScorer
* Use BitSlice for deleted points, fix check point logic, clarify names
* Extract div_ceil function to shared module
* We can use unchecked set and replace because we just checked the length
* Add deleted count function to vector storage
* Add vector storage point deletion tests
* Keep deleted state in simple vector storage with update_from, add test
* Keep deleted state in memmap vector storage with update_from, add test
* Simplify div_ceil
* Improve deletion handling in update_from in mmap vector storage
* Improve performance, use trickery to get BitSlice view over deleted mmap
* Use BitSlice where possible, construct BitVec more efficiently
* Incorporate vector specific delete flags in quantized raw scorer
* Don't pin MmapMut, it is not required
* With quantization, keep mmap deleted flags in RAM for better performance
* Advice the kernel to prepare deleted flags mmap for faster future access
* Simplify deleted bitslice access, add bound check, remove unused function
* Fix compilation on Windows
* Cleanup
* Rename delete functions to delete_{point,vec} to prevent confusion
* Use then_some rather than match a boolean
* Lock deleted flags in memory only when quantization is available
* Add docs and stabilize issue link to dev_ceil
* Flush deleted mmap when closing segment
This requires us to to wrap the memory map struct in an Arc and Mutex.
Though this may look inefficient, it doesn't have a negative side effect
on deleted flag performance, because the flags are accessed through a
BitSlice that is separate and doesn't use locking.
* Rename some point functions to vec because that makes more sense
* Simplify delete flag fetching option, use deref func instead of asterisk
* Do not calculate slice size manually, use size_of_val
* remove test raw scorer
* use deref in check
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Change BTreeSet/BTreeMap to HashSet/HashMap
Use Vec in Document
Use Vec in Postinglist
* fix document removal index out of range
* add common vocabulary dictionary to inverted index
* use radix-trie to store vocabulary
* add documentation for seemingly dangerous unwraps
* Implement BitVec representation of PostingList
Add dynamic switching between Vec and BitVec representation
Split PostingList into its own module
* use same persistent storage for documents
* clean up
* remove unused comments
* keep stored document format the same
* add reverse dictionary for faster document building
* get rid of reverse vocabulary
* review + fix tests
* review + fix clippy
* fix typo
* use u32 as token id
* fmt
* remove patricia_tree
* fmt
* only use vec for posting
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* chore: fix clippy warnings
- Prefer `unwrap_or_else` over `unwrap_or` as the later is always executed.
- Use `clamp`.
* chore: convert `default_quantization_ignore_value` and `default_quantization_rescore_value` to `const fn`
* chore: revert `clamp` change
* Add QuantizationConfigDiff type
* Add quantization config diff to vector parameters
* Prefer vector config over collection config for quantization
* Update OpenAPI specification
* Validate quantization configuration quantile in 0.5-1.0 range
As per https://github.com/qdrant/qdrant/pull/1681
* Add test if check if vector specific quantization config is persisted
* Alias quantization to quantization_config in vector parameters
* Remove quantization config diff, use full vector specific config instead
* Regenerate OpenAPI specification and gRPC docs
* Fix compilation error
* Add error handling to quantization config conversions
* Fix quantization integration test, make HNSW test stricter
* Change file mode of some .sh files to make them executable
* Remove whitespace on empty lines from YAML configurations
* Fix unused import warning on Windows
* Validate VectorConfig/VectorParams, remove obsolete validation
* Add HNSW config diff to vector parameters
* Validate params in collection config
* Add HNSW config to segment vector data config
* Add VectorsConfig params iterator for more elegant conversions
* Prefer vector HNSW config over collection config for building HNSW index
* Base segment vector param HNSW config on collection config
* General improvements
* Rewrite HNSW ef_construct extract function to also consider vector configs
* Update OpenAPI specification
* Add test to check if vector specific HNSW config is persisted
* review changes
* review changes
* Regenerate gRPC docs
* Fix test on Windows
* Regenerate OpenAPI specification
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Verify quantization configuration on collection creation
* Update OpenAPI specification
* Update range in doc to match validation definition
* Update OpenAPI specification
* use common channel pool for healthcheck requests
* add dynamic channel pool
* fmt
* lower parallel connections num
* fmt
* fix clippy
* fix clippy
* better status error message
* smarter channel dropping conditions
* per-channel stats
* refactor pool
* rollback dockerfile
* health-check timeout
* prevent reporting last active peer as dead
* introduce update rate limit
* only rate-limit client requests
* ability to save snapshot without locking wal (#1685)
* ability to save snapshot without locking wal
* fix empty wal saving
* skip waiting on local shard in listener mode even if it is a direct request to listener shard
* snapshot recovery test + better handling of segment versions
* use latest wal
* review changes
* actix validation
* add check for memmap/indexing_threshold
* fix actix json settings
* Validate settings configuration on start, print pretty warnings on fail
* Add more validation rules for settings and nested types
* Move shared validation logic into collection/operations
* Show validation warning in log when loading some internal configs
* Show prettier actix JSON validation errors
* Stubs for pretty handling of query errors, reformat validation errors
* Use crate flatten function, the hard work was already done for us
We don't have to flatten validation errors into their qualified field
names ourselves because there is a utility function for this.
* Configure actix path validator
* Actix endpoints don't require public
* Extend validation to more actix types
* Validate all remaining actix path and query properties
* Rephrase range validation messages to clearly describe they're inclusive
* Validate all query params to respond with pretty deserialize errors
* Nicely format JSON payload deserialize error responses
* Improve error reporting for upsert point batches
* Add basic validation test that checks a path, query and payload value
* Add some validation constraints
* Add simple validation error render test
* Update Cargo.lock
---------
Co-authored-by: timvisee <tim+github@visee.me>
* add minimal working is_null filter
* add is_null condition to grpc api (backward compatible)
* add unit tests is_null and is_empty conditions
* add is_null to points.proto file
* add some failing OpenAPI tests
* fix a failing test due to change in collection data
* refactor MultiValue's check for is_null
* fix is_empty condition not picking up "key":[]
* remove duplicate OpenAPI integration test
* reuse same variable in condition checker tests
* update grpc docs
* fix is_null cardinality estimation to match is_empty
* update openapi specs
* remove unused debug statements
* add new test points to original test_collection
* fix failing tests according to newly added points
* add the `"key":[null]` test_case
