* Optimize JSON-Proto conversion and add unit tests
* mproved memory allocation by eliminating raw loops
* Enhanced readability by reducing nesting and restructuring logic for clarity
* Added tests to assert correctness and reliability of the JSON-Proto conversion process
* Resolve clippy issues
* Strict mode config: Max collection size
* api specs
* Add tests + set/update payload check
* Improve function names and add comments
* rename config to separate vectors and payload
* fix tests
* Adjust configs docs
* add benchmark
* improve performance by caching shard info
* add bench for size_info() and fix tests
* Also limit the batch-size for vector updates (#5508)
* Also limit the batch-size for vector updates
* clippy
* add lost commit
* Load cache on collection initialization
* add unit type to parameter name
* fix renaming in test
* clearer error message
* fix test
* review remarks
* remove unused function for now
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* Add unit test for point deduplication bug
* Fix point deduplication bug, versions were reused acrosos point IDs
* Add unit test with randomized points and versions
* Remove obsolete comment
* Add link to pull request
* Simplify test a bit
* In proxy segment, apply proxied point delete with its operation version
* Also use correct delete version when propagating changes back to wrapped
* Also delete points with version in segment optimizer build new segment
* Track original/operation point versions separately on proxied deletes
* Fix test compilation failures
* Use explicit types when initializing default
* Keep temporary segment if not empty
* Fix is empty condition
* Swap condition because it's cheaper
* Keep segment if no appendable segment or if not empty
* gpu hnsw construction
* fix build
* are you happy clippy
* are you happy clippy
* review remarks
* more comments
* external gpu vector storage
* decompose gpu inserter
* are you happy codespell
* are you happy clippy
* implement mmap sparse vector storage
* add to VectorStorageEnum
* clippy
* add tests, fix both simple and mmap storages
* smol correction on total_vector_count
* add sparse storage type to config
* fix reading config without storage type
* generate openapi
* use blob_store by path
* hidden setting to enable new storage
* validate existing path in `BlobStore::open()`
* use new dir for each sparse vector name
* fix and rename `max_point_offset`
Plus some extra refactors
* add storage compat test, to always check both storages work
* fix opening of storage + other misc fixes
* FIX!!!
`Unset` operations in the Tracker weren't updating the
`next_pointer_id`. So, when reopening the storage, those points wouldn't
get marked as deleted in the bitslice, thus creating the illusion that
they should exist, when they did not.
* refactor naming from `iter_*` to `for_each_*`
* fix checking for BlobStore existance
* fix typo
* fix error message
* better docs for open_or_create
* fix after rebase
* test to demonstrate inconsistency failure during optimization
* Fix apply points mixing point versions, only apply operation to latest (#5528)
* Patch apply points, only apply point operations to latest point version
* Simplify test, we don't need owned strings in JSON macro
* Use ahash in apply points because we map to simple numbers
* Minor improvements
* preserve the update version before doing move_if_exists in proxy (#5531)
* preserve the update version before doing move_if_exists in proxy
* Simplify move_if_exists with flat if statements and early return
* Fix inverted condition, consider deleted if delete is new
* Use hash map keys iterator as more idiomatic Rust
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: timvisee <tim@visee.me>
* parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012991 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012977 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012901 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012766 +0100
parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012730 +0100
vulkan api wrapper crate
fmt
gpu is not a workspace member
exclude gpu from workspace
disable tests for gpu crate
ash version up
change collection for pipeline builder
gpu test
gpu is not optional
check device features
remove feature
make all fields private
more comments
are you happy fmt
qdrant team as authors
gpu vectors
move changes from magic branch
move storage updates
are you happy codespell
are you happy clippy
don't make storage_impl public
review remarks
are you happy clippy
are you happy fmt
* Update lib/segment/src/index/hnsw_index/gpu/gpu_vector_storage/tests.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* review remarks
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Expose storage size of RocksDB payload storage
* no strict check as numbers vary on various OS
* loose assertions
* try again for Windows
* assert storage size range
