mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-06 10:00:58 -05:00
2de122adea
* implement mmap sparse vector storage * add to VectorStorageEnum * clippy * add tests, fix both simple and mmap storages * smol correction on total_vector_count * add sparse storage type to config * fix reading config without storage type * generate openapi * use blob_store by path * hidden setting to enable new storage * validate existing path in `BlobStore::open()` * use new dir for each sparse vector name * fix and rename `max_point_offset` Plus some extra refactors * add storage compat test, to always check both storages work * fix opening of storage + other misc fixes * FIX!!! `Unset` operations in the Tracker weren't updating the `next_pointer_id`. So, when reopening the storage, those points wouldn't get marked as deleted in the bitslice, thus creating the illusion that they should exist, when they did not. * refactor naming from `iter_*` to `for_each_*` * fix checking for BlobStore existance * fix typo * fix error message * better docs for open_or_create * fix after rebase
Storage compatibility test
In order to detect quickly breakage of storage compatibility, we check that the current code understands the storage format from the latest stable release.
To not burden the git repository with tracking the large binary files, we are pushing storage archives to our GCP bucket.
As features are added, the storage format may change. This means updating the reference storage data and snapshot. This is done by running the gen_storage_compat_data.sh script.
Regenerate storage data
Follow those steps to recreate the reference storage data and snapshot.
- run
./tests/storage-compat/gen_storage_compat_data.sh - make sure to pick the right version when asked for which system generated the files
- push the new archives to the GCP bucket (ask for the credentials if you don't have them)