* Separate set and unset lists for each Gridstore pointer update
This makes the list of pointer updates much easier to grasp. The
simplification is desired because this exact structure has been a cause
for bugs multiple times now.
* Add more aggressive debug assertions
* Use consistent terminology
* Correct removal of set
* Simplify drain function
* Add safe guard to ignore empty pointer updates
* Rework tracker getter, explicitly branch variants
* Patch transmute, require Sized type
* rename and adjust comments
- set/unset is now current/to_free
- adjusted descriptions and comments for this nomenclature too
* clippy
* clippyyy
not 4, not 2, but 3 spaces
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Fix delete_page not dropping sequential mmap
* In Gridstore Page, support using a single mmap
* In Gridstore Page, only open multiple mmaps if supported at runtime
* Support opening single mmap in UniversalMmapChunk
* Rename environment var, log warning when QDRANT_NO_MULTI_MMAP is set
* Support opening single mmap in MmapDenseVectors
* Remove unused result response
* Import LazyLock
* Add function to detect if storage is using RocksDB
* Add feature flag to toggle payload index migration
* Add naive payload index migration, rebuild from scratch if RocksDB
* Adjust payload index rebuild logs
* When rebuilding payload indices, clean up existing storages
* Update OpenAPI spec
* Use weak references to Gridstore in mutable payload index flushers
* Rework payload index clear/wipe functions, make them consistent
* Naively migrate RocksDB before loading indices
* Fix mmap payload index load, staged like others (#6837)
* Add staged loading of mmap numeric payload index
* Add staged loading of mmap map payload index
* Add staged loading of mmap geo payload index
* Add staged loading of mmap full text payload index
* Add staged loading of mmap bool and null payload index
* Minor tweaks
* Don't fail clearing file cache if file does not exist
---------
Co-authored-by: jojii <jojii@gmx.net>
* check filesystem on start and do some simple mmap test to make sure it is persisted
* fmt
* fix fs type read on non-linux
* feature-flag for fs-check supported platforms + fs check with sub-folder
* fmt
* try larger magic file
* minor fixes
* fix compile on windows
* Add FAT, exFAT and APFS file systems
* Update lib/common/memory/src/checkfs.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
* don't propagate offset into prefetches
* I want to see CI with that change
* Move import into platform specific scope
* edit test to make sure offset is not propagated
* fix planned query test
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: timvisee <tim@visee.me>
* wip: implement explicit populate and clear_cache functions for all components
* fmt
* implement clear and populate for vector storages
* fmt
* implement clear and populate for payload storage
* wip: implement explicit populate and clear_cache functions payload indexes
* implement explicit populate and clear_cache functions payload indexes
* fix clippy on CI
* only compile posix_fadvise on linux
* only compile posix_fadvise on linux
* implement explicit populate and clear_cache functions for quantized vectors
* fmt
* remove post-load prefault
* fix typo
* implement is-on-disk for payload indexes, implement clear on drop for segment, implement clear after segment build
* fmt
* also evict quantized vectors after optimization
* re-use and replace advise_dontneed
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* vulkan api wrapper crate
* fmt
* gpu is not a workspace member
* are you happy codespell
* try to fix CI
* exclude gpu from workspace
* fix build for macos
* disable tests for gpu crate
* fix codespell
* provide gpu feature into gpu trait
* ash version up
* change collection for pipeline builder
* gpu test
* gpu is not optional
* check extensions and layers presense
* check device features
* remove feature
* make all fields private
* more comments
* shader compilation options
* are you happy clippy
* are you happy fmt
* Update lib/gpu/Cargo.toml
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* qdrant team as authors
* merge more changes from magic branch
* are you happy clippy
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* Optimize mmap sequential access for large scoring
* use it
* make windows happy
* add missing CPU counters
* Will not need
* more unix build
* add benchmark
* fix and bench cmp
* larger batch size
* start appendable mmap implem
* reenable debug for bench
* remove old impl for comparison
* impl for chunk mmap
* fix windows
* debug assert
* errare human est
* windows cfg
* generalize peek_top_iter
* remove blanket implementation to force explicit decision
* less TODO
* account for size of T
* fmr
* simplify chunk impl
* relax constraint on contiguous ids
* remove madvice WILLNEED and clean batching infra
* some premature optimizations
* extra sequential mmap for MMapVectorStorage
* add read-only typed mmap
* add read-only typed mmap
* fix comments
* implement batch reads for chuncked mmap
* fix is_read_with_prefetch_efficient
* cleanup
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* initial structures for reading mmap text index
* creation of text index mmap
* WIP create mmap inverted index
* use same file writing technique as in MmapHashmap
* Save points_to_tokens_count in two files
* clippy
* fill slice with iterator
* review fixes
* fix after review
* implement loading of MmapInvertedIndex
* WIP: prepare for converting ChunkReader into trait
* fmt
* WIP: prepare ChunkReader to be a trai
* WIP: replace ChunkReaderImpl with trait
* implement chunk reader for mmap view
* rollback ChunkReader trait and remove CompressedMmapPostingListView
* move ImmutableInvertedIndex and MutableInvertedIndex into separate files
* impl ops for MmapInvertedIndex
* make mmap hashmap value param generic (#5042)
* make mmap hashmap value param generic
* test + fixes
* Update lib/common/common/src/mmap_hashmap.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Update lib/common/common/src/mmap_hashmap.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* fixups
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* test and fix immutable to mmap conversion
* fix case 4 and refactor MmapBitSlice::create
* more tests
* fmt
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* refactor: introduce AdviceSetting for open_read_mmap/open_write_mmap
* perf: use Advice::Normal in a few places
* perf: use MADV_DONTNEED after reading
* perf: use POSIX_FADV_DONTNEED after reading
* make mmap_type independent from segment structures
* make bitvec and thiserror workspace dependencies
* move mmap_type into common/memory subcrate
* fmt
* Assert data alignment in `transmute_from_u8*` functions
* WIP: Switch `transmute_from_u8*` alignment asserts from `debug_assert` to `assert`...
...to make sure *all* tests running on CI will enforce the alignment
* Add descriptive message to alignment assertions
* Fix alignment in graph links file (#3807)
* fix alignment in graph links file
* fix alignment while reading
* Revert "fix alignment while reading"
This reverts commit e2d1cee890.
* Revert "Revert "fix alignment while reading""
This reverts commit 7a4fdc9aea.
* small refactor
* Switch `transmute_from_u8*` alignment asserts back to `debug_assert`
Additionally:
- Remove extra `0x` from slice address in assert messages
* Trim offsets padding bytes from the end of `links_mmap` slice
---------
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* add `truncate(true)` where unspecified
* remove unused `WriteGuard` wrapper
* Don't truncate if we explicitly set the size later
---------
Co-authored-by: timvisee <tim@visee.me>
