* use pageout to clear mmap cache
* Also clear cache of deleted flags in mmap dense vector storage
* Add reference to madvise man pages for probe logic
* use deconstruct
---------
Co-authored-by: timvisee <tim@visee.me>
* [ai] Replace manual into mappings with Into::into
* Reformat
* [ai] Use implicit .iter
* Don't iterate over keys too
* [ai] Replace unwrap_or
* Reformat
* [ai] Use as_deref and then_some
* [ai] Use more to_string
* [ai] Use explicitly typed into conversions
* Reformat
* [ai] More explicit into conversions
* Reformat
* Don't insert deferred points into sparse index
# Conflicts:
# lib/segment/src/segment/tests.rs
# lib/segment/src/segment_constructor/segment_constructor_base.rs
# Conflicts:
# lib/segment/src/segment_constructor/segment_constructor_base.rs
* Clippy
* Assert consistency of deferred_internal_id var
* Return before SparseVector conversion in case of deferred point
* Use debug_assert instead
* Use array_windows when statically sized windows were used
* Bump MSRV to 1.94
* fix in gpu code
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Unify parking_lot/arc_lock feature
* Move lib/common/{io,memory}/* -> lib/common/common/*
- Mmap-related items are grouped into `common::mmap` sub-module:
- `memory/src/chunked_utils.rs` -> `common/src/mmap/chunked.rs`
- `memory/src/madvise.rs` -> `common/src/mmap/advice.rs`
- `memory/src/mmap_ops.rs` -> `common/src/mmap/ops.rs`
- `memory/src/mmap_type_readonly.rs` -> `common/src/mmap/mmap_readonly.rs`
- `memory/src/mmap_type.rs` -> `common/src/mmap/mmap_rw.rs`
- Filesystem-related items are grouped into `common::fs` sub-module:
- `common/src/fs.rs` -> `common/src/fs/sync.rs`
- `io/src/file_operations.rs` -> `common/src/fs/ops.rs`
- `io/src/move_files.rs` -> `common/src/fs/move.rs`
- `io/src/safe_delete.rs` -> `common/src/fs/safe_delete.rs`
- `memory/src/checkfs.rs` -> `common/src/fs/check.rs`
- `memory/src/fadvise.rs` -> `common/src/fs/fadvise.rs`
- Rest is moved straight into `common`:
- `io/src/storage_version.rs` -> `common/src/storage_version.rs`
The old `io` and `memory` are now hollow crates that re-export items
from `common`. These hollow crates will be removed in next commits.
* Replace uses of `io` and `memory` with new paths in `common`
Since `io` and `memory` are just re-exports of `common`, these
replacements are no-op.
* Remove `io` and `memory` crates
* Implement `Optional<T>` type
Define a type with the same presumed layout as `Option<T>`, but with defined behavior.
* Make `transmute_*` functions unsafe
The functions `memory::mmap_ops::transmute_*` are inherently unsafe, but
are not marked as are. Their usage is documented, but it is not always clear
if the code is correct.
* Add `CsrHeader` to resolve another unsoundness
Tuples have no defined layout.
* wip: generalization trait for queries
* implement generalization for point operations
* fmt
* log priority queue
* wip: SlowRequestsListener
* fmt
* fix clippy
* simplify generalization
* fmt
* implement collection of requests profiles for update API
* implement API for viewing slow requests log
* add collection name to update worker
* add datetime to log
* fmt
* probabilistic counter of unique requests
* rename
* compute hash before converting into json value
* move logable out of generalizable
* fmt
* log query request
* fmt
* some fixes
* move measurement into local shard
* fmt
* upd openapi (not important)
* For enum variants, has discriminant
* Make SearchParams Copy
* Hash 0.0 and -0.0 the same
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Correctly hash enum variants and float values
* Hash through ordered float instead
* Fix priority queue not keeping longest request for hash
* SearchParams implements Copy
* Fix clippy warning
* Add unordered_hash_unique
* skip serialization if none
* Use OrderedFloat for hashing a float
* Use OrderedFloat for hashing a float
* only log updates if they are performed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* wip: implement explicit populate and clear_cache functions for all components
* fmt
* implement clear and populate for vector storages
* fmt
* implement clear and populate for payload storage
* wip: implement explicit populate and clear_cache functions payload indexes
* implement explicit populate and clear_cache functions payload indexes
* fix clippy on CI
* only compile posix_fadvise on linux
* only compile posix_fadvise on linux
* implement explicit populate and clear_cache functions for quantized vectors
* fmt
* remove post-load prefault
* fix typo
* implement is-on-disk for payload indexes, implement clear on drop for segment, implement clear after segment build
* fmt
* also evict quantized vectors after optimization
* re-use and replace advise_dontneed
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* get total sparse size from inverted index
* populate on load if not present
* use debug assertion
* correct size calculation
* r u happy clippy
* fix total sparse size in immutable ram
* don't use options. move size_of_available_vectors to other traits
* adjust test
* fix rebase
* implement mmap sparse vector storage
* add to VectorStorageEnum
* clippy
* add tests, fix both simple and mmap storages
* smol correction on total_vector_count
* add sparse storage type to config
* fix reading config without storage type
* generate openapi
* use blob_store by path
* hidden setting to enable new storage
* validate existing path in `BlobStore::open()`
* use new dir for each sparse vector name
* fix and rename `max_point_offset`
Plus some extra refactors
* add storage compat test, to always check both storages work
* fix opening of storage + other misc fixes
* FIX!!!
`Unset` operations in the Tracker weren't updating the
`next_pointer_id`. So, when reopening the storage, those points wouldn't
get marked as deleted in the bitslice, thus creating the illusion that
they should exist, when they did not.
* refactor naming from `iter_*` to `for_each_*`
* fix checking for BlobStore existance
* fix typo
* fix error message
* better docs for open_or_create
* fix after rebase
* Add HardwareCounterCell and counting for sparse plain search
* Add measurement for indexed sparse vector
* add tests for calculations
* move SegmentQueryContent higher in the call stack
* move hardware counters inside the query context
* fix clippy
* Fix applying hardware measurements and add test for this
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* initial structures for reading mmap text index
* creation of text index mmap
* WIP create mmap inverted index
* use same file writing technique as in MmapHashmap
* Save points_to_tokens_count in two files
* clippy
* fill slice with iterator
* review fixes
* fix after review
* implement loading of MmapInvertedIndex
* WIP: prepare for converting ChunkReader into trait
* fmt
* WIP: prepare ChunkReader to be a trai
* WIP: replace ChunkReaderImpl with trait
* implement chunk reader for mmap view
* rollback ChunkReader trait and remove CompressedMmapPostingListView
* move ImmutableInvertedIndex and MutableInvertedIndex into separate files
* impl ops for MmapInvertedIndex
* make mmap hashmap value param generic (#5042)
* make mmap hashmap value param generic
* test + fixes
* Update lib/common/common/src/mmap_hashmap.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* Update lib/common/common/src/mmap_hashmap.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* fixups
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* test and fix immutable to mmap conversion
* fix case 4 and refactor MmapBitSlice::create
* more tests
* fmt
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* refactor: introduce AdviceSetting for open_read_mmap/open_write_mmap
* perf: use Advice::Normal in a few places
* perf: use MADV_DONTNEED after reading
* perf: use POSIX_FADV_DONTNEED after reading
* Move build_index out of VectorIndex
* Build index in HNSWIndex::open()
* Introduce HnswIndexOpenArgs
* Proper deletion
* Improve tests
* HNSW::open(): add warn, comment and assert
* Revert to making up the config if it does not exist
* Rename InvertedIndex* -> InvertedIndexCompressed*
* Extract method VectorIndexEnum::fill_idf_statistics
* Extend VectorIndexEnum with new variants
* Introduce sparse::InvertedIndex::Version
* Replace SparseVectorIndexVersion -> InvertedIndex::Version
* Introduce sparse_vector_index::OpenArgs
* SparseVectorIndex::open: do not build index if directory is empty
Otherwise it would build the index twice since `SegmentBuilder::build()`
calls `::open()`, then `::build_index()`. This restores the old (<=v1.9)
behavior.
* Renames
* include old vector into update function and clean posting lists accordingly
* add integration test
* fix counter
* also remove old vector if the insertion is empty
* clippy
* borrow once
* fix max_next_weight correcton on delete + test
* vector index responsible for updating vector storage
* review fixes
* add debug assert
