- Rename `GraphLinksConverter` to `GraphLinksSerializer`.
- Extract some structs from `graph_links.rs` into additional files:
- `graph_links/header.rs`: `HeaderPlain` and `HeaderCompressed`.
- `graph_links/serializer.rs`: for `GraphLinksSerializer`.
- `graph_links/view.rs`: `GraphLinksView`.
- In `header.rs`, remove `HeaderCompressed::version` doc comment about
`HEADER_VERSION_COMPRESSED` as being outdated after merging #5749.
- Add doc comment to `GraphLinksView`.
- In `view.rs`/`GraphLinksView`, move some associated functions to the
module level:
- `read_level_offsets`
- `get_slice`
- `error_unsufficent_size`
* Measure read io for payload storage
* Add Hardware Counter to update functions
* Fix tests and benches
* Rename (some) *_measured functions back to original
* Add limits for filter and conditions
* clippy
* Review remarks + nested condition test
* Fix opnapi specs
* Improve error message by giving info about limits and usage
* fix load of the graph links mmap
* decide compression based on filename rather than bit hacking
* fix test
* fix conversion: always create a new file for compressed links
* fix conversion: adjust compressed flag after conversion
* fix test (again)
* Fixes
* let GraphLayers manage its own files
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Strict Mode: distributed checking of max collection size
* add size projections in distributed mode
* Add consensus tests
* New Test: All nodes in cluster
* fix tests
* Update lib/collection/src/collection/mod.rs
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* increase upsert delay
* add TODO for resharding
* wait for strict mode config to be applied on second node
* remove delays
* Also wait for strict mode in other test
* clearify strict mode config option
---------
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
* use mmap bitslice and a metadata file to avoid loading
* fix and test reported files
* add new files to flusher
* update rocksdb+blob_store compatibility test
* switch to DynamicMmapFlags
* fix setting deleted to `false` on short bitslice
* remove `MmapBitSlice::extend`
* add some more tests to mmap sparse vector storage
* calculate deleted_count and next_point_offset on load
* get rid of metadata file
* cleanup
* fix test
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* get total sparse size from inverted index
* populate on load if not present
* use debug assertion
* correct size calculation
* r u happy clippy
* fix total sparse size in immutable ram
* don't use options. move size_of_available_vectors to other traits
* adjust test
* fix rebase
* add on_disk option for bool index
* test that all files are covered
* generate openapi and docs
* clippy
* remove `populate` changes
* use `walkdir` crate
* Apply clippy suggestions
---------
Co-authored-by: timvisee <tim@visee.me>
* allow to read related issues with collection-level access
* refactor to use requirements
* patch test
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* add `MmapBitSlice::extend` helper
* mmap bool index implementation
* unit test both implementations
* switch to `DynamicMmapFlags`
* tidy up
* recalculate `indexed_count` on load
* grow bitslice aligned to the mmap page size
* ergonomic get_slice_for
* fix for growing mmap
* use more `get_slice_for()`
