* integrate quantized data to storages
* revert gitignore
* are you happy clippy
* quantize in optimizer
* provide flag
* fix segfault
* skip quantization flag, update scores
* use quantization flag
* are you happy fmt
* use quantization flag
* quantized search test
* are you happy fmt
* refactor test, refactor scorer choosing
* are you happy fmt
* run quantization on segment builder
* decrease testing parameters
* simplify segment
* update version
* remove use_quantization flag
* provide quantization config
* quantization version up
* euclid dist
* add euclid test
* saveload
* fix initialization bugs
* quantization lib version up
* fix arm build
* refactor scorer selecting
* quant lib version up
* are you happy fmt
* are you happy fmt
* are you happy clippy
* add save/load test for simple storage
* add comments
* quantiles
* quantization mmap
* remove f32
* mmap test
* fix mmap slice
* fix mmap test
* use chunks for quantization storage
* fix build
* are you happy fmt
* update quantization library
* update quantization lib
* update quantization lib
* integrate api changes
* are you happy fmt
* change quantization api
* additional checks in tests
* update quantization version
* fix unit tests
* add quantization to storage config
* use quantization for all cardinality search cases
* Integrate quantization suggestions 2 (#1520)
* review api
* wip: refactor quantization integrations
* wip: refactor quantization integrations
* wip: fmt
* include quantization into snapshot
* fmt
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* WIP: Fix `Segment::take_snapshot`
TODO:
- This commit, probably, breaks snapshotting of segments with memmapped vector storage
- `ProxySegment::take_snapshot` seems to potentially similar bug
* WIP: Fix `Segment::take_snapshot`
- Fix snapshotting of `StructPayloadIndex`
- Fix snapshotting of segments with memmapped vector storage
- Temporarily break `ProxySegment::take_snapshot`
* Fix `ProxySegment::take_snapshot`
* Remove `copy_segment_directory` test
* nitpicking
* clippy fixes
* use OperationError::service_error
* Cleanup `TinyMap` trait bounds and derive `Debug`
* Fix `test_snapshot` test
- Derive `Debug` for `NamedVectors`
* Move utility functions from `segment.rs` to `utils` module
* Contextualize `segment::utils::fs::move_all` a bit more carefully
* Fix a typo
* add backward compatibility with old snapshot formats
* fmt
* add snapshot for compatibility test
* git lfs is a piece of shit
* Nitpicking
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
- Add a simple abstraction/wrapper over `memmap2::Advice` and a global switch
to select "memory advice" used for memmapped storage to the `segment` crate
- Add `mmap_advice` configuration option to the `storage::types::StorageConfig`
- Implement `search-points` benchmark
* many named vectors per point (segment-level)
* operation result for dim function
* beautifulized vector name
* fix naming bug
* segment version migration
* fmt
* add segment tests
* are you happy clippy
* fix build
* [WIP] many named vectors per point (collection-level) (#975)
* config and search
* fix placeholders for proxy segment move
* remove VectorType from collection
* are you happy fmt
* vectors in grps messages
* create collections with vectors
* segment holder fixes
* are you happy fmt
* remove default vector name placeholders
* are you happy fmt
* are you happy clippy
* fix build
* fix web api
* are you happy clippy
* are you happy fmt
* record vector&vectors
* openapi update
* fix openapi integration tests
* segment builder fix todo
* vector names for update from segment
* remove unwrap
* backward compatibility
* upd openapi
* backward compatible PointStruct
* upd openapi
* fix record back-comp
* fmt
* vector configuration backward compatibility
* fix vetor storage size estimation
* fmt
* multi-vec segment test + index test
* fmt
* api integration tests
* [WIP] Named vectors struct (#1002)
* move to separate file
* named vectors as struct
* use cow
* fix build
* keys iterator
* avoid copy in PointStruct -> get_vectors
* avoid another copy
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* improve alias tests
* introduce new alias persistence and remove Sled
* introduce FileStorageError
* use new error in AliasMapping
* aliases are kept in memory and saved to disk when modified
* better naming
* make alias renaming atomic
* fmt
* use correct alias and docs
* code review: cleaner error conversion
* fix clippy warnings
- doc links
- explicit deref
- if instead of match for single bool condition
- combine similar match branches
* revert removal of transmute
* return Some when vec is not empty
* [CLIPPY] Fixed the warning for references of the user defined types
* [CLIPPY] Fix module naming issue
* [CLIPPY] Fix the last set of warnings and enable clippy check during CI
* Moved cargo fmt and cargo clippy into it's own action
* raw points scorer
* raw point scorer for memmap storage
* search interface prepare
* graph binary saving + store PointOffsetId as u32
* WIP: entry points
* connect new link method
* update libs + search layer method + visited list + search context + update rust
* implement Euclid metric + always use MinHeap for priority queue
* small refactor
* search for 0 level entry
* update visited pool to be lock free and thread safe
* use ef_construct from graph layer struct + limit visited links to M
* add metric pre-processing before on vector upsert
* old hnsw heuristic
* save hnsw graph for export
* search method + tests
* small fixes
* add benchmark and profiler
* build time optimizations
* use SeaHash
* remove unsed benchmark
* merge hnsw graph function
* WIP:HNSW index build function
* HNSW build_index with additional indexing
* refactor fixtures
* graph save and load test
* test and fixes for filterable HNSW
* enable hnsw index for query planning
* fix cardinality estimation tests + remove query planner as class
* small refactor
* store full copy of collection settings with collection + allow partial override on creation #16
* API for updating collection parameters #16
* refactor: move collection error -> types
* report collection status in info API #17
* update OpenAPI Schema
