* On Windows ARM64 builds, disable usage of neon
* Also disable optimized popcount on Windows ARM64
* fix quantization build
* revert changes in BQ
---------
Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
* Create and load for an appendable quantization
remove feature flag
more todo
review remarks
review remarks
* fix after rebase
* Rename is_appendable to supports_appendable
---------
Co-authored-by: timvisee <tim@visee.me>
* Quantization storage builder for chunked mmap
* fix after rebase
* Update lib/segment/src/vector_storage/quantized/quantized_chunked_mmap_storage.rs
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Add common::typelevel module
* Use common::typelevel for score_bytes
* Revert "Split FilteredScorer into FilteredScorer and FilteredQuantizedScorer"
This reverts commit ff3a93ee95.
* FilteredQuantizedScorer
* Extract `struct Filters` from `FilteredScorer` and `FilteredQuantizedScorer`
* Revert "QuantizedVectors::query_scorer_bytes"
This reverts commit 9dee824afd.
* Partial revert "Add QueryScorerBytes::score_bytes"
This reverts commit d80af4f698, but keeps
the QueryScorerBytes trait while removing implementations.
* Partial revert "Add EncodedVectorsBytes::score_point_vs_bytes"
This partially reverts commit c741e42f9a.
- trait `EncodedVectorsBytes` and its implementations: reverted/removed
- calls to `EncodedVectorsBytes::score_point_vs_bytes`: replaced with
calls to `EncodedVectors::score_bytes`
* Remove unused method FilteredScorer::new_from_raw
* use vector statistics for scalar bq query
* fix minor error
* remove test_binary_scalar_internal test
* do NOT use special file for storing vector stats
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* # This is a combination of 7 commits.
* SameAsStorage default value
* fix coderabbit warnings
* neon for u8 bq
* sse for u8 bq
* fix windows build
* rename function
* add comments
* fmt
* fix arm build
* review remarks
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
* Make EncodedVectors::EncodedQuery associated type
* Make EncodedStorageBuilder::Storage associated type
* Make QueryScorer::TVector associated type
* Remove TVector from RawScorerImpl generic parameters
* Fix vector_io_read for multivector recommendation
* fix
* cleanup
* single disposable
* do not use disposable hardware counter for quantized multivectors
* wip: implement explicit populate and clear_cache functions for all components
* fmt
* implement clear and populate for vector storages
* fmt
* implement clear and populate for payload storage
* wip: implement explicit populate and clear_cache functions payload indexes
* implement explicit populate and clear_cache functions payload indexes
* fix clippy on CI
* only compile posix_fadvise on linux
* only compile posix_fadvise on linux
* implement explicit populate and clear_cache functions for quantized vectors
* fmt
* remove post-load prefault
* fix typo
* implement is-on-disk for payload indexes, implement clear on drop for segment, implement clear after segment build
* fmt
* also evict quantized vectors after optimization
* re-use and replace advise_dontneed
* Cardinality estimation measurements
* Apply hw measurements to latest changes from dev
* Clippy
* Also measure cardinality estimation for geo index
* Make measured units 'bytes'
* Use PointOffsetType instead of u32 for size calculation
* fix memory cost for check_values_any in mmap index
* fix double counting for value reading in mmap, remove hw_counter from mmap hashmap
* fmt
* fix hw measurement for text index
* Remove non necessary lifetime annotations
---------
Co-authored-by: generall <andrey@vasnetsov.com>
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
