Luis Cossío
5403ea68ab
Chunked vectors with UniversalWrite storage ( #8233 )
...
* use CowMultiVector as return type from storages
* add advice to OpenOptions
* Implement ChunkedVectors with generic storage
* rename ChunkedVectors->VolatileChunkedVectors and ChunkedMmapVectors-> ChunkedVectors
* propagate everywhere
fix tests
* [auto] rename BytesRange -> ElementsRange
* [auto] rename BytesOffset -> ElementOffset
* coderabbit nits
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2026-02-27 12:47:15 -03:00
dependabot[bot]
18a7587d4b
build(deps): bump rand_distr from 0.5.1 to 0.6.0 ( #8148 )
...
* build(deps): bump rand_distr from 0.5.1 to 0.6.0
Bumps [rand_distr](https://github.com/rust-random/rand_distr ) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/rust-random/rand_distr/releases )
- [Changelog](https://github.com/rust-random/rand_distr/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand_distr/compare/0.5.1...0.6.0 )
---
updated-dependencies:
- dependency-name: rand_distr
dependency-version: 0.6.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Migrate main code base to rand 0.10
* Migrate tests
* Migrate benches
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: timvisee <tim@visee.me >
2026-02-25 14:15:04 +01:00
Ivan Pleshkov
c14f83eff9
use enum for SQ query and meta ( #7669 )
...
fix CI
review remarks
review remarks
review remarks
review remarks
simplify encoding of internal vector
fix ci
fix ci
review remarks
review remarks
2025-12-17 22:35:14 +01:00
Luis Cossío
366ab07ba2
fix typo Uint8 -> Int8 ( #7666 )
2025-12-02 09:54:34 +03:00
Ivan Pleshkov
8df3dda006
Scalar quantization encoding parameter ( #7602 )
...
* scalar quantization encoding parameter
fix ci
remove method
review remarks
fix ci
* uint8 - int8
2025-12-01 12:03:36 +01:00
Ivan Pleshkov
a2be9e7fff
refactor sq score_bytes ( #7613 )
2025-11-27 10:49:33 +01:00
Ivan Pleshkov
879da486af
Remove quantization load from trait ( #7113 )
...
* remove quantization load
* refactor load
* fix fmt error
* fix clippy
2025-08-22 14:37:32 +02:00
Ivan Pleshkov
3f75fae516
EncodedStorage upsert vector ( #7048 )
...
* EncodedStorage upsert vector
2025-08-18 17:39:23 +02:00
Ivan Pleshkov
8961eeaf0a
Remove quantization save functions ( #7043 )
...
* remove quantization save
* revert convert_binary_encoding fn
* review remarks
* review remark
* review remarks
2025-08-18 13:55:07 +02:00
Ivan Pleshkov
a3e457d387
Wrap quantization Vec<u8> in a separate structure ( #7013 )
...
* separate struct for vec u8 quantization storage
* cgf testing for test storage
2025-08-12 13:20:48 +02:00
Ivan Pleshkov
766b527ac8
Appendable quantization storage ( #6935 )
...
* appendable qunatization storage
fmt
deprecate count in quantization config
fix arm tests build
qunatized storage vectors count
fix ci
are you happy clippy
fix compat tests
undo rename to `deprecated_count`
remove flusher
remove obsolete functions, don't use hardware counter in ram storage
are you happy clippy
fix gpu build
check ci when revert option
revert last commit
* debug ci
* log offsets
* log offsets
* fix compatibility tests
* deprecate count
* fix arm tests
* fix benches
* Update lib/quantization/src/encoded_vectors_binary.rs
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com >
---------
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com >
2025-08-06 13:51:01 +02:00
Ivan Pleshkov
bac28164ed
use vector statistics for scalar bq query ( #6835 )
...
* use vector statistics for scalar bq query
* fix minor error
* remove test_binary_scalar_internal test
* do NOT use special file for storing vector stats
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2025-07-11 10:15:28 +02:00
Ivan Pleshkov
b414a402bb
Merge pull request #6728
...
* # This is a combination of 7 commits.
* SameAsStorage default value
* fix coderabbit warnings
* neon for u8 bq
* sse for u8 bq
* fix windows build
* rename function
* add comments
* fmt
* fix arm build
* review remarks
2025-07-03 14:09:12 +02:00
Ivan Pleshkov
5895877577
Fix sq internal vector encoding ( #6763 )
...
* Fix SQ internal vector encoding
* add test
* clippy warning
2025-06-26 15:08:50 +02:00
Ivan Pleshkov
fe5becdadd
Merge pull request #6663
...
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
2025-06-19 18:21:00 +02:00
Arnaud Gourlay
33d062a3a7
Minor cleanups ( #6291 )
2025-04-01 13:01:10 +02:00
Tim Visée
3e536347e1
Bump Rust edition to 2024 ( #6042 )
...
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
2025-02-25 11:21:25 +01:00
Kumar Shivendu
38e1f4720b
Quantization test improvements ( #5976 )
...
* minor change in test
* apply to other tests as well
* imporve var name
* add test for loading empty EncodedVectorsBin from disk
* fmt
2025-02-12 09:45:18 +01:00
Luis Cossío
af74d1b96a
bump and migrate to rand 0.9.0 ( #5892 )
...
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2025-01-28 16:19:11 +01:00
Jojii
e0d0e233d8
Timeout aware hardware counter ( #5555 )
...
* Make hardware counting timeout aware
* improve test
* rebuild everything
* fmt
* post-rebase fixes
* upd tests
* fix tests
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2024-12-10 12:12:36 +01:00
xzfc
4548ebe4bf
Reduce debug size ( #5556 )
...
* debug size: RawScorerImpl::peek_top_iter
* debug size: TypedMultiDenseVector::multi_vectors() uses
* debug size: stop_condition -> stopped
2024-12-02 14:33:35 +00:00
Arnaud Gourlay
5bce09a1ed
Merge quantization integration tests ( #5473 )
2024-11-19 13:10:01 +01:00
Jojii
7b677fb7fa
Hardware counting for quantization ( #5369 )
...
* add cpu measurement for quantization
* clippy
* fix tests
* discard hardware counters in benchmarks
* Forwarding hardware counter in distributed setup (#5371 )
* make hardware counter available in distributed setup
* clippy
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2024-11-06 16:23:02 +01:00
Ivan Pleshkov
f27b6909bd
Move quantization repo ( #5096 )
...
* move quantization repo
* are you happy fmt
* are you happy clippy
* remove dumping pq to image
* workspace deps
* are you happy clippy
2024-09-17 16:30:26 +02:00