Luis Cossío
3441e395e4
Chunked vectors with UniversalWrite storage ( #8233 )
...
* use CowMultiVector as return type from storages
* add advice to OpenOptions
* Implement ChunkedVectors with generic storage
* rename ChunkedVectors->VolatileChunkedVectors and ChunkedMmapVectors-> ChunkedVectors
* propagate everywhere
fix tests
* [auto] rename BytesRange -> ElementsRange
* [auto] rename BytesOffset -> ElementOffset
* coderabbit nits
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2026-03-26 17:25:24 +01:00
dependabot[bot]
a7739dbf2d
build(deps): bump rand_distr from 0.5.1 to 0.6.0 ( #8148 )
...
* build(deps): bump rand_distr from 0.5.1 to 0.6.0
Bumps [rand_distr](https://github.com/rust-random/rand_distr ) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/rust-random/rand_distr/releases )
- [Changelog](https://github.com/rust-random/rand_distr/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand_distr/compare/0.5.1...0.6.0 )
---
updated-dependencies:
- dependency-name: rand_distr
dependency-version: 0.6.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
* Migrate main code base to rand 0.10
* Migrate tests
* Migrate benches
---------
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: timvisee <tim@visee.me >
2026-03-26 17:09:49 +01:00
Ivan Pleshkov
75c6b0981d
use enum for SQ query and meta ( #7669 )
...
fix CI
review remarks
review remarks
review remarks
review remarks
simplify encoding of internal vector
fix ci
fix ci
review remarks
review remarks
2025-12-18 17:29:18 +01:00
Luis Cossío
9a1499c3e9
fix typo Uint8 -> Int8 ( #7666 )
2025-12-03 10:23:54 +01:00
Ivan Pleshkov
e1c6ec71a9
Scalar quantization encoding parameter ( #7602 )
...
* scalar quantization encoding parameter
fix ci
remove method
review remarks
fix ci
* uint8 - int8
2025-12-03 10:23:25 +01:00
Ivan Pleshkov
83bb3a3598
refactor sq score_bytes ( #7613 )
2025-12-03 10:19:10 +01:00
Ivan Pleshkov
f12b17545e
Remove quantization load from trait ( #7113 )
...
* remove quantization load
* refactor load
* fix fmt error
* fix clippy
2025-08-26 13:18:36 +02:00
Ivan Pleshkov
6bc3cc813c
EncodedStorage upsert vector ( #7048 )
...
* EncodedStorage upsert vector
2025-08-26 13:16:40 +02:00
Ivan Pleshkov
18ba202a76
Remove quantization save functions ( #7043 )
...
* remove quantization save
* revert convert_binary_encoding fn
* review remarks
* review remark
* review remarks
2025-08-26 13:16:39 +02:00
Ivan Pleshkov
cdf05b30c3
Wrap quantization Vec<u8> in a separate structure ( #7013 )
...
* separate struct for vec u8 quantization storage
* cgf testing for test storage
2025-08-14 14:29:25 +02:00
Ivan Pleshkov
f4ebdd9c9d
Appendable quantization storage ( #6935 )
...
* appendable qunatization storage
fmt
deprecate count in quantization config
fix arm tests build
qunatized storage vectors count
fix ci
are you happy clippy
fix compat tests
undo rename to `deprecated_count`
remove flusher
remove obsolete functions, don't use hardware counter in ram storage
are you happy clippy
fix gpu build
check ci when revert option
revert last commit
* debug ci
* log offsets
* log offsets
* fix compatibility tests
* deprecate count
* fix arm tests
* fix benches
* Update lib/quantization/src/encoded_vectors_binary.rs
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com >
---------
Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com >
2025-08-11 13:17:02 +02:00
Ivan Pleshkov
918a8e456b
use vector statistics for scalar bq query ( #6835 )
...
* use vector statistics for scalar bq query
* fix minor error
* remove test_binary_scalar_internal test
* do NOT use special file for storing vector stats
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2025-07-17 13:56:04 +02:00
Ivan Pleshkov
bd273ab68f
Merge pull request #6728
...
* # This is a combination of 7 commits.
* SameAsStorage default value
* fix coderabbit warnings
* neon for u8 bq
* sse for u8 bq
* fix windows build
* rename function
* add comments
* fmt
* fix arm build
* review remarks
2025-07-17 13:45:51 +02:00
Ivan Pleshkov
e53ff4bb33
Fix sq internal vector encoding ( #6763 )
...
* Fix SQ internal vector encoding
* add test
* clippy warning
2025-07-17 13:34:26 +02:00
Ivan Pleshkov
3a916c5d69
Merge pull request #6663
...
* bq encodings
* are you happy clippy
* are you happy clippy
* are you happy clippy
* are you happy clippy
* gpu tests
* update models
* are you happy fmt
* move additional bits to the end
* fix tests
* Welford's Algorithm
* review remarks
* are you happy clippy
* remove debug println in test
* coderabit nitpicks
* remove unnecessary clone and partialeq
* Use f64 for Welford's Algorithm
* try fix ci
* revert cargo-nextest
* add debug assertions
2025-07-17 13:16:17 +02:00
Arnaud Gourlay
89667860dd
Minor cleanups ( #6291 )
2025-04-20 23:42:17 +02:00
Tim Visée
8ad2b34265
Bump Rust edition to 2024 ( #6042 )
...
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
2025-03-21 11:38:56 +01:00
Kumar Shivendu
f18994c1f0
Quantization test improvements ( #5976 )
...
* minor change in test
* apply to other tests as well
* imporve var name
* add test for loading empty EncodedVectorsBin from disk
* fmt
2025-02-17 10:20:44 +01:00
Luis Cossío
f110328296
bump and migrate to rand 0.9.0 ( #5892 )
...
* bump and migrate to rand 0.9.0
also bump rand_distr to 0.5.0 to match it
* Migrate AVX2 and SSE implementations
* Remove unused thread_rng placeholders
* More random migrations
* Migrate GPU tests
* bump seed
---------
Co-authored-by: timvisee <tim@visee.me >
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com >
2025-02-11 14:36:20 +01:00
Jojii
5aee24cc08
Timeout aware hardware counter ( #5555 )
...
* Make hardware counting timeout aware
* improve test
* rebuild everything
* fmt
* post-rebase fixes
* upd tests
* fix tests
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2025-01-08 14:03:52 +01:00
xzfc
8bbe66c0fb
Reduce debug size ( #5556 )
...
* debug size: RawScorerImpl::peek_top_iter
* debug size: TypedMultiDenseVector::multi_vectors() uses
* debug size: stop_condition -> stopped
2024-12-09 11:16:38 +01:00
Arnaud Gourlay
61f38ec072
Merge quantization integration tests ( #5473 )
2024-12-09 10:47:02 +01:00
Jojii
faf645ec2e
Hardware counting for quantization ( #5369 )
...
* add cpu measurement for quantization
* clippy
* fix tests
* discard hardware counters in benchmarks
* Forwarding hardware counter in distributed setup (#5371 )
* make hardware counter available in distributed setup
* clippy
---------
Co-authored-by: generall <andrey@vasnetsov.com >
2024-11-08 11:26:39 +01:00
Ivan Pleshkov
21a3c108bb
Move quantization repo ( #5096 )
...
* move quantization repo
* are you happy fmt
* are you happy clippy
* remove dumping pq to image
* workspace deps
* are you happy clippy
2024-09-23 16:03:29 +02:00