19 Commits

Author SHA1 Message Date
Arnaud Gourlay
c196d2eb1a Benches: use SmallRng instead of ChaCha12-based generators (#9887)
* Benches: use SmallRng instead of ChaCha12-based generators

All benchmarks used StdRng or rand::rng() (ThreadRng), both backed by the
ChaCha12 block cipher in rand 0.10. Benchmarks do not need crypto-strength
randomness, and several draw random values inside the timed closure, so
cipher work was included in the measurement itself.

Switch every bench target to SmallRng (Xoshiro256++), and key the HNSW
graph cache and sparse index cache by RNG algorithm so stale caches built
from the old generator are not reused against newly generated vectors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Benches: replace free-function rand::random with local SmallRng

Addresses review: rand::random draws from the thread RNG (ChaCha12),
including inside the timed loop of the pq score benchmark.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 17:22:27 +02:00
Arnaud Gourlay
cad112bb1c Fix Clippy 1.97 (#9716)
* Remove from_iter_instead_of_collect from workspace lints

The lint was removed from clippy (beta) and now triggers
renamed_and_removed_lints warnings in every crate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix clippy::chunks_exact_to_as_chunks

Replace chunks_exact with a constant chunk size by as_chunks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix clippy::needless_late_init

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix clippy::useless_borrows_in_formatting

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix clippy::uninlined_format_args

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix clippy::for_kv_map

Iterate map values directly instead of discarding keys.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Allow clippy::result_large_err on QueueProxyShard::new_from_version

The Err variant intentionally hands the LocalShard back to the caller.
Same pattern as the existing allow on ForwardProxyShard::new.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Allow clippy::result_unit_err on wait_for_consensus_commit

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 14:53:15 +02:00
xzfc
faf2714f4b Warn on clippy::wildcard_enum_match_arm (#9096) 2026-05-19 18:14:15 +00:00
Luis Cossío
757a168167 make quantized_vector_size method static (#9060) 2026-05-15 18:31:23 -04:00
Luis Cossío
5403ea68ab Chunked vectors with UniversalWrite storage (#8233)
* use CowMultiVector as return type from storages

* add advice to OpenOptions

* Implement ChunkedVectors with generic storage

* rename ChunkedVectors->VolatileChunkedVectors and ChunkedMmapVectors-> ChunkedVectors

* propagate everywhere

fix tests

* [auto] rename BytesRange -> ElementsRange

* [auto] rename BytesOffset -> ElementOffset

* coderabbit nits

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2026-02-27 12:47:15 -03:00
dependabot[bot]
18a7587d4b build(deps): bump rand_distr from 0.5.1 to 0.6.0 (#8148)
* build(deps): bump rand_distr from 0.5.1 to 0.6.0

Bumps [rand_distr](https://github.com/rust-random/rand_distr) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/rust-random/rand_distr/releases)
- [Changelog](https://github.com/rust-random/rand_distr/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand_distr/compare/0.5.1...0.6.0)

---
updated-dependencies:
- dependency-name: rand_distr
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Migrate main code base to rand 0.10

* Migrate tests

* Migrate benches

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: timvisee <tim@visee.me>
2026-02-25 14:15:04 +01:00
Ivan Pleshkov
3f75fae516 EncodedStorage upsert vector (#7048)
* EncodedStorage upsert vector
2025-08-18 17:39:23 +02:00
Ivan Pleshkov
8961eeaf0a Remove quantization save functions (#7043)
* remove quantization save

* revert convert_binary_encoding fn

* review remarks

* review remark

* review remarks
2025-08-18 13:55:07 +02:00
Ivan Pleshkov
a3e457d387 Wrap quantization Vec<u8> in a separate structure (#7013)
* separate struct for vec u8 quantization storage

* cgf testing for test storage
2025-08-12 13:20:48 +02:00
Ivan Pleshkov
766b527ac8 Appendable quantization storage (#6935)
* appendable qunatization storage

fmt

deprecate count in quantization config

fix arm tests build

qunatized storage vectors count

fix ci

are you happy clippy

fix compat tests

undo rename to `deprecated_count`

remove flusher

remove obsolete functions, don't use hardware counter in ram storage

are you happy clippy

fix gpu build

check ci when revert option

revert last commit

* debug ci

* log offsets

* log offsets

* fix compatibility tests

* deprecate count

* fix arm tests

* fix benches

* Update lib/quantization/src/encoded_vectors_binary.rs

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>

---------

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
2025-08-06 13:51:01 +02:00
Ivan Pleshkov
6521b0df7d bq scalar query benches (#6831)
* bq scalar query benches

* review remarks
2025-07-14 19:28:07 +02:00
Ivan Pleshkov
b414a402bb Merge pull request #6728
* # This is a combination of 7 commits.

* SameAsStorage default value

* fix coderabbit warnings

* neon for u8 bq

* sse for u8 bq

* fix windows build

* rename function

* add comments

* fmt

* fix arm build

* review remarks
2025-07-03 14:09:12 +02:00
Ivan Pleshkov
fe5becdadd Merge pull request #6663
* bq encodings

* are you happy clippy

* are you happy clippy

* are you happy clippy

* are you happy clippy

* gpu tests

* update models

* are you happy fmt

* move additional bits to the end

* fix tests

* Welford's Algorithm

* review remarks

* are you happy clippy

* remove debug println in test

* coderabit nitpicks

* remove unnecessary clone and partialeq

* Use f64 for Welford's Algorithm

* try fix ci

* revert cargo-nextest

* add debug assertions
2025-06-19 18:21:00 +02:00
Tim Visée
3e536347e1 Bump Rust edition to 2024 (#6042)
* Bump Rust edition to 2024

* gen is a reserved keyword now

* Remove ref mut on references

* Mark extern C as unsafe

* Wrap unsafe function bodies in unsafe block

* Geo hash implements Copy, don't reference but pass by value instead

* Replace secluded self import with parent

* Update execute_cluster_read_operation with new match semantics

* Fix lifetime issue

* Replace map_or with is_none_or

* set_var is unsafe now

* Reformat
2025-02-25 11:21:25 +01:00
Luis Cossío
af74d1b96a bump and migrate to rand 0.9.0 (#5892)
* bump and migrate to rand 0.9.0

also bump rand_distr to 0.5.0 to match it

* Migrate AVX2 and SSE implementations

* Remove unused thread_rng placeholders

* More random migrations

* Migrate GPU tests

* bump seed

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2025-01-28 16:19:11 +01:00
Jojii
e0d0e233d8 Timeout aware hardware counter (#5555)
* Make hardware counting timeout aware

* improve test

* rebuild everything

* fmt

* post-rebase fixes

* upd tests

* fix tests

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-12-10 12:12:36 +01:00
xzfc
4548ebe4bf Reduce debug size (#5556)
* debug size: RawScorerImpl::peek_top_iter

* debug size: TypedMultiDenseVector::multi_vectors() uses

* debug size: stop_condition -> stopped
2024-12-02 14:33:35 +00:00
Jojii
7b677fb7fa Hardware counting for quantization (#5369)
* add cpu measurement for quantization

* clippy

* fix tests

* discard hardware counters in benchmarks

* Forwarding hardware counter in distributed setup (#5371)

* make hardware counter available in distributed setup

* clippy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-11-06 16:23:02 +01:00
Ivan Pleshkov
f27b6909bd Move quantization repo (#5096)
* move quantization repo

* are you happy fmt

* are you happy clippy

* remove dumping pq to image

* workspace deps

* are you happy clippy
2024-09-17 16:30:26 +02:00