Commit Graph

11 Commits

Author SHA1 Message Date
Jojii
e957d6f44a Cardinality estimation IO measurements (#6117)
* Cardinality estimation measurements

* Apply hw measurements to latest changes from dev

* Clippy

* Also measure cardinality estimation for geo index

* Make measured units 'bytes'

* Use PointOffsetType instead of u32 for size calculation

* fix memory cost for check_values_any in mmap index

* fix double counting for value reading in mmap, remove hw_counter from mmap hashmap

* fmt

* fix hw measurement for text index

* Remove non necessary lifetime annotations

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-03-14 11:05:38 +01:00
Arnaud Gourlay
423a39f301 Remove duplicated mmap utils (#6129) 2025-03-10 12:20:22 +01:00
Tim Visée
3e536347e1 Bump Rust edition to 2024 (#6042)
* Bump Rust edition to 2024

* gen is a reserved keyword now

* Remove ref mut on references

* Mark extern C as unsafe

* Wrap unsafe function bodies in unsafe block

* Geo hash implements Copy, don't reference but pass by value instead

* Replace secluded self import with parent

* Update execute_cluster_read_operation with new match semantics

* Fix lifetime issue

* Replace map_or with is_none_or

* set_var is unsafe now

* Reformat
2025-02-25 11:21:25 +01:00
Luis Cossío
af74d1b96a bump and migrate to rand 0.9.0 (#5892)
* bump and migrate to rand 0.9.0

also bump rand_distr to 0.5.0 to match it

* Migrate AVX2 and SSE implementations

* Remove unused thread_rng placeholders

* More random migrations

* Migrate GPU tests

* bump seed

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2025-01-28 16:19:11 +01:00
Andrey Vasnetsov
2a5b46e780 use fsync instead of flush (#5629) 2024-12-10 00:13:03 +01:00
Ivan Pleshkov
1447d1efa8 atomic save quantization metadata (#5627) 2024-12-09 21:24:24 +01:00
xzfc
4548ebe4bf Reduce debug size (#5556)
* debug size: RawScorerImpl::peek_top_iter

* debug size: TypedMultiDenseVector::multi_vectors() uses

* debug size: stop_condition -> stopped
2024-12-02 14:33:35 +00:00
Ivan Pleshkov
a770a7f3a0 Gpu vectors (#5339)
* parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012991 +0100

parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012977 +0100

parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012901 +0100

parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012766 +0100

parent 33b2c176cb
author Ivan Pleshkov <pleshkov.ivan@gmail.com> 1729676438 +0200
committer Ivan Pleshkov <pleshkov.ivan@gmail.com> 1732012730 +0100

vulkan api wrapper crate

fmt

gpu is not a workspace member

exclude gpu from workspace

disable tests for gpu crate

ash version up

change collection for pipeline builder

gpu test

gpu is not optional

check device features

remove feature

make all fields private

more comments

are you happy fmt

qdrant team as authors

gpu vectors

move changes from magic branch

move storage updates

are you happy codespell

are you happy clippy

don't make storage_impl public

review remarks

are you happy clippy

are you happy fmt

* Update lib/segment/src/index/hnsw_index/gpu/gpu_vector_storage/tests.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* review remarks

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2024-11-27 09:50:10 +01:00
Arnaud Gourlay
2a0f9e9212 Use references for less cloning when possible (#5409) 2024-11-11 10:26:17 +01:00
Jojii
7b677fb7fa Hardware counting for quantization (#5369)
* add cpu measurement for quantization

* clippy

* fix tests

* discard hardware counters in benchmarks

* Forwarding hardware counter in distributed setup (#5371)

* make hardware counter available in distributed setup

* clippy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2024-11-06 16:23:02 +01:00
Ivan Pleshkov
f27b6909bd Move quantization repo (#5096)
* move quantization repo

* are you happy fmt

* are you happy clippy

* remove dumping pq to image

* workspace deps

* are you happy clippy
2024-09-17 16:30:26 +02:00