Commit Graph

5465 Commits

Author SHA1 Message Date
dependabot[bot]
8ec5d041f1 build(deps): bump uuid from 1.17.0 to 1.18.0 (#7021)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 1.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 11:25:36 +02:00
dependabot[bot]
593e107ef0 build(deps): bump tokio-util from 0.7.15 to 0.7.16 (#7023)
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.15 to 0.7.16.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.15...tokio-util-0.7.16)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-version: 0.7.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 11:08:18 +02:00
dependabot[bot]
90f39208c9 build(deps): bump actions/download-artifact from 4 to 5 (#7018)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 11:02:15 +02:00
dependabot[bot]
8ffae5b2cb build(deps): bump actions/checkout from 4 to 5 (#7026)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 11:00:41 +02:00
dependabot[bot]
580e0eadd6 build(deps): bump bytemuck from 1.23.1 to 1.23.2 (#7020) 2025-08-12 06:46:50 +00:00
dependabot[bot]
f2f42b8593 build(deps): bump clap from 4.5.41 to 4.5.43 (#7028)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.41 to 4.5.43.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.41...clap_complete-v4.5.43)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.43
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 08:17:01 +02:00
dependabot[bot]
39849018b7 build(deps): bump io-uring from 0.7.8 to 0.7.9 (#7027)
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.7.8 to 0.7.9.
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-version: 0.7.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 08:07:07 +02:00
dependabot[bot]
b264f62961 build(deps): bump sysinfo from 0.36.1 to 0.37.0 (#7029)
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.36.1 to 0.37.0.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.36.1...v0.37.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 08:06:08 +02:00
dependabot[bot]
de64a6beff build(deps): bump roaring from 0.11.1 to 0.11.2 (#7030)
Bumps [roaring](https://github.com/RoaringBitmap/roaring-rs) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/RoaringBitmap/roaring-rs/releases)
- [Commits](https://github.com/RoaringBitmap/roaring-rs/compare/v0.11.1...v0.11.2)

---
updated-dependencies:
- dependency-name: roaring
  dependency-version: 0.11.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 08:02:43 +02:00
dependabot[bot]
2f76d15b34 build(deps): bump cc from 1.2.30 to 1.2.32 (#7024)
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.2.30 to 1.2.32.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.2.30...cc-v1.2.32)

---
updated-dependencies:
- dependency-name: cc
  dependency-version: 1.2.32
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 07:30:53 +02:00
dependabot[bot]
be7acc6d68 build(deps): bump serde_json from 1.0.141 to 1.0.142 (#7025)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.141 to 1.0.142.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.141...v1.0.142)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-version: 1.0.142
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 07:30:05 +02:00
dependabot[bot]
da4d52e409 build(deps): bump lukemathwalker/cargo-chef (#7019)
Bumps lukemathwalker/cargo-chef from latest-rust-1.88.0 to latest-rust-1.89.0.

---
updated-dependencies:
- dependency-name: lukemathwalker/cargo-chef
  dependency-version: latest-rust-1.89.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-12 07:29:35 +02:00
xzfc
079ef6fb13 Implement GraphLinksFormat::CompressedWithVectors (#6982)
* Simplify graph_links tests

* Rstest-ify hnsw_index::tests::test_save_load

* Rename EncodedVectorsU8::{get_quantized_vector => get_quantized_vector_offset_and_code}

Reason: it was inconsistent with other implementations of
get_quantized_vector().

* Add QuantizedVectors::{get_quantized_vectors, get_quantized_vector_layout}

* Rename `links_map` to `for_each_link`

* pack_links: expose the updated `raw_links` order

* TestRawScorerProducer: support quantization

* Make GraphLinksSerializer::new return Result

* Rename links/compressed_links into `neighbors`

* Implement GraphLinksFormat::CompressedWithVectors

* GraphLayersWithVectors

* bitpacking: add packed_links_size

* Reorder neighbors: N__VVVVVVVcccccccccc -> Ncccccccccc__VVVVVVV
2025-08-11 17:28:43 +00:00
Tim Visée
6c592c7b6d Fix GPU build in CI due to outdated package index (#7010)
* Update APT package index before installing

* Install clang and other packages with single command

* Don't quiet package index update
2025-08-11 14:16:40 +02:00
tellet-q
f0612d6680 Refactor bash tests: tls (#6995)
* Refactor tls bash test

* Update codespell

* Address review
2025-08-11 10:05:47 +02:00
Tim Visée
d748f4cf57 Show unique index types in error message (#6998) 2025-08-07 16:48:39 +02:00
Tim Visée
1bba3891ec Fix clippy warnings for Rust 1.89 (#6997) 2025-08-07 14:34:52 +02:00
Kumar Shivendu
34e18eb22c Allow retaining more closed segments (#6976)
* Retain more closed segments

* Add back rocksdb

* Update WalOptions/WalConfig across the code

* Use NonZeroUsize

* Expose via APIs

* Update gRPC docs

* Fix stoarge compat test

* recompile openapi.json with rocksdb

* default wal retain closed fn

* update openapi.json

* Use qdrant/wal latest commit and remove from config.yaml
2025-08-07 13:44:26 +02:00
Kyamran
2cd230dda5 Add buffer_size_b to on_disk (#6993)
* Add buffer_size_b to on_disk

* Rename buffer_size_b to buffer_size_bytes
2025-08-07 13:42:59 +02:00
Luis Cossío
0a1a3d620b Use BitvecFlags in vector storages (#6994)
* put all flags impls in a single module

* use in appendable_dense_vector_storage

* use in appendable_mmap_multi_dense_storage

* use in mmap_sparse_vector_storage

* clippy

* Add comment we cannot rely on length of deleted vectors bitvec

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-07 11:54:14 +02:00
Luis Cossío
6344ab82ac BitvecFlags: bitvec container with dynamic mmap flags storage (#6990)
* separate flags buffering and in-memory container

* new bitvec flags

* fmt

* clippy

* make BufferedDynamicFlags public to crate only

* clear cache after loading

* clippyyy

* improve comment

* Use resize with a constant to eliminate a hot closure

* Add comment on why we take subslice of bitslice

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-07 11:30:42 +02:00
Andrey Vasnetsov
b58fd263d2 add example of multi-node docker-compose file for easier experiemnting (#6953)
* add example of multi-node docker-compose file for easier experiemnting

* rename peers and add healthcheck
2025-08-07 00:49:21 +02:00
Roman Titov
8be5e1b07f Refactor Collection::restore_shard_snapshot (#6988) 2025-08-06 17:48:04 +02:00
Roman Titov
a08c89082c Track payload index schema version (#6819) 2025-08-06 16:35:43 +02:00
Roman Titov
563194febf Fix test_create_shard_key_read_availability flakiness (#6902) 2025-08-06 16:14:13 +02:00
Luis Cossío
6ca50680f5 Buffered mutable bool index (#6977)
* use buffered dynamic flags in bool index

* put base_dir out of Storage

* Adjust populate comment

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-06 09:10:27 -05:00
Ivan Pleshkov
766b527ac8 Appendable quantization storage (#6935)
* appendable qunatization storage

fmt

deprecate count in quantization config

fix arm tests build

qunatized storage vectors count

fix ci

are you happy clippy

fix compat tests

undo rename to `deprecated_count`

remove flusher

remove obsolete functions, don't use hardware counter in ram storage

are you happy clippy

fix gpu build

check ci when revert option

revert last commit

* debug ci

* log offsets

* log offsets

* fix compatibility tests

* deprecate count

* fix arm tests

* fix benches

* Update lib/quantization/src/encoded_vectors_binary.rs

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>

---------

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
2025-08-06 13:51:01 +02:00
Arnaud Gourlay
1c74c9f162 Fix Clippy 1.89 again (#6987) 2025-08-06 12:25:40 +02:00
xzfc
542f03b033 TestRawScorerProducer to use VectorStorageEnum (#6965)
* TestRawScorerProducer to use VectorStorageEnum

* TestRawScorerProducer:🆕 accept distance as an argument

* Drop TMetric from TestRawScorerProducer

* Remove `impl VectorStorage for TestRawScorerProducer`

* Add TestRawScorerProducer::internal_scorer

* Remove TestRawScorerProducer::get_vector

* Rename TestRawScorerProducer::{get_scorer => scorer}
2025-08-06 09:41:13 +00:00
tellet-q
9597a8bee1 Hardcode the link (#6985)
* Hardcode the link

* Fix image build
2025-08-06 11:24:16 +02:00
Kumar Shivendu
a4220fdd84 Partial snapshots update recovery ts (#6980)
* Update recovery ts despite empty partial snapshots

* fmt

* apply suggestions with some improvements
2025-08-06 10:48:03 +02:00
Voyager 3
e3f94a22cd bump rust 1.88 (#6984)
fix build error if use rust 1.87
2025-08-06 09:42:52 +02:00
Arnaud Gourlay
255dd51e5a Fix Clippy 1.89 (#6981) 2025-08-05 18:06:04 +02:00
Arnaud Gourlay
33d15fd452 Bump Python integration test deps for Dependabot (#6979) 2025-08-05 18:05:36 +02:00
Arnaud Gourlay
a2b9a50381 Remove hardware measurement from fulltext index (#6884) 2025-08-05 17:28:49 +02:00
Tim Visée
6a116a54a9 Fix flushing of dynamic mmap flags (#6966)
* Directly flush flags, don't use single-use flusher

* Explicitly flush before resizing flags file, prevent losing dirty pages

* Don't create now obsolete flusher when opening mmap

* clippy

* Move flush call

* flush before resizing only

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-08-05 17:05:29 +02:00
Luis Cossío
9fb92ca55d New mutable null index (#6954)
* Introduce BufferedDynamicFlags

* clippy and extra changes

* fix flushing

* add mutable index impl

* use MutableNullIndex instead of MmapNullIndex

* use BufferedDynamicFlags

* remove mmap_null_index

* take updates before returning the flusher

* bump total_point_count on add and remove point

* improve test case

* Only clean up mutable null index files if index exists

* Remove mention of cached in context of length

* refactor flags into a single `Option<Storage>`

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-05 17:04:40 +02:00
Luis Cossío
4d92a33f74 cleanup versions on segment repair (#6975) 2025-08-05 14:38:15 +02:00
tellet-q
f87e505439 Refactor bash tests: low-ram (#6911)
* Refactor low-ran bash test

* Address review
2025-08-05 13:05:39 +02:00
Jojii
c2a9183f5f BM25 in the Core (#6891)
* Uncouple tokenizer and TextIndexParams

* Refactor token processing in `TokenizerConfig` (#6912)

* Refactor token processing in TokenizerConfig

* fix max length checking

* [Bm25] Execution implementation (#6939)

* Add Bm25

* Execute BM25 if config available

* cargo format

* Add mocked tests for inference and bm25

* Properly apply InferenceType

* Fix tests

* Review remarks

* Review remarks

* Add overwritten model name fix again

* use enum instead of multiple constants

* ensure handling all fields of InferenceInput in infer_local

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* review fixes

* fmt

* spell-check

* deduplicate code

---------

Co-authored-by: Luis Cossío <luis.cossio@qdrant.com>
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2025-08-03 20:43:20 -06:00
Luis Cossío
a7701587f4 [decay formula] allow [0,1] range for midpoint in lin_decay (#6959)
* allow [0,1] range for midpoint in lindecay

* clippyyyyy

* Better error message
2025-08-01 13:49:54 -06:00
Roman Titov
bd34d52578 Cleanup partial snapshot stuff (#6925) 2025-08-01 18:33:32 +02:00
Andrey Vasnetsov
260ca844e0 HardwareUsage -> Usage (#6967) 2025-07-31 10:28:57 -06:00
xzfc
e7038130e1 Add Distance::preprocess_vector (#6964) 2025-07-31 13:11:12 +00:00
Jojii
4b6806bdc6 Fix config path & Webui Assets in Debian package (#6958)
* Fix debian config location & WebUI assets

* remove config overwrite env variable again
2025-07-31 10:02:17 +02:00
xzfc
83005cd2db QueryScorerBytes (#6945)
* Add EncodedVectorsBytes::score_point_vs_bytes

* Add QueryScorerBytes::score_bytes

* QuantizedVectors::query_scorer_bytes

* Split FilteredScorer into FilteredScorer and FilteredQuantizedScorer
2025-07-31 03:05:10 +00:00
Kumar Shivendu
76cc63359d Bump dev version to 1.15.2-dev (#6955)
* Bump dev version to 1.15.2-dev

* Updated cargo.lock

* Trigger CI
2025-07-29 22:26:24 +05:30
Kumar Shivendu
edd3f12899 Fix coverage failure after recent changes (#6948) 2025-07-29 21:28:58 +05:30
Roman Titov
e8625ba903 Avoid creating partial snapshot if snapshot manifest contains no changes (#6924) 2025-07-29 14:18:57 +02:00
Luis Cossío
ba059b3e71 use roaring bitmap in map index (#6926) 2025-07-29 12:51:26 +02:00