Commit Graph

5465 Commits

Author SHA1 Message Date
Arnaud Gourlay
f30e7f7273 Fix missing REST API validation for point batch update (#7219)
* Fix missing REST API validation for point batch update

* use a valid vector
2025-09-05 18:25:37 +02:00
Jojii
1e1d683391 Bump segment version on {delete,update,upsert}-by-filter requests (#7157)
* Bump segment versions on *-by-filter operations to acknowledge WAL

* Assert exactly one segment with the new operation version

* Review remarks

* Separate segment version number to not affect partial snapshots

* Adjust unit test

* Remove old bump_segment_version implementations

* Clarify max_persisted_segment_version_overwrite in flush_all

* Move trait implementations - Clippy

* Apply segment bumping to all *-by-filter functions

* Remove AtomicOptionU64

* Update function name and comments

* Remove max condition, start with overwrite value

* Add assertion, can never have zero segments

* Make max_persisted_segment_version_overwrite monotonic with fetch_max

---------

Co-authored-by: timvisee <tim@visee.me>
2025-09-05 14:29:14 +02:00
Arnaud Gourlay
dd30fce77f Fix missing REST API Filter validations (#7218) 2025-09-05 13:53:18 +02:00
Arnaud Gourlay
3420311fd7 Fix missing API validation for MinShould (#7217) 2025-09-05 13:16:09 +02:00
Andrey Vasnetsov
852c58e689 make sure that grpc::Vector can always use new format (#7185)
* make sure that grpc::Vector can always use new format

* specify target version for ToDo

* Update lib/api/src/conversions/vectors.rs

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-09-05 13:13:06 +02:00
tellet-q
23a20261cf Add e2e test for snapshots (#7066)
* Add e2e test that mimics 1 crasher's scenario
2025-09-05 09:14:59 +02:00
Andrey Vasnetsov
b39ff122fa refactor matrix api to use query instead of deprecated search (#7215) 2025-09-04 21:19:35 -04:00
Arnaud Gourlay
8235a76fde Bump scc 2.1.0 to avoid yanked version (#7214) 2025-09-04 16:25:48 +02:00
Arnaud Gourlay
83da3c4ee7 Bump http 0.2.12 (#7212)
* Bump http 0.2.12

* remove unused deps
2025-09-04 15:48:17 +02:00
Arnaud Gourlay
da44fc00af Remove assertions for deleted version when propagating deletes (#7162) 2025-09-04 15:34:40 +02:00
Tim Visée
e1a9cb57ca Add comment on why we do/don't share deleted points in proxies (#7209) 2025-09-04 14:27:28 +02:00
Tim Visée
ac1914227f Unit test with proxy that propagates older delete to wrapped segment (#7208)
* Add unit test with proxy that propagates older delete to wrapped segment

* Link to pull request

* Disable faulty debug assertion, replace with comment noting semantics

* Minor tweaks

* Point version check in optimizer makes sense, assert no double segment
2025-09-04 14:25:48 +02:00
xzfc
3baa7d9d7d HNSW: use new graph format during search (#7195)
* point_scorer: generic `Filters` -> non-generic `ScorerFilters`

point_scorer: rename FilteredQuantizedScorer -> FilteredBytesScorer

* Use ScorerFilters when appropriate

* {CustomQueryScorer,MetricQueryScorer}: SupportsBytes = True

* Add GraphLayersWithVectors::search_with_vectors

* Use GraphLayersWithVectors::search_with_vectors in HNSWIndex::search_with_graph

* Move `BoxCow` and `SimpleCow` to a common module

* Review remarks
2025-09-04 11:48:26 +00:00
xzfc
193306f2d7 Fixup remaining PeerId anonymization (#7207) 2025-09-04 09:54:39 +02:00
Ivan Pleshkov
33837423e9 Enable BQ in appendable segments (#7145)
* Create and load for an appendable quantization

remove feature flag

more todo

review remarks

review remarks

* fix after rebase

* Enable BQ in appendable segments

* remove const placeholder

* fix after rebase

* fix after rebase

* fix config mismatch optimizer

* fix sq infinity loop

* add comment
2025-09-03 13:56:32 +02:00
Roman Titov
6cb54ca380 Use Path instead of str in ConsensusWal::new (#7205) 2025-09-03 11:21:53 +02:00
xzfc
1e4a39a5a0 CompressedWithVectors: include base/full vectors (#7171)
* Add VectorStorageEnum::{get_vector_bytes_opt, get_vector_layout}

* Update CompressedWithVectors format to include base vectors

* StorageGraphLinksVectors

* Clarify code
2025-09-03 07:28:28 +00:00
Arnaud Gourlay
74a37e56be Fix scroll lock ordering (#7203) 2025-09-02 13:28:21 +02:00
dependabot[bot]
db9b29aed9 build(deps): bump procfs from 0.17.0 to 0.18.0 (#7200)
Bumps [procfs](https://github.com/eminence/procfs) from 0.17.0 to 0.18.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](https://github.com/eminence/procfs/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-version: 0.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 13:20:16 +02:00
dependabot[bot]
a8a18f39ea build(deps): bump cc from 1.2.34 to 1.2.35 (#7202)
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.2.34 to 1.2.35.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.2.34...cc-v1.2.35)

---
updated-dependencies:
- dependency-name: cc
  dependency-version: 1.2.35
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 13:09:42 +02:00
Arnaud Gourlay
60de32949e Chain nested locks for proxy segments (#7196)
* Chain nested locks for proxy segments

* Remove get_cloned function on locked segment

---------

Co-authored-by: timvisee <tim@visee.me>
2025-09-02 12:54:12 +02:00
dependabot[bot]
4382b089ef build(deps): bump clap from 4.5.45 to 4.5.46 (#7201)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.45 to 4.5.46.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.45...clap_complete-v4.5.46)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.46
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 12:14:02 +02:00
dependabot[bot]
d313d3c73c build(deps): bump geo from 0.30.0 to 0.31.0 (#7199)
Bumps [geo](https://github.com/georust/geo) from 0.30.0 to 0.31.0.
- [Changelog](https://github.com/georust/geo/blob/main/CHANGES.md)
- [Commits](https://github.com/georust/geo/compare/geo-0.30.0...geo-0.31.0)

---
updated-dependencies:
- dependency-name: geo
  dependency-version: 0.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 12:13:43 +02:00
dependabot[bot]
1985a15432 build(deps): bump actix-files from 0.6.6 to 0.6.7 (#7198)
Bumps [actix-files](https://github.com/actix/actix-web) from 0.6.6 to 0.6.7.
- [Release notes](https://github.com/actix/actix-web/releases)
- [Changelog](https://github.com/actix/actix-web/blob/v0.6.7/CHANGES.md)
- [Commits](https://github.com/actix/actix-web/compare/v0.6.6...v0.6.7)

---
updated-dependencies:
- dependency-name: actix-files
  dependency-version: 0.6.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 12:06:48 +02:00
dependabot[bot]
93824e1835 build(deps): bump config from 0.15.14 to 0.15.15 (#7197)
Bumps [config](https://github.com/rust-cli/config-rs) from 0.15.14 to 0.15.15.
- [Changelog](https://github.com/rust-cli/config-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/config-rs/compare/v0.15.14...v0.15.15)

---
updated-dependencies:
- dependency-name: config
  dependency-version: 0.15.15
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 10:25:25 +02:00
Ivan Pleshkov
2d1b92c16b Create and load for an appendable quantization (#7193)
* Create and load for an appendable quantization

remove feature flag

more todo

review remarks

review remarks

* fix after rebase

* Rename is_appendable to supports_appendable

---------

Co-authored-by: timvisee <tim@visee.me>
2025-09-01 15:49:09 +02:00
Arnaud Gourlay
5c6ccf50a5 Fix duplicate ids in upsert_points (#7194) 2025-09-01 15:29:08 +02:00
xzfc
febf72a47e Generic AccessPattern (#7166) 2025-09-01 11:16:20 +00:00
Tim Visée
7d2adc1b64 build(deps): bump tracing-subscriber from 0.3.19 to 0.3.20 (#7192)
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.19 to 0.3.20.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.19...tracing-subscriber-0.3.20)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.20
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-01 12:07:31 +02:00
tellet-q
ef6e926826 Add hw usage to Facet response (#7189)
* Add usage stats to Facet response
2025-09-01 11:25:17 +02:00
Tim Visée
094fb02e6e Revert "Fix flaky save_on_disk test race condition (#7126)" (#7182)
* Revert "Fix flaky save_on_disk test race condition (#7126)"

This reverts commit 80f0b53cea.

Related flakiness reports:
- https://github.com/qdrant/qdrant/issues/7156
- https://github.com/qdrant/qdrant/issues/7181

* Fix comment
2025-09-01 10:51:23 +02:00
Andrey Vasnetsov
2c6cc22325 do not anonymize peer id (#7177) 2025-08-29 20:09:51 +02:00
Kumar Shivendu
50e9e6f075 Don't push tag with commit id to dockerhub (#7180) 2025-08-29 19:40:47 +05:30
Ivan Pleshkov
b03853c33e Quantization storage builder for chunked mmap (#7174)
* Quantization storage builder for chunked mmap

* fix after rebase

* Update lib/segment/src/vector_storage/quantized/quantized_chunked_mmap_storage.rs

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2025-08-29 16:10:30 +02:00
Ivan Pleshkov
3be4ca8805 Populate multivector quantization offsets (#7173) 2025-08-29 14:05:48 +02:00
Ivan Pleshkov
4b52affe52 Dont use quantization in exact plain search (#7179) 2025-08-29 14:05:06 +02:00
Tim Visée
bb51da71ea Use locked segment by reference without cloning Arc (#7176)
* Use locked segment by reference without cloning Arc

* Remove unnecessary clone

* Return list of lock references
2025-08-29 13:26:51 +02:00
ElekN
971bbffc2b Fix anchor link for Demo projects in README (#7175) 2025-08-29 10:07:53 +00:00
Arnaud Gourlay
71a8d96372 Fix potential deadlock in ProxySegment all_vectors (#7172)
* Fix potential deadlock in ProxySegment all_vectors

* Add comment

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-29 11:06:10 +02:00
Roman Titov
126a3cf558 Move CoreSearchRequest and QueryEnum from collection into shard crate (#7169) 2025-08-29 10:15:58 +02:00
Tim Visée
894692919c Remove chunked mmap vectors mlock flag (#7165) 2025-08-28 15:40:13 +02:00
Tim Visée
d78290b08c build(deps): bump h2 from 4.2.0 to 4.3.0 in /tests (#7167)
Bumps [h2](https://github.com/python-hyper/h2) from 4.2.0 to 4.3.0.
- [Changelog](https://github.com/python-hyper/h2/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/python-hyper/h2/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: h2
  dependency-version: 4.3.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-28 15:36:21 +02:00
Roman Titov
6111bd2634 Move segments_updater from collection into shard crate (#7163) 2025-08-28 15:01:54 +02:00
Roman Titov
1327604648 Make update_worker_fn a blocking task, instead of async (#7015) 2025-08-27 17:26:33 +02:00
tellet-q
5936926c88 Refactor bash test for snapshots recovery (#7008)
* Refactor bash test for snapshots recovery

* Address review: minio check

* Address review
2025-08-27 15:25:27 +02:00
Roman Titov
d17f54c853 Fixes to UpdateHandler::flush_worker (#7151)
- rename `wal_flash_job` to `wal_flush_job`
- correctly handle/report WAL flush errors
2025-08-27 14:16:57 +02:00
tellet-q
6c75566ac4 Add tests for immutable text index (#7153)
* Add unit test

* Flush text index deletes before reopening them

* Fix unused warning if compiling without RocksDB

* Add e2e test

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-27 12:18:58 +02:00
Fabrício Bracht
80f0b53cea Fix flaky save_on_disk test race condition (#7126)
* fix flaky save_on_disk tests with barrier synchronization

* apply suggestions from coderabbitai bot

* fix timeout in test_wait_for_multiple_notifications

* add TEST_IO_SLACK for disk write latency in tests

* make concurrent readers count explicit in test

* relax timing assertions and use timeout variable for consistency

* refactor wait_for to use deadline-based loop to prevent false timeouts

* fix stale variable name in comment

* relax immediate test timing bound to reduce CI flakes

* add overflow protection for deadline calculation

* apply "checked_sub here and unwrap" suggestion

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* track remaining time instead of deadline to avoid overflow issues

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-27 10:55:25 +02:00
Tim Visée
931b467dde Bump dev version to v1.15.5-dev (#7154) 2025-08-27 10:54:15 +02:00
Bastian Hofmann
c975f01193 Fix permissions in Docker Image pipeline (#7152) 2025-08-27 10:17:06 +02:00