Commit Graph

5465 Commits

Author SHA1 Message Date
dependabot[bot]
e5d83fa46e build(deps): bump arc-swap from 1.8.0 to 1.8.1 (#8041)
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 1.8.0 to 1.8.1.
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/compare/v1.8.0...v1.8.1)

---
updated-dependencies:
- dependency-name: arc-swap
  dependency-version: 1.8.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 10:15:12 +01:00
dependabot[bot]
06493fc560 build(deps): bump zerocopy from 0.8.33 to 0.8.37 (#8038)
Bumps [zerocopy](https://github.com/google/zerocopy) from 0.8.33 to 0.8.37.
- [Release notes](https://github.com/google/zerocopy/releases)
- [Changelog](https://github.com/google/zerocopy/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google/zerocopy/compare/v0.8.33...v0.8.37)

---
updated-dependencies:
- dependency-name: zerocopy
  dependency-version: 0.8.37
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 10:15:04 +01:00
dependabot[bot]
8b3af722d0 build(deps): bump bytemuck from 1.24.0 to 1.25.0 (#8037)
Bumps [bytemuck](https://github.com/Lokathor/bytemuck) from 1.24.0 to 1.25.0.
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md)
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.24.0...v1.25.0)

---
updated-dependencies:
- dependency-name: bytemuck
  dependency-version: 1.25.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 10:14:57 +01:00
dependabot[bot]
b4942988b5 build(deps): bump mockito from 1.7.1 to 1.7.2 (#8035)
Bumps [mockito](https://github.com/lipanski/mockito) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/lipanski/mockito/releases)
- [Commits](https://github.com/lipanski/mockito/compare/1.7.1...1.7.2)

---
updated-dependencies:
- dependency-name: mockito
  dependency-version: 1.7.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 10:14:50 +01:00
dependabot[bot]
b22e808a8b build(deps): bump clap from 4.5.54 to 4.5.56 (#8036)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.54 to 4.5.56.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.54...clap_complete-v4.5.56)

---
updated-dependencies:
- dependency-name: clap
  dependency-version: 4.5.56
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-03 10:14:43 +01:00
Anush
446a76318e docs: Add Rust Example For Qdrant Edge (#8020)
Signed-off-by: Anush008 <mail@anush.sh>
2026-02-02 18:18:22 +01:00
Tim Visée
7211d58b77 Apply clippy suggestions (#8029) 2026-02-02 15:11:43 +01:00
Daniel Boros
be0ba7814b feat/ext-api-keys (#7809)
* feat: forward api keys header

* fix: linter

* feat: add api_keys to grpc

* fix: imports

* fix: extract api keys from grpc headers

* refactor: make apikeys extracor more compact

* fix: linter issue

* feat: update header key logic

* fix: clippy issues

* make external keys a part of InferenceParams::new argument, so we wont forget using it

* avoid cloning metadata on header extraction

* combine inference token with ext api key for simpler usage

* clippy

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2026-02-02 14:25:46 +01:00
Anush
43c5746000 chore: Use qdrant/murmur3 fork instead (#8021)
Signed-off-by: Anush008 <mail@anush.sh>
2026-02-02 14:23:22 +05:30
Kyamran Shakhaev
a51c260f92 Remove redundant code (#8013) 2026-02-01 12:19:31 +01:00
Arnaud Gourlay
9f09dbfd3b Update queue info into CollectionInfo (#8010)
* Update queue info into CollectionInfo

* basic test

* fix

* cleaner

* Update lib/collection/src/shards/local_shard/mod.rs

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>

* fix the fix

* rename to op_num

* skip_serializing_if just in case

* first step

* test queue length with staging feature

* gate integration test based on binary feature

---------

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
2026-01-30 17:23:53 +01:00
Tim Visée
36824b9aff Use local telemetry for peer version check, not cluster level telemetry (#8016) 2026-01-30 13:18:03 +01:00
Luis Cossío
4c0d55da7b [disk cache] benchmark other in-memory caches (#7953)
* benchmark against other caches

* use cycle

* use an enum for cache names

* make multithreaded bench with rayon

* don't include pregenerated keys in allocation

* improve benches

* improve keys with zipf + scan

* misc improvements
2026-01-29 14:45:37 -03:00
Tim Visée
cbb212ce3b Remove TODOs for assigning clock tags in forward proxy (#8005) 2026-01-28 12:05:08 +01:00
Arnaud Gourlay
5e986e10d3 Persist applied seq at regular interval (#7976)
* Persist applied seq at regular interval

* Error handling

* fix brain fart

* don't reset update_count

* handles explicitly failure to load

* init from WAL instead of zero

* fixes

* try to regenerate file on corruption

* fix

* trust the WAL if higher watermark most likely from a transfer

* use last WAL index & handles file shard lifecycle

* last WAL index is inclusive

* include applied_seq in partial snapshot

* snapshot applied_seq file

* fix

* remove outdated comment

* Describe AppliedSeq structure, it is used for persisting

---------

Co-authored-by: timvisee <tim@visee.me>
2026-01-28 11:11:04 +01:00
Ivan Pleshkov
9f57ecd6af reduce update signal size by optimizing hw (#8003) 2026-01-28 10:27:36 +01:00
Andrey Vasnetsov
15361e2fb8 more reliable version check (#8002) 2026-01-27 17:07:31 +01:00
dependabot[bot]
112eba2b7b build(deps): bump colored from 3.0.0 to 3.1.1 (#7998)
Bumps [colored](https://github.com/mackwic/colored) from 3.0.0 to 3.1.1.
- [Release notes](https://github.com/mackwic/colored/releases)
- [Changelog](https://github.com/colored-rs/colored/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mackwic/colored/compare/v3.0.0...v3.1.1)

---
updated-dependencies:
- dependency-name: colored
  dependency-version: 3.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 16:38:04 +01:00
dependabot[bot]
6fb437a536 build(deps): bump cc from 1.2.53 to 1.2.54 (#7990)
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.2.53 to 1.2.54.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.2.53...cc-v1.2.54)

---
updated-dependencies:
- dependency-name: cc
  dependency-version: 1.2.54
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 16:02:18 +01:00
tellet-q
00039e1537 Track shard transfer stages and duration (#8001)
* Track shard transfer stages and duration
2026-01-27 15:32:17 +01:00
Ivan Pleshkov
8db893a3b0 Update queue dont keep ops in ram (#7951)
* update queue dont keep ops in ram showcase

* always load operation from WAL

* revert operations buffering

cound only pending in update worker operations

fix typo

use channel size instead of wal index

* remove result expect

* decrease buffering const

* review remarks

* are you happy codespell
2026-01-27 15:18:24 +01:00
Andrey Vasnetsov
5a3793dc63 do not block ToC on in-collection operations by arc-ing collections (#7999)
* do not block ToC on in-collection operations by arc-ing collections

* review suggestion fix
2026-01-27 14:26:06 +01:00
dependabot[bot]
363da13743 build(deps): bump uuid from 1.19.0 to 1.20.0 (#7991)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.19.0 to 1.20.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.19.0...v1.20.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 1.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 10:42:11 +01:00
dependabot[bot]
bc82e0d277 build(deps): bump serde_json from 1.0.148 to 1.0.149 (#7992)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.148 to 1.0.149.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.148...v1.0.149)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-version: 1.0.149
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 10:41:47 +01:00
dependabot[bot]
62ba875400 build(deps): bump quote from 1.0.43 to 1.0.44 (#7995)
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.43 to 1.0.44.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.43...1.0.44)

---
updated-dependencies:
- dependency-name: quote
  dependency-version: 1.0.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 10:19:07 +01:00
dependabot[bot]
ff4f78cc20 build(deps): bump proc-macro2 from 1.0.105 to 1.0.106 (#7996)
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.105 to 1.0.106.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.105...1.0.106)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-version: 1.0.106
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 10:18:34 +01:00
dependabot[bot]
18ef0758ca build(deps): bump sysinfo from 0.37.2 to 0.38.0 (#7994)
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.37.2 to 0.38.0.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.37.2...v0.38.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 09:43:56 +01:00
dependabot[bot]
33f72e409a build(deps): bump nix from 0.30.1 to 0.31.1 (#7997)
Bumps [nix](https://github.com/nix-rust/nix) from 0.30.1 to 0.31.1.
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.30.1...v0.31.1)

---
updated-dependencies:
- dependency-name: nix
  dependency-version: 0.31.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 09:43:07 +01:00
dependabot[bot]
d3c90b3f42 build(deps): bump lukemathwalker/cargo-chef (#7989)
Bumps lukemathwalker/cargo-chef from latest-rust-1.92.0-bookworm to latest-rust-1.93.0-bookworm.

---
updated-dependencies:
- dependency-name: lukemathwalker/cargo-chef
  dependency-version: latest-rust-1.93.0-bookworm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-27 09:41:27 +01:00
Andrey Vasnetsov
81e7ab72fe introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert (#7963)
* introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert

* add test

* upd dockstring

* require resharding once all peers have updated version

* use service error

* fix clippy again

* wait for same version before resharding in tests
2026-01-27 00:39:25 +01:00
xzfc
d7a0f2f882 Update GET collections/{}/optimizations (#7946)
* refactor(SegmentOptimizer::name): make it `&'static`

* refactor(TrackerStatus): add is_running()

* refactor: add TrackerSegmentInfo

* feat: rework optimizations endpoint

* feat(collections/{}/optimizations): trim spaces
2026-01-26 18:03:52 +01:00
Andrey Vasnetsov
64040511bc gridstore iter livelock (#7983)
* granular lock for writing and flushing pending updates in gridstore tracker

* avoid grigstore tracker lock in iterator
2026-01-26 11:47:26 +01:00
Luis Cossío
38da52c819 sort shards by id and peer_id (#7980) 2026-01-26 10:52:00 +01:00
Ivan Pleshkov
ec7c0beaa6 remove obsolete collection update lock (#7979) 2026-01-25 12:06:07 +01:00
xzfc
ae607ee213 Fix hnsw_incremental_build benchmark (#7978)
* Fix hnsw_incremental_build benchmark

* More docs
2026-01-23 15:48:15 +01:00
Ivan Pleshkov
8a729118a3 Remove ChunkedVectorStorage trait (#7977)
* Remove ChunkedVectorStorage trait

* fix rocksdb feature

* fix multidense rocksdb
2026-01-23 13:01:45 +01:00
Andrey Vasnetsov
f1fb912bbc in ram single mmap file (#7971)
* WIP: introduce new vector store type

* handling of InRamMmap

* fmt

* feature-flag

* fmt

* Use if else

Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>

* Update lib/common/common/src/flags.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* also choose madvise for single-file in-ram-mmap

* simplify generics

* gpu fix

* fix bug

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
2026-01-23 11:12:32 +01:00
Roman Titov
1ef06b7fdd Improve SnapshotManifest/SegmentManifest construction (#7961)
* Improve `SnapshotManifest`/`SegmentManifest` construction

* fixup! Improve `SnapshotManifest`/`SegmentManifest` construction

Fix stupidity 😬
2026-01-23 11:09:03 +01:00
Tim Visée
168f2fc625 Hotfix: fix search aggregator error with limit 0 (#7972)
* Hotfix: fix search aggregator error with limit 0

* Don't handle points if limit is zero
2026-01-23 10:15:06 +01:00
Lior.Chen
88567f2b45 fix: replica state peer id remap (#7969) 2026-01-22 19:20:35 +01:00
Tim Visée
8ecd7aa1f1 build(deps): bump lodash in /tools/schema2openapi (#7968)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.17.23.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.21...4.17.23)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.17.23
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-22 10:40:02 +01:00
krapcys1-maker
55352a5933 fix: improve datetime_range RFC3339 parse error (#7919)
* fix: improve REST datetime_range RFC3339 error message

* docs: add docstrings for datetime_range parsing

* fix: avoid serde_json intermediary for non-human-readable deserialization

* fix: avoid serde_json intermediary for non-human-readable range deserialization

* refactor(segment): reduce Condition JSON routing boilerplate

* fix(segment): preserve binary datetime deserialization compatibility

* fix(segment): preserve binary datetime deserialization

* test(segment): add datetime deserialization regression tests

* chore: rerun CI

* chore: rerun CI (2)

* refactor(segment): simplify datetime deserialization
2026-01-22 09:55:05 +01:00
xzfc
95fffe8a12 Sync LocalShard::optimizers (#7962) 2026-01-21 21:11:30 +00:00
xzfc
92b8913084 Enforce segment UUIDs (#7958)
* Swap docstrings for segment_ids/segment_uuids

Terse internal docs, detailed user-facing docs, not vice versa.

* Enforce segment UUIDs

* Export upcoming segment UUID to telemetry

* Add `optimize_for_test` wrapper

* Tune log message

Reason: the UUID is now guaranteed.

* Improve argument naming/docs
2026-01-21 15:43:32 +00:00
Kumar Shivendu
07a6aac584 Allow partial snapshots even when no local shard exists (#7959)
* Allow partial snapshots even when no local shard exists

* Avoid checking for empty old manifest

* Handle CollectionError::NotFound and throw error for others

* fmt

* remove comment

* Cleaner error handling

* Simplify code

* Minor fix
2026-01-21 19:49:51 +05:30
Tim Visée
692bc76f97 Fix optimizer error if segment is already under optimization (#7960) 2026-01-21 14:57:57 +01:00
Luis Cossío
9fca971d93 More graceful handle of flush cancellation (#7798)
* Reapply "return an error when cancelling a flush (#7781)" (#7799)

This reverts commit 4a1103acb7.

* log::trace cancellation of components

* gracefully handle flush cancellation inside segment flusher

* Update lib/segment/src/segment/entry.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* don't wrap cancellation errors

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2026-01-21 10:06:50 -03:00
Arnaud Gourlay
d6afde3d5e Remove unecessary flag in OperationData (#7957) 2026-01-21 13:05:41 +01:00
Roman Titov
7bc947cef3 Always Copy-on-Write when updating payload in immutable segments (#7952) 2026-01-21 11:09:17 +01:00
Andrey Vasnetsov
25252aba37 Generate *.pyi files for EDGE (#7943)
* generate pui files for edge functions

* fix stup for `lambda_`
2026-01-20 18:28:40 +01:00