Commit Graph

5465 Commits

Author SHA1 Message Date
Roman Titov
8764454837 Implement __repr__ for Qdrant Edge types (Part 2) (#7742)
* Implement `__repr__` for `PyJsonPath`

* Implement `__repr__` for `PyFilter`

* Implement `__repr__` for `PyQuery`

* Implement `__repr__` for `PyQueryRequest`

* Implement `__repr__` for `PySearchRequest`

* Implement `__repr__` for `PyFilter` using `pyclass_repr` attribute

* Implement `__repr__` for `PyQuery` using `pyclass_repr` attribute

* Implement `__repr__` for `PyQueryRequest` using `pyclass_repr` attribute

* Implement `__repr__` for `PySearchRequest` using `pyclass_repr` attribute
2025-12-16 03:27:57 +01:00
Roman Titov
1bf968bd10 Implement __repr__ for Qdrant Edge types (#7695)
* Refactor `PyUpdateOperation` constructors

* Add default parameters to `PyVectorDataConfig::new`

* Add `Repr` trait and `WriteExt` helper

* Implement `__repr__` for config types

* fixup! Implement `__repr__` for config types

Use `Copy` instead of `Clone`

* fixup! Implement `__repr__` for config types

Add basic test

* Implement `__repr__` for `PyPointId`

* Implement `__repr__` for `PyVector`

* Implement `__repr__` for `PyVectorInternal`

* Implement `__repr__` for `PyPayload`

* Implement `__repr__` for `PyValue`

* Implement `__repr__` for `PyPoint`

* Implement `__repr__` for `PyPointVectors`

* Implement `__repr__` for `PyRecord`

* Move `PyScoredPoint` into a separate file

* Implement `__repr__` for `PyScoredPoint`

* Cleanup examples

* fixup! Implement `__repr__` for `PyScoredPoint`

* Move `PyOrderValue` into separate file

* Add `PyScoredPoint::order_value`

* Implement `pyclass_repr` proc-macro attribute

* Implement `__repr__` for config types using `pyclass_repr` attribute

* Implement `__repr__` for `PySparseVector` using `pyclass_repr` attribute

* Implement `__repr__` for `PyPoint` using `pyclass_repr` attribute

* Implement `__repr__` for `PyPointVectors` using `pyclass_repr` attribute

* Implement `__repr__` for `PyRecord` using `pyclass_repr` attribute

* Implement `__repr__` for `PyScoredPoint` using `pyclass_repr` attribute

* Minor fixes and cleanups

* fixup! Minor fixes and cleanups

* rollback copy for quantization config

* rollback copy for quantization config

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-12-16 02:17:37 +01:00
Kumar Shivendu
5b71996247 urlencode collection_name on calling partial snapshot create API (#7782) 2025-12-16 00:16:30 +01:00
Luis Cossío
4e91702fe8 Introduce IsAliveLock (#7751)
* introduce IsAliveLock

* add tests

* rename to mark_dead

* update gridstore comment

* Update some comments

* Add must_use attribute to lock_if_alive

* Rename mark_dead to blocking_mark_dead

* make the handle take a `Weak` reference

* make dropping explicit

* use `Mutex::lock_arc` instead

* less nesting

* clippy

---------

Co-authored-by: timvisee <tim@visee.me>
2025-12-15 12:22:30 -03:00
Arnaud Gourlay
9d3fb28862 Cleanup shard level search timeout (#7754) 2025-12-15 11:27:08 +01:00
Andrey Vasnetsov
9401f15684 use urlencode to prevent breaking collection name with % in it (#7759)
* use urlencode to prevent breaking collection name with % in it

* do the same for snapshot_description.name
2025-12-14 12:50:31 +01:00
Arnaud Gourlay
5bb2db5837 Cross reference telemetry timeout value (#7753) 2025-12-12 11:42:04 +01:00
Tim Visée
aadadd172e Gridstore: split pointer updates set/unset, simplify draining (#7749)
* Separate set and unset lists for each Gridstore pointer update

This makes the list of pointer updates much easier to grasp. The
simplification is desired because this exact structure has been a cause
for bugs multiple times now.

* Add more aggressive debug assertions

* Use consistent terminology

* Correct removal of set

* Simplify drain function

* Add safe guard to ignore empty pointer updates

* Rework tracker getter, explicitly branch variants

* Patch transmute, require Sized type

* rename and adjust comments

- set/unset is now current/to_free
- adjusted descriptions and comments for this nomenclature too

* clippy

* clippyyy

not 4, not 2, but 3 spaces

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-12-11 18:07:15 +01:00
Arnaud Gourlay
104f6ab4eb Fix Clippy v1.92 (#7748) 2025-12-11 17:05:24 +01:00
Tim Visée
1d03df1f8b Remove unused swap_existing function from segment holder (#7736) 2025-12-11 16:49:39 +01:00
Arnaud Gourlay
cb70be4969 Test Gridstore corruption on flush (#7713)
* Gridstore decompress failure

* more logs

* more logs

* hardening

* clean

* no compression

* cleanup

* typo

* misc

* wait for flush & less logs

* add compression type to the mix

* optimize
2025-12-11 15:13:16 +01:00
Arnaud Gourlay
6b6981cef0 Bump misc. integration test deps (#7746) 2025-12-11 13:25:37 +01:00
Tim Visée
14ebe94c90 Fix Gridstore flushing, correctly implement drain persist (#7741)
* Fix Gridstore tracker drain and persist not working properly

* Patch existing test

* Add new test to assert buggy scenario we found

* Mention PR in test
2025-12-11 10:27:40 +01:00
Arnaud Gourlay
21d52ba832 Optimize gridstore flush (#7740)
* Optimize gridstore flush

* lovely early return
2025-12-10 19:42:57 +01:00
Daniel Boros
69db4c56e7 feat/update-timeout (#7694)
* feat: add timeout to update params

* feat: update timeout params

* feat: add timeout for grpc

* fix: add missing timeout params

* fix: clippy warnings

* feat: update openapi definition

* chore: format

* fix: redoc master.json

* feat: add missing timeout to grpc api

* fix: missing timeout

* chore: update comments
2025-12-10 13:37:01 +01:00
tellet-q
82ca36ed37 Download archives once (#7711) 2025-12-10 10:04:19 +01:00
Andrey Vasnetsov
12815e81ff ignore shards with no active replicas (#7707) 2025-12-09 16:55:25 +01:00
Andrey Vasnetsov
152172f143 refactor replica set (#7706)
* peer-id change on snapshot recovery

* move peer state management into a dedicated file
2025-12-09 14:09:52 +01:00
Tim Visée
bdd3871f29 Fix flusher data race in Gridstore (#7702)
* Add lease structure to invalidate pending flushers after wipe/clear

* We don't use bitmask as barrier anymore

* Add missing early return

* Add test

* Replace flush lease with is alive boolean we also use elsewhere

* Reimplement test, now it does fail on the old implementation

* In test, cover both new fixed and old broken flushing

* Remove old test case because it is flaky

* Fix clippy

* Fix outdated comment
2025-12-09 11:23:56 +01:00
Arnaud Gourlay
76bfa2ef7c Fix to run all tests on CI (#7732)
* Fix to run all tests on CI

* adjust for macos since we bumped rocksdb

* more flags
2025-12-09 11:09:52 +01:00
Jojii
1e3f67cf47 Metrics incoming and outgoing shard transfers (#7701)
* Metrics incoming and outgoing shard transfers

* clippy

* Revert counting separately
2025-12-09 10:04:56 +01:00
dependabot[bot]
0794ba104c build(deps): bump indexmap from 2.11.4 to 2.12.1 (#7719)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.11.4 to 2.12.1.
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/main/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.11.4...2.12.1)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-version: 2.12.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 08:37:33 +01:00
dependabot[bot]
742b447c15 build(deps): bump criterion from 0.7.0 to 0.8.1 (#7721)
Bumps [criterion](https://github.com/criterion-rs/criterion.rs) from 0.7.0 to 0.8.1.
- [Release notes](https://github.com/criterion-rs/criterion.rs/releases)
- [Changelog](https://github.com/criterion-rs/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/criterion-rs/criterion.rs/compare/criterion-plot-v0.7.0...criterion-v0.8.1)

---
updated-dependencies:
- dependency-name: criterion
  dependency-version: 0.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 08:33:16 +01:00
dependabot[bot]
36b8c0a58d build(deps): bump self_cell from 1.2.0 to 1.2.1 (#7724)
Bumps [self_cell](https://github.com/Voultapher/self_cell) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/Voultapher/self_cell/releases)
- [Commits](https://github.com/Voultapher/self_cell/compare/v1.2.0...v1.2.1)

---
updated-dependencies:
- dependency-name: self_cell
  dependency-version: 1.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 08:03:36 +01:00
dependabot[bot]
d920f7dec8 build(deps): bump flate2 from 1.1.4 to 1.1.5 (#7726)
Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.1.4...1.1.5)

---
updated-dependencies:
- dependency-name: flate2
  dependency-version: 1.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 07:53:17 +01:00
dependabot[bot]
9be2f1fce9 build(deps): bump mockito from 1.7.0 to 1.7.1 (#7728)
Bumps [mockito](https://github.com/lipanski/mockito) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/lipanski/mockito/releases)
- [Commits](https://github.com/lipanski/mockito/compare/1.7.0...1.7.1)

---
updated-dependencies:
- dependency-name: mockito
  dependency-version: 1.7.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-09 07:52:10 +01:00
xzfc
0a490dc420 Fix shell.nix for tikv-jemallocator 0.6.1 (#7697) 2025-12-09 00:49:22 +00:00
xzfc
e617a4b777 Ignore deprecation warnings for schemathesis (#7730) 2025-12-09 00:16:37 +00:00
xzfc
77c71712db Indexing progress (#7625)
* Add progress_tracker.rs

* Pass progress tracker around

* Populate progress tracker with actual data

* Expose progress on `/collections/{name}/optimizations` endpoint
2025-12-09 00:15:36 +00:00
Kumar Shivendu
1962d89978 Log snapshot download duration (#7715)
* Log snapshot download duration

* only 2 digits in filesize

* Log for normal snapshots too

* fmt

* Log download stats for s3 snapshot API and cleanup
2025-12-09 01:13:09 +05:30
dependabot[bot]
93d2f5f96c build(deps): bump actix-web from 4.11.0 to 4.12.1 (#7720)
Bumps [actix-web](https://github.com/actix/actix-web) from 4.11.0 to 4.12.1.
- [Release notes](https://github.com/actix/actix-web/releases)
- [Changelog](https://github.com/actix/actix-web/blob/main/CHANGES.md)
- [Commits](https://github.com/actix/actix-web/compare/web-v4.11.0...web-v4.12.1)

---
updated-dependencies:
- dependency-name: actix-web
  dependency-version: 4.12.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-08 20:35:22 +01:00
dependabot[bot]
a93c194899 build(deps): bump pyo3 from 0.27.1 to 0.27.2 (#7725)
Bumps [pyo3](https://github.com/pyo3/pyo3) from 0.27.1 to 0.27.2.
- [Release notes](https://github.com/pyo3/pyo3/releases)
- [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pyo3/pyo3/compare/v0.27.1...v0.27.2)

---
updated-dependencies:
- dependency-name: pyo3
  dependency-version: 0.27.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-08 20:34:59 +01:00
dependabot[bot]
dd0508fb1b build(deps): bump actix-files from 0.6.8 to 0.6.9 (#7722)
Bumps [actix-files](https://github.com/actix/actix-web) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/actix/actix-web/releases)
- [Changelog](https://github.com/actix/actix-web/blob/v0.6.9/CHANGES.md)
- [Commits](https://github.com/actix/actix-web/compare/v0.6.8...v0.6.9)

---
updated-dependencies:
- dependency-name: actix-files
  dependency-version: 0.6.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-08 20:00:28 +01:00
dependabot[bot]
2048fa85b2 build(deps): bump memmap2 from 0.9.8 to 0.9.9 (#7723)
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.9.8 to 0.9.9.
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.9.8...v0.9.9)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-version: 0.9.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-08 19:57:56 +01:00
Andrey Vasnetsov
6fe46d0d60 refactor update workers (#7642)
* refactor: move flush workers into a dedicated file

fmt

* move optimizer worker into a dedicated file

* move update worker into a dedicated file

* Update lib/collection/src/update_workers/optimization_worker.rs

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update lib/collection/src/update_workers/optimization_worker.rs

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-12-08 11:51:02 +01:00
tellet-q
20aca3f1f0 Add staging endpoint for testing point operation delays (#7687)
* Add endpoint to execute delay instead of point update

---------

Co-authored-by: timvisee <tim@visee.me>
2025-12-08 08:16:09 +01:00
Jojii
55e5702e9d Metrics cluster working state and last update (#7670)
* cluster working state and last update in metrics

* Rename metric

* Remove error string

* Use timestamp instead

* Fix prometheus help text

* Change metric type to counter
2025-12-05 15:47:37 +01:00
Roman Titov
7967c12b4a Fix test_collection_recovery test (#7700) 2025-12-05 13:20:37 +01:00
Arnaud Gourlay
c145ae051e Align version of jemalloc crates (#7699) 2025-12-05 12:16:46 +01:00
Copilot
8954a499f1 Use Duration type for timeout errors instead of truncating to seconds (#7689)
* Initial plan

* Update timeout error handling to use milliseconds for better precision

Co-authored-by: agourlay <606963+agourlay@users.noreply.github.com>

* Use Duration type and debug format for timeout errors

Changed timeout() methods to accept Duration directly instead of u128 milliseconds.
This uses Rust's built-in Duration debug formatting ({:?}) which provides clean output:
- Sub-second: "500ms"
- Exact seconds: "1s"
- Mixed: "2.5s"

Updated all call sites to pass Duration directly instead of timeout.as_millis().

Co-authored-by: agourlay <606963+agourlay@users.noreply.github.com>

* fmt/clippy

* make it consistent

* Inline some arguments and imports

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: agourlay <606963+agourlay@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: timvisee <tim@visee.me>
2025-12-05 11:27:51 +01:00
Tim Visée
64690cda53 Hotfix for Windows ARM64 builds, disable some Neon features (#7690)
* On Windows ARM64 builds, disable usage of neon

* Also disable optimized popcount on Windows ARM64

* fix quantization build

* revert changes in BQ

---------

Co-authored-by: Ivan Pleshkov <pleshkov.ivan@gmail.com>
2025-12-05 10:56:48 +01:00
Arnaud Gourlay
e8f8e3fc86 Print backtrace when incomplete shard drop (#7675)
* Print backtrace when incomplete shard drop

* comment by default to avoid noisy output on shutdown
2025-12-04 16:37:01 +01:00
Luis Cossío
d0fcc10c10 [cluster telemetry] internal service types (#7599)
* stripped down grpc types for peer telemetry

* return err instead of panic

* use correct definitions

* improve types
2025-12-04 12:08:19 -03:00
Arnaud Gourlay
df023e7cdb Improve telemetry timeout for shard size_stats (#7685)
* Improve telemetry timeout for shard size_stats

* remove explicit drops
2025-12-04 16:06:11 +01:00
Arnaud Gourlay
da8ccdd253 Timeout on point retrieve when acquiring segment holder (#7588)
* Timeout on point retrieve when acquiring segment holder

* review: less radical and cleaner

* highly embarassing I hope no one sees this!

* coderabbit
2025-12-04 15:26:40 +01:00
dependabot[bot]
a1ab6c3cc1 build(deps): bump nix from 0.29.0 to 0.30.1 (#7655)
* build(deps): bump nix from 0.29.0 to 0.30.1

Bumps [nix](https://github.com/nix-rust/nix) from 0.29.0 to 0.30.1.
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.29.0...v0.30.1)

---
updated-dependencies:
- dependency-name: nix
  dependency-version: 0.30.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix interface change

b2f34f7423 (diff-234e7e6580542ac96403821955043ffefa4cef1e0659216a9ee170cad6315c7dL1388)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2025-12-04 13:19:37 +01:00
Roman Titov
bc7b76c0db Qdrant Edge Python bindings improvements (Part 2) (#7639) 2025-12-04 12:02:21 +01:00
Tim Visée
3d3c230461 Include RocksDB features in Docker dev build (#7683)
* Include RocksDB features in Docker dev build

* Configure flags in a different way, without else branch
2025-12-04 11:55:00 +01:00
Roman Titov
4c9b843db4 Mark *new* local replicas as locally-disabled, when recovering Raft snapshot (#7684)
* Remove faulty mark-as-dead condition when recovering Raft snapshot 😅

* Mark new local replicas as locally-disabled when recovering Raft snapshot

* Add test
2025-12-03 18:12:07 +01:00
Arnaud Gourlay
e01babffab Cleanup a few clones (#7681) 2025-12-03 13:53:17 +01:00