630 Commits

Author SHA1 Message Date
xzfc
ad6d946327 rusdoc: use stable URLs 2026-04-07 21:13:58 +00:00
Arnaud Gourlay
a4059c15af Remove unused dependency crates (#8589) 2026-04-01 18:15:18 +02:00
Kyamran Shakhaev
ee7baa668f Use OperationError funcs (#8587)
* Use OperationError funcs

* Inline variable

* Inline variable

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2026-04-01 15:25:11 +02:00
Daniel Boros
974fcb63c7 Feat/storage api over universal io (#8311)
* feat: add storage_read_service proto definition

* feat: update qdrant tonic definition

* feat: add list_files

* feat: add read_bytes

* feat: add read_bytes_stream

* feat: add read_batch & read_whole api

* feat: add read_multi

* fix: mod.rs

* feat: update resolve_path

* feat: add path resolve

* feat: add more tests and fix stream api

* feat: better error handling

* feat: add collection_base_path

* fix: potential EOF error

* fix: nested runtime panic on tests

* fix: github actions

* fix: auth tests

* fix: coderabbit pr review

* fix: linter

* fix: clippy

* fix: range validation and path resolve

* fix: linter

* fix: clippy

* fix: clippy

* fix: universal io error

* fix: incoming changes

* fix: compiler error

* feat: spilt impl

* fix: pr review

* fix: linter

* fix: dev changes

* fix: linter

* review fixes

* fmt

* chore: fix naming

* fix: pr reviews

* fix: rebase dev

* fix: clippy

* fix: tests

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2026-03-29 23:08:16 +02:00
Bhagirath Kapdi
fb704e5d13 Feature/strict mode search max batchsize (#8469)
* feat: add search_max_batchsize to strict mode config

* added test case for search_max_batchsize

* Changes for fixing CI issue dure openapi

* Modify check_strict_mode_batch

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2026-03-26 16:26:09 +01:00
qdrant-cloud-bot
ccf6f7a680 Add REST API for reading audit logs across the cluster (#8498)
* Add REST API for reading audit logs across the cluster

Introduces a new `GET /audit/logs` endpoint that retrieves audit log
entries from all peers in the cluster. The API supports filtering by
time range (time_from/time_to), dynamic key=value field filters, and
a built-in limit parameter to prevent reading too many logs at once.

- Add `audit_reader` module in storage crate for efficient file-based
  log retrieval, selecting only files whose date range overlaps the
  query window
- Add `GetAuditLog` internal gRPC RPC for cross-peer log retrieval
- Add `GET /audit/logs` REST endpoint restricted to management access
- Aggregate and sort results from all peers by timestamp (newest first)

Made-with: Cursor

* [AI] Update filter_files_by_time_range make it aware of the log rotation granulatity (either hourly or daily)

* [AI] For AuditLogParams, try to use DateTime types natively into serde, instead of manual parsing

* [AI] Refactor API into separate file, propagate timeout, use spawn-blocking

* [AI] introduce cancellation token

* [AI] move timestamp to constant

* small manual fixes

* review fixes part 1

* review: switch to POST instead of GET

* [AI] review: sorting update

* [AI] use strict typing

---------

Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2026-03-25 12:46:53 +01:00
qdrant-cloud-bot
dcd5f7a7ed Add struct destructuring to catch missing fields at compile time (#8497)
Destructure structs in conversions, key/match functions, and config
builders so the compiler will error when new fields are added but not
handled. This prevents a common class of bugs where a newly added
struct field is silently ignored.

Covered spots:
- RecommendPointGroups → RecommendGroupsRequestInternal (15 fields)
- ShardTransferTelemetry ↔ ShardTransferInfo (7 fields, both directions)
- ReshardingTelemetry ↔ ReshardingInfo (6 fields, both directions)
- HwMeasurementAcc → HardwareUsage (7 fields, 3 call sites)
- VectorParams destructuring in optimizers_builder and config (7 fields)
- SparseVectorParams destructuring in optimizers_builder (2 fields)
- ShardTransfer → ShardTransferRestart (5 fields)
- ReshardState::matches and ReshardState::key (5 fields each)
- ShardTransfer::key and ShardTransferRestart::key (4 fields each)
- SnapshotDescription → grpc (4 fields)
- WithLookup REST → internal (3 fields)
- SegmentConfigV5 → SegmentConfig (5 fields)

Made-with: Cursor

Co-authored-by: Cursor Agent <agent@cursor.com>
2026-03-24 15:53:56 +01:00
Andrey Vasnetsov
77e5c326e9 [AI] implement wait override (#8476) 2026-03-23 13:13:55 +01:00
Jojii
c015b86515 Add deferred point count to UpdateQueueInfo (#8414)
* Add deferred point count to UpdateQueueInfo

* Hide if prevent_unoptimized is false

* Clippy

* Openapi

* Iterate over appendable segments only

* Only calculate deferred point count if prevent_unoptimized is true

* Rebase fixes

* Codespell
2026-03-19 15:27:23 +01:00
xzfc
8ab0057566 DiscoveryQuery -> DiscoverQuery (#8378) 2026-03-12 19:54:34 +01:00
qdrant-cloud-bot
d7ad2d45f7 fix: codespell 2.4.2 - pre-selected -> preselected, pre-select -> preselect (#8303)
* fix: codespell 2.4.2 - pre-selected -> preselected, pre-select -> preselect

Fixes CI failure with codespell 2.4.2 which flags hyphenated forms.
Updated in: types.rs, query/mod.rs, qdrant.rs, points.proto

Made-with: Cursor

* chore: regenerate OpenAPI spec (tools/generate_openapi_models.sh)

Updates oversampling description to use preselected spelling.

Made-with: Cursor

---------

Co-authored-by: Cursor Agent <agent@cursor.com>
2026-03-06 11:47:16 +01:00
xzfc
628195fafe Remove shard dependency on api (#8284)
* Move `DenseVector`/`MultiDenseVector` from `api` to `segment`

* Move `OrderByInterface` from `api` to `segment`

Reason: it's used in `edge` which shouldn't depend on `api`.

* Make `shard` -> `api` dependency optional

* Remove `api` from the amalgamation

* Don't install protoc in edge Actions
2026-03-05 04:38:02 +00:00
generall
730439a62a new formatter rules 2026-02-28 23:21:07 +01:00
dependabot[bot]
18a7587d4b build(deps): bump rand_distr from 0.5.1 to 0.6.0 (#8148)
* build(deps): bump rand_distr from 0.5.1 to 0.6.0

Bumps [rand_distr](https://github.com/rust-random/rand_distr) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/rust-random/rand_distr/releases)
- [Changelog](https://github.com/rust-random/rand_distr/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand_distr/compare/0.5.1...0.6.0)

---
updated-dependencies:
- dependency-name: rand_distr
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Migrate main code base to rand 0.10

* Migrate tests

* Migrate benches

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: timvisee <tim@visee.me>
2026-02-25 14:15:04 +01:00
xzfc
28d9c5be12 Single edge crate (#8173)
* Fixups of amalgamator

Fix issues that break `qdrant-edge` build process:
- `use … as segment;` - this causes `ast-grep` rules to replace wrong
  paths. So, rename to avoid collisions.
- `#[macro_use]` and `extern crate` required be in the top-level
  `lib.rs`.
- `format!("…", crate::something::…)` - `ast-grep` can't fix paths
  inside macros. Fixed by moving `crate::something::…` out of the macro.

* Add lib/edge/publish workspace and amalgamation script

* Move `lib/edge/examples` into `lib/edge/publish/` workspace

And fix them to use the generated `qdrant-edge` crate.

* Add github workflow

* Cleanup `qdrant-edge` public API

Removes empty modules. Checked by `cargo doc`.
2026-02-24 16:43:59 +00:00
Tim Visée
c73cf450e8 Bump dev version to 1.17.1-dev (#8161) 2026-02-19 13:36:14 +01:00
Andrey Vasnetsov
6e79f8bf02 Untagged Enum for FeedbackStrategy (#8171)
* [manual] make untagged enum for consistent API for FeedbackStrategy

* add #[validate(nested)]

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2026-02-18 16:55:47 +01:00
Roman Titov
5bb06d9ec0 Fix too_many_internal_resets error (#8128)
Patch `tonic` and `hyper` crates to expose `max_local_error_reset_streams`,
and *disable* it when creating internal gRPC connections
2026-02-13 18:21:56 +01:00
Arnaud Gourlay
5ffc6a45a8 Remove update queue opnum from CollectionInfo (#8127) 2026-02-13 17:36:54 +01:00
Andrey Vasnetsov
171b7cffba distributed optimization info (#8120)
* [AI] initial implementation

* [AI] review fixes

* [manaul] some small fix

* [AI] fix issue about serializing optional fields

* upd schema

* [manual] review fixes

* [manual] only ask updatable replicas about optimizations
2026-02-13 13:58:17 +01:00
Andrey Vasnetsov
e4bafb04f5 undeprecated vector conversions (#8114) 2026-02-12 13:34:55 +01:00
Luis Cossío
a43780941b [relevance feedback] add rest and grpc interfaces (#7399)
* add rest and grpc interfaces

Also handle inference for this query

* follow refactor from base branch

* renaming from Ms Cooper

* get started on validation testing

* more validations

* test equivalence with query when less than 2 feedback elements

* rename feedback query to relevance_feedback query

* fix extraction of context pairs

* rename feedback vector to example

* make coderabbit happier

* upd test

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2026-02-11 16:05:11 +01:00
Andrey Vasnetsov
4437edb775 Weighted rrf (#8063)
* weighted rrf implementation

* test

* fmt

* fix edge

* validate number of sources and number of weights

* do not partial match

* upd schema

* review fixes

* update formula

* remove calcualtions from tests

* update comment, because AI have OCD

* fmt
2026-02-06 19:36:27 +01:00
Arnaud Gourlay
c7438cce9d Attempt to Fix RST_STREAM by draining http2 channels (#8019)
* Fix RST_STREAM by draining http2 channels

* remove controversial duration time for now

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
2026-02-05 14:07:48 +01:00
Arnaud Gourlay
9f09dbfd3b Update queue info into CollectionInfo (#8010)
* Update queue info into CollectionInfo

* basic test

* fix

* cleaner

* Update lib/collection/src/shards/local_shard/mod.rs

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>

* fix the fix

* rename to op_num

* skip_serializing_if just in case

* first step

* test queue length with staging feature

* gate integration test based on binary feature

---------

Co-authored-by: xzfc <5121426+xzfc@users.noreply.github.com>
2026-01-30 17:23:53 +01:00
Andrey Vasnetsov
81e7ab72fe introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert (#7963)
* introduce update_mode parameter for upsert operation to control if we want to insert, update, or upsert

* add test

* upd dockstring

* require resharding once all peers have updated version

* use service error

* fix clippy again

* wait for same version before resharding in tests
2026-01-27 00:39:25 +01:00
Luis Cossío
b4c9df1f3f [cluster telemetry] Include shards information (#7886)
* make some existing fields optional

particularly, RemoteShardTelemetry's `searches` and `updates`, and
LocalShardTelemetry's `optimizations`

* Include shards information in cluster telemetry

* upd openapi

* nits

* only include partial snapshot info if non-empty

* fixes
2026-01-19 11:34:37 -03:00
Andrey Vasnetsov
b94a13fd6c implement a new configuration for read fan-out delay (#7929)
* implement a new configuration for read fan-out delay

* upd schema

* Update read_fan_out_delay_ms comment

---------

Co-authored-by: timvisee <tim@visee.me>
2026-01-19 14:38:54 +01:00
Andrey Vasnetsov
921b1ce83d Prevent unoptimized updates (#7643)
* wip: wait for optimization before applying update

* implement api parameter

* nits

* fix deadloack when no optimizers are running

* release handle mutex

* Derive PartialEq

* Remove unused function

* fix missing kb conversion in threshold

* Update lib/api/src/grpc/proto/collections.proto

Co-authored-by: Tim Visée <tim+github@visee.me>

* sync comment change

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
2026-01-14 16:37:03 +01:00
Sapphire
a494452524 Add enable_hnsw option for payload field schema (#7887)
* feat: Add enable_hnsw option for payload field indexes

Add optional enable_hnsw parameter to all payload index types to control
whether additional HNSW graph links are built for each indexed field.

- Add enable_hnsw field to all 8 payload index param types
- Update gRPC proto definitions and conversions
- Update OpenAPI schema
- Modify HNSW graph builder to respect enable_hnsw flag
- Add enable_hnsw() helper methods to PayloadSchemaParams and PayloadFieldSchema
- Update all tests to include new field (default: None)

When enable_hnsw is true and payload_M > 0, additional HNSW links will
be built for the payload field. Default value is true for backward compatibility.

* Fix Some format problems

* fix: address comment problem

---------

Co-authored-by: EC2 Default User <ec2-user@ip-10-78-171-148.ec2.internal>
2026-01-09 11:39:44 +01:00
Daniel Boros
1cbf297f9e feat/shard-ops-timeout (#7750)
* feat: add shard-ops timeout

* feat: add shard-ops timeout

* feat: add missing timeout

* fix: format

* fix: type serialization

* fix: bad wal flush logic

* feat: add seperate wait_time

* fix: linter issues

* fix: linter

* chore: make select more readable

* fix: linter issues

* chore: remove comment

* chore: adjust comment

* chore: update definition

* chore: remove extra wait_timeout -> make timeout global

* chore: remove wait_timeout from point_ops

* chore: remove wait_timeout from grpc

* refactor: make shard_ops more readable

* chore: remove timeout from update_all

* feat: propagate wait_timeout error

* fix: clippy recommendation

* chore: update wait timeout

* refactor: update merge_successful_update_results

* fix: tests inconsistency

* fix: revert global ord, partialord for UpdateStatus

* fix: integration test

* fix: res ingoring

* fix: review requests

* fix: clippy

* chore: revert back futures unordered

* fix: resolve response status

* force timeout for remote operations

* fix: pr reviews

* fix: switch to duration

* Bump OpenAPI spec

* feat: add serde_as

* fix: timeout statu condition

* Minor tweaks

* Use tokio timeout directly rather than deadline

* Revert schema generator permissions

* Remove obsolete semi-colon

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: timvisee <tim@visee.me>
2026-01-09 10:24:45 +01:00
Luis Cossío
7dbb994f6f [cluster telemetry] Include peers versions (#7848)
* wrap details in another struct, less Options

* include version in peers info

* update openapi

* include app telemetry in internal service

* move debug assertion after check
2026-01-08 14:35:31 -03:00
tellet-q
c0b70f1eae Add a test for recovery after kill during Partial (#7762)
* Add a test for recovery after kill during Partial

* Address AI review

* simplify test

* Merge pull request #7829

* introduce a new state

* switch to ManualRecovery for user-initiated snapshot operations

* fmt

* fix test and regen api

* test fix: instead of force recovery on update failed snapshots now re…

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2026-01-02 14:33:47 +01:00
Luis Cossío
b44596f477 allow collection-level jwt access (#7758) 2025-12-29 14:29:02 -06:00
Luis Cossío
8ab86b6736 [cluster telemetry] Internal telemetry endpoint (#7672)
* move QdrantInternalService to its own file

* implement internal telemetry endpoint

cleaner diff on src/main.rs

upd response in proto

* use try_from conversion

* rename request/response in proto
2025-12-23 11:32:44 -06:00
Luis Cossío
64c806d8aa [cluster telemetry] Internal service conversions (#7631)
* implement conversions

* clippy

* rename `ReshardStage` -> `ReshardingStage`

* impl conversions to grpc

* upd response in proto

* convert TelemetryData

* add peer info

* upd openapi

* add conversion from TelemetryData to PeerTelemetry

* review nit

* re-style

* make `method` optional

* change comments to reference `ReshardingStreamRecords`

* missing nits
2025-12-23 11:00:25 -06:00
Tim Visée
9296874f9b Bump dev version to 1.16.4-dev (#7807) 2025-12-19 13:23:40 +01:00
xzfc
d093bd97ce Re-style .proto files (#7475)
* proto: doc comment style

* proto: clang-format

* proto: manually reflow some comments

* proto: proofread
2025-12-16 22:22:54 +00:00
Luis Cossío
e1b38e1723 [feedback query] Rename to Naive strategy (#7756)
* Rename to `Naive` strategy. Remove generic `TStrategy`

* refactor in async_raw_scorer too

* review nit
2025-12-16 15:17:25 -03:00
Daniel Boros
69db4c56e7 feat/update-timeout (#7694)
* feat: add timeout to update params

* feat: update timeout params

* feat: add timeout for grpc

* fix: add missing timeout params

* fix: clippy warnings

* feat: update openapi definition

* chore: format

* fix: redoc master.json

* feat: add missing timeout to grpc api

* fix: missing timeout

* chore: update comments
2025-12-10 13:37:01 +01:00
Luis Cossío
d0fcc10c10 [cluster telemetry] internal service types (#7599)
* stripped down grpc types for peer telemetry

* return err instead of panic

* use correct definitions

* improve types
2025-12-04 12:08:19 -03:00
Tim Visée
d523d9fb99 Bump dev version to 1.16.3-dev (#7682) 2025-12-03 13:51:14 +01:00
AMIR
4f13b994e3 Add list shard keys API (#7615) 2025-11-28 21:50:09 -03:00
Tim Visée
e630cf59fa Bump dev version to 1.16.2-dev (#7604) 2025-11-25 15:01:34 +01:00
Andrey Vasnetsov
59ee168dcd rename common -> qdrant_common to avoid name with other python packages (#7571) 2025-11-21 15:01:54 +01:00
Arnaud Gourlay
b784799b69 Missing validations for shard transfers (#7557)
* Missing validations for shard transfers

* some more

* fix test
2025-11-18 13:08:51 +01:00
Tim Visée
77cd5962c2 Bump dev version to v1.16.1-dev (#7548) 2025-11-17 15:45:10 +01:00
Andrey Vasnetsov
355cf3624c remove java_outer_classname (#7539) 2025-11-14 18:24:58 +01:00
Andrey Vasnetsov
4420165464 Implement ReplicatePoints in grpc (#7536)
* implement ReplicatePoints in grpc

* move filter into a separate file
2025-11-14 16:12:29 +01:00
Andrey Vasnetsov
e34a6e126e Mark legacy output vector fields deprecated (#7183)
* mark legacy output vector fields deprecated

* grpc docs

* Also mark legacy vector input fields as deprecated

---------

Co-authored-by: timvisee <tim@visee.me>
2025-11-12 15:21:54 +01:00