upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 10:00:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 344 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Tim Visée
168f2fc625 Hotfix: fix search aggregator error with limit 0 (#7972) 
* Hotfix: fix search aggregator error with limit 0

* Don't handle points if limit is zero
 2026-01-23 10:15:06 +01:00
Lior.Chen
88567f2b45 fix: replica state peer id remap (#7969) 2026-01-22 19:20:35 +01:00
krapcys1-maker
55352a5933 fix: improve datetime_range RFC3339 parse error (#7919) 
* fix: improve REST datetime_range RFC3339 error message

* docs: add docstrings for datetime_range parsing

* fix: avoid serde_json intermediary for non-human-readable deserialization

* fix: avoid serde_json intermediary for non-human-readable range deserialization

* refactor(segment): reduce Condition JSON routing boilerplate

* fix(segment): preserve binary datetime deserialization compatibility

* fix(segment): preserve binary datetime deserialization

* test(segment): add datetime deserialization regression tests

* chore: rerun CI

* chore: rerun CI (2)

* refactor(segment): simplify datetime deserialization
 2026-01-22 09:55:05 +01:00
xzfc
95fffe8a12 Sync LocalShard::optimizers (#7962) 2026-01-21 21:11:30 +00:00
xzfc
92b8913084 Enforce segment UUIDs (#7958) 
* Swap docstrings for segment_ids/segment_uuids

Terse internal docs, detailed user-facing docs, not vice versa.

* Enforce segment UUIDs

* Export upcoming segment UUID to telemetry

* Add `optimize_for_test` wrapper

* Tune log message

Reason: the UUID is now guaranteed.

* Improve argument naming/docs
 2026-01-21 15:43:32 +00:00
Kumar Shivendu
07a6aac584 Allow partial snapshots even when no local shard exists (#7959) 
* Allow partial snapshots even when no local shard exists

* Avoid checking for empty old manifest

* Handle CollectionError::NotFound and throw error for others

* fmt

* remove comment

* Cleaner error handling

* Simplify code

* Minor fix
 2026-01-21 19:49:51 +05:30
Tim Visée
692bc76f97 Fix optimizer error if segment is already under optimization (#7960) 2026-01-21 14:57:57 +01:00
Luis Cossío
9fca971d93 More graceful handle of flush cancellation (#7798) 
* Reapply "return an error when cancelling a flush (#7781)" (#7799)

This reverts commit 4a1103acb7.

* log::trace cancellation of components

* gracefully handle flush cancellation inside segment flusher

* Update lib/segment/src/segment/entry.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* don't wrap cancellation errors

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
 2026-01-21 10:06:50 -03:00
Arnaud Gourlay
d6afde3d5e Remove unecessary flag in OperationData (#7957) 2026-01-21 13:05:41 +01:00
Roman Titov
7bc947cef3 Always Copy-on-Write when updating payload in immutable segments (#7952) 2026-01-21 11:09:17 +01:00
Andrey Vasnetsov
25252aba37 Generate *.pyi files for EDGE (#7943) 
* generate pui files for edge functions

* fix stup for `lambda_`
 2026-01-20 18:28:40 +01:00
Luis Cossío
3cc1a8b4dc move cfg statements (#7954) 2026-01-20 14:14:23 -03:00
Tim Visée
6857e7d42e Fix integer overflow in query batch when using high limits (#7950) 2026-01-20 15:10:48 +01:00
Kumar Shivendu
9d71b021ab Expose segment UUIDs in telemetry response (#7735) 
* Add segment ID in telemetry response

* Migrate optimization log segment IDS to use UUIDs

* fmt

* clippy

* Bring back proxy segment SegmentEntry impl

* update openapi spec

* Bring back segment ID

* update comment

* Precompute segment uuid from filesyste while creating segment

* Update return types

* Update OpenAPI spec

* remove segment_ prefix from uuid field and use inline block

* Update OpenAPI spec

* Trigger CI

* clippy

* Improve comments to differentiate between segment id and uuid

* Remove duplicate function that shadowed segment entry function

---------

Co-authored-by: timvisee <tim@visee.me>
 2026-01-20 13:59:22 +01:00
Andrey Vasnetsov
438a9c5d87 rollback experimental fix (#7948) 2026-01-20 12:32:30 +01:00
dependabot[bot]
7cd33e5288 build(deps): bump url from 2.5.7 to 2.5.8 (#7936) 
Bumps [url](https://github.com/servo/rust-url) from 2.5.7 to 2.5.8.
- [Release notes](https://github.com/servo/rust-url/releases)
- [Commits](https://github.com/servo/rust-url/compare/v2.5.7...v2.5.8)

---
updated-dependencies:
- dependency-name: url
  dependency-version: 2.5.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-20 11:29:07 +01:00
dependabot[bot]
4370fbee22 build(deps): bump bitpacking from 0.9.2 to 0.9.3 (#7935) 
Bumps [bitpacking](https://github.com/quickwit-oss/bitpacking) from 0.9.2 to 0.9.3.
- [Release notes](https://github.com/quickwit-oss/bitpacking/releases)
- [Changelog](https://github.com/quickwit-oss/bitpacking/blob/master/CHANGELOG.md)
- [Commits](https://github.com/quickwit-oss/bitpacking/commits)

---
updated-dependencies:
- dependency-name: bitpacking
  dependency-version: 0.9.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-20 11:29:03 +01:00
dependabot[bot]
b8a4ba16c3 build(deps): bump object_store from 0.13.0 to 0.13.1 (#7934) 
Bumps [object_store](https://github.com/apache/arrow-rs-object-store) from 0.13.0 to 0.13.1.
- [Changelog](https://github.com/apache/arrow-rs-object-store/blob/main/CHANGELOG-old.md)
- [Commits](https://github.com/apache/arrow-rs-object-store/compare/v0.13.0...v0.13.1)

---
updated-dependencies:
- dependency-name: object_store
  dependency-version: 0.13.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-20 11:27:29 +01:00
xzfc
e5dd8488f1 GET /optimizations: re-plan on each request (#7945) 2026-01-20 11:07:35 +01:00
Andrey Vasnetsov
21deeec9ec async scroll (#7928) 
* implementation of async batch vectors reading

* EXPERIMENT: async-io for reading on scroll

* disable on non-linux

* make retrieve sequential for test

* wip: implement vector reading via callback

* simplify operations, remove duplicates

* use batch retrieve also for post-processing search results

* clippy

* fix tests

* review fixes

* Replace big match statement with simple option filter and equal check

* Inline format arguments

---------

Co-authored-by: timvisee <tim@visee.me>
 2026-01-19 18:43:40 +01:00
Andrey Vasnetsov
d9393acac6 simplify edge config (#7933) 
* simplify edge config

* Accept default vector config or explicit vector config map when creating `EdgeConfig`

* fixup! Accept default vector config or explicit vector config map when creating `EdgeConfig`

Update examples

* upd tests

* bump edge version to v0.5.0

---------

Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
 2026-01-19 18:32:11 +01:00
Andrey Vasnetsov
a2550eb938 do not generate new ids if not needed (#7932) 2026-01-19 16:12:33 +01:00
Luis Cossío
b4c9df1f3f [cluster telemetry] Include shards information (#7886) 
* make some existing fields optional

particularly, RemoteShardTelemetry's `searches` and `updates`, and
LocalShardTelemetry's `optimizations`

* Include shards information in cluster telemetry

* upd openapi

* nits

* only include partial snapshot info if non-empty

* fixes
 2026-01-19 11:34:37 -03:00
Arnaud Gourlay
7739b2c4b6 Optimize SegmentBuilder update with proper Vec sizing (#7921) 2026-01-19 15:24:04 +01:00
Andrey Vasnetsov
b94a13fd6c implement a new configuration for read fan-out delay (#7929) 
* implement a new configuration for read fan-out delay

* upd schema

* Update read_fan_out_delay_ms comment

---------

Co-authored-by: timvisee <tim@visee.me>
 2026-01-19 14:38:54 +01:00
Arnaud Gourlay
c548566e56 Optimize allocation in max_available_vectors_size_in_bytes (#7924) 2026-01-19 12:43:04 +01:00
Roman Titov
a9865fbed2 Update qdrant-edge-py version to 0.4.0 (#7926) 
* Update `qdrant-edge-py` version to 0.4.0

* fixup! Update `qdrant-edge-py` version to 0.4.0

🙄🙄🙄
 2026-01-16 23:27:03 +01:00
Roman Titov
1c295ebfe2 Rename edge::Shard into edge::EdgeShard (#7925) 2026-01-16 17:22:23 +01:00
Andrey Vasnetsov
28215e6125 Validate vector name in matrix request (#7923) 
* validate `using` param

* fix slop
 2026-01-16 12:20:40 +01:00
xzfc
76944fbb6e Expose planned optimizations in API (#7784) 
* Expose pending optimizations in API

* Address review comments
 2026-01-15 21:31:31 +00:00
Roman Titov
e733e4204c Flush edge::Shard on Drop (#7911) 
And add explicit `flush` and `close` methods
 2026-01-15 10:55:33 +01:00
Andrey Vasnetsov
921b1ce83d Prevent unoptimized updates (#7643) 
* wip: wait for optimization before applying update

* implement api parameter

* nits

* fix deadloack when no optimizers are running

* release handle mutex

* Derive PartialEq

* Remove unused function

* fix missing kb conversion in threshold

* Update lib/api/src/grpc/proto/collections.proto

Co-authored-by: Tim Visée <tim+github@visee.me>

* sync comment change

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
 2026-01-14 16:37:03 +01:00
Ivan Pleshkov
ec3ef1cb67 Use p square to find ranges (#7733) 
* use p square to find ranges

* use sample size

* add stopper checks

* review remarks

* review remarks
 2026-01-14 08:51:08 +01:00
xzfc
a4ca865f33 OptimizationPlanner (#7752) 
* refactor(SegmentHolder::iter): use SegmentId, not &SegmentId

* refactor(SegmentHolder): Add SegmentHolder::iter_original

* refactor: inline *Optimizer::worst_segment() into SegmentOptimizer::check_condition

* refactor(ConfigMismatchOptimizer): extract config_mismatch function

* Fix comments

* refactor: consistent variable naming

It's `segment_id`/`_segment_id`, not `idx`/`sid`/`x.0`/`_`.

* refactor(IndexingOptimizer): extract large chunks of logic

* refactor(SegmentOptimizer): simplify unwrap_proxy/handle_cancellation

* SegmentHolder: add running_optimizations counter

* OptimizationPlanner

* MergeOptimizer: allow merging two segments, update doc
 2026-01-13 12:09:40 +00:00
dependabot[bot]
ce6f6fbd42 build(deps): bump flate2 from 1.1.5 to 1.1.8 (#7905) 
Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.1.5 to 1.1.8.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.1.5...1.1.8)

---
updated-dependencies:
- dependency-name: flate2
  dependency-version: 1.1.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-13 11:38:45 +01:00
Andrey Vasnetsov
dc3dfcbbe0 bump edge version to 0.2.0 2026-01-13 09:41:47 +01:00
dependabot[bot]
3521cd5b9a build(deps): bump ndarray from 0.17.1 to 0.17.2 (#7906) 
Bumps [ndarray](https://github.com/rust-ndarray/ndarray) from 0.17.1 to 0.17.2.
- [Release notes](https://github.com/rust-ndarray/ndarray/releases)
- [Changelog](https://github.com/rust-ndarray/ndarray/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-ndarray/ndarray/compare/0.17.1...0.17.2)

---
updated-dependencies:
- dependency-name: ndarray
  dependency-version: 0.17.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-01-13 09:37:04 +01:00
Roman Titov
9940a4b272 Implement info request for Qdrant Edge (#7890) 
* Implement `edge::Shard::info`

* Add `ReprStr` marker-trait

* Add Python bindings for payload index types

* Implement `PyShard::info`

* fixup! Add Python bindings for payload index types

Add `enable_hnsw` fields

* add __repr__ and exted tests

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2026-01-13 09:30:29 +01:00
Andrey Vasnetsov
a2e31318e0 Add pyo3 signature annotations with default values for all Py* structs (#7891) 
* Add pyo3 signature annotations with default values for all Py* structs

- Added #[pyo3(signature = (...))] attributes to all Py* structs with optional parameters
- Ensured non-default parameters come before parameters with defaults
- Follows the pattern established in PyScrollRequest
- Updated structs: PySearchRequest, PySearchParams, PyQuantizationSearchParams,
  PyAcornSearchParams, PyQueryRequest, PyPrefetch, PyOrderBy, PyMmr, PyPoint,
  PyFormula, PyShard::load, PySparseVectorDataConfig, PySparseIndexConfig

* fix params ordering
 2026-01-12 22:55:17 +01:00
Andrey Vasnetsov
aa8740ff23 disposable hygienics (#7893) 
* no not overuse HwMeasurementAcc::disposable everywhere, for better code navigation

* fmt
 2026-01-12 22:54:25 +01:00
tellet-q
34daa19ed5 Move TestDelay to CollectionUpdateOperations and add generic staging API (#7716) 
* Move TestDelay to CollectionUpdateOperations

* Address ai review

* Modify endpoint's path

* Address review
 2026-01-12 11:40:22 +01:00
Andrey Vasnetsov
a7cdfd27d5 log mutable id tracker mapping updates (#7894) 
* add log for persisting id-tracker changes

* trace log flush topology

* fmt

* make trace
 2026-01-11 23:58:45 +01:00
Roman Titov
fb9d36e913 Implement scroll and count requests for Qdrant Edge (#7880) 
* Cleanup `shard` crate module declarations

* Move `ScrollRequestInternal` into `shard` crate

* fixup! Move `ScrollRequestInternal` into `shard` crate

Fix imports

* fixup! Move `ScrollRequestInternal` into `shard` crate

`const fn default_*`

* Implement `edge::Shard::scroll`

* fixup! Implement `edge::Shard::scroll`

Re-export `OrderByInterface`

* Cleanup `edge` module declarations

* Cleanup `qdrant-edge-py` module declarations

* Move `PyWithPayload` and `PyWithVector` into `types::query`

* Add `PyScrollRequest` type

* Implement `PyShard::scroll`

* Move `CountRequestInternal` into `shard` crate

* fixup! Move `CountRequestInternal` into `shard` crate

Fix imports

* fixup! Move `CountRequestInternal` into `shard` crate

Rename `default_exact_count` into `CountRequestInternal::default_exact`

* Implement `edge::Shard::count`

* Implement `PyShard::count`

* review: offset for scroll, default values, examples

* ai review

---------

Co-authored-by: generall <andrey@vasnetsov.com>
 2026-01-10 12:20:30 +01:00
Arnaud Gourlay
6966a84168 Improve logs regarding cleanup on startup (#7885) 2026-01-09 12:04:15 +01:00
Sapphire
a494452524 Add enable_hnsw option for payload field schema (#7887) 
* feat: Add enable_hnsw option for payload field indexes

Add optional enable_hnsw parameter to all payload index types to control
whether additional HNSW graph links are built for each indexed field.

- Add enable_hnsw field to all 8 payload index param types
- Update gRPC proto definitions and conversions
- Update OpenAPI schema
- Modify HNSW graph builder to respect enable_hnsw flag
- Add enable_hnsw() helper methods to PayloadSchemaParams and PayloadFieldSchema
- Update all tests to include new field (default: None)

When enable_hnsw is true and payload_M > 0, additional HNSW links will
be built for the payload field. Default value is true for backward compatibility.

* Fix Some format problems

* fix: address comment problem

---------

Co-authored-by: EC2 Default User <ec2-user@ip-10-78-171-148.ec2.internal>
 2026-01-09 11:39:44 +01:00
Ivan Pleshkov
ec4b98bcc8 id tracker persisted mappings offset (#7877) 
* id tracker persisted mappings offset

* remove obsolete check

* review remarks

* review remarks

* When persisting mappings, truncate file that is larger than we expect

* If persisting mappings fails, truncate file to what we had before

This isn't necessary, but it is nice to clean up partial mappings.

* Fix typos

* Fix typos

* Just truncate the file, we don't have to seek anymore

* Rename length variable to mappings_expected_len

---------

Co-authored-by: timvisee <tim@visee.me>
 2026-01-09 11:38:20 +01:00
xzfc
38e620ffd7 Put unreliable unsafe block behind "testing" feature flag (#7884) 2026-01-09 10:41:30 +01:00
Daniel Boros
1cbf297f9e feat/shard-ops-timeout (#7750) 
* feat: add shard-ops timeout

* feat: add shard-ops timeout

* feat: add missing timeout

* fix: format

* fix: type serialization

* fix: bad wal flush logic

* feat: add seperate wait_time

* fix: linter issues

* fix: linter

* chore: make select more readable

* fix: linter issues

* chore: remove comment

* chore: adjust comment

* chore: update definition

* chore: remove extra wait_timeout -> make timeout global

* chore: remove wait_timeout from point_ops

* chore: remove wait_timeout from grpc

* refactor: make shard_ops more readable

* chore: remove timeout from update_all

* feat: propagate wait_timeout error

* fix: clippy recommendation

* chore: update wait timeout

* refactor: update merge_successful_update_results

* fix: tests inconsistency

* fix: revert global ord, partialord for UpdateStatus

* fix: integration test

* fix: res ingoring

* fix: review requests

* fix: clippy

* chore: revert back futures unordered

* fix: resolve response status

* force timeout for remote operations

* fix: pr reviews

* fix: switch to duration

* Bump OpenAPI spec

* feat: add serde_as

* fix: timeout statu condition

* Minor tweaks

* Use tokio timeout directly rather than deadline

* Revert schema generator permissions

* Remove obsolete semi-colon

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: timvisee <tim@visee.me>
 2026-01-09 10:24:45 +01:00
Luis Cossío
7dbb994f6f [cluster telemetry] Include peers versions (#7848) 
* wrap details in another struct, less Options

* include version in peers info

* update openapi

* include app telemetry in internal service

* move debug assertion after check
 2026-01-08 14:35:31 -03:00
xzfc
65141c9a2c Use syncfs (#7883) 
* Remove call to Archive::set_sync

Also, this method was the last remaining part of our `tar-rs` fork,
so we can switch to the upstream version now.

* Do syncfs
 2026-01-08 15:07:52 +00:00