Commit Graph

475 Commits

Author SHA1 Message Date
Kumar Shivendu
c0f580bddd Shard key initial state (#7371)
* Allow specifying initial state while creating shard key

* Dont wait for Active state if initial_state is set

* Generate gRPC docs and OpenAPI spec

* Apply suggestions

* Skip from scheamrs not serde

* Add integration test

* minor suggestions
2025-10-13 17:58:34 +05:30
Tim Visée
b9243c5af4 Forbid peer to join cluster with URI that is already used (#7375)
* Disallow peer to join with URI that is already used

* Add test for rejecting peer join with duplicate URI

* Improve peer rejection logic

* Try to rejoin twice, we expect a consistent result
2025-10-13 12:32:51 +02:00
xzfc
95b4bbf978 Rename HnswConfig::{copy_vectors -> inline_storage} (#7389) 2025-10-13 10:08:06 +00:00
Andrey Vasnetsov
34596530d1 remove default explicit config for strict mode (#7369)
* remove default explicit config for strict mode

* Don't prefix commented out properties with space to remain consistent

* Don't crash test if strict mode config is not set at all

---------

Co-authored-by: timvisee <tim@visee.me>
2025-10-09 12:00:01 +02:00
Arnaud Gourlay
a24d380ee0 Investigate flaky snapshot transfer test (#7339) 2025-10-09 10:47:23 +02:00
Arnaud Gourlay
fe94fdc5ce Bump Python 3.13 on CI (#7365)
* Bump Python 3.13 on CI

* Regenerate tls certificates

---------

Co-authored-by: tellet-q <elena.dubrovina@qdrant.com>
2025-10-08 12:50:34 +02:00
tellet-q
c3ca182423 Refactor bash tests: storage-compatibility (#7299)
* Add e2e test for compatibility
2025-10-08 08:33:55 +02:00
Andrey Vasnetsov
8dd915c0b1 non appendable proxies (#7345)
* WIP: make proxy segment non-appendable

* make deleted points and index changes private to Proxy Segment

* consistently remove empty unused segment from holder on unproxifying and the end of optimization

* remove/fix obsolete test

* Fix comment, some general adjustments

* Also don't remove non-original segments

* Use ahash for map of proxy index changes

* Inline format arguments

* Simplify has point branching

* Assert that proxy flushes to latest wrapped segment version

* make vector deletetion a CoW operation

* relax debug assertions

* Update lib/shard/src/proxy_segment/mod.rs

* fix test_partial_snapshot_empty

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
2025-10-08 00:19:41 +02:00
Tim Visée
a1efafbbf3 Only wait for green collection status in empty snapshot test (#7360) 2025-10-07 18:12:59 +02:00
Tim Visée
3856ff6bdb Before doing actual partial snapshot tests, wait for green status (#7358) 2025-10-07 15:57:46 +02:00
xzfc
780d406ff8 Add CollectionInfo::warnings field (#7293)
* refactor: swap DiffConfig parmeters

was: diff.update(&config)
now: config.update(diff)

* refactor: introduce DiffConfig::update_opt

* feat: add CollectionInfo::configuration_status field

* rename ConfigurationStatus to CollectionWarning
2025-09-29 18:17:03 +00:00
Arnaud Gourlay
f5d824508e Regen storage compat. v1.15.4 (#7321)
* Regen storage compat. 1.14 & 1.15

* bump prev patch to v1.15.4

* skip v1.14.1

* cleaner

* sleepy
2025-09-29 16:37:16 +02:00
Arnaud Gourlay
04d8d05aa5 Fix deadlock on Actix RT in Collection Info (#7277) 2025-09-22 11:14:27 +02:00
Arnaud Gourlay
8a31160297 Always run Telemetry on a blocking thread (#7269)
* Always run Telemetry on a blocking thread

* run Telemetry on general runtime instead of the blocking Actix one

* Polish test
2025-09-19 10:28:09 +02:00
Arnaud Gourlay
f66dc31452 Fix deadlock on Actix RT in Cluster Info (#7267) 2025-09-17 15:54:11 +02:00
Arnaud Gourlay
5238d16909 Fix deadlock on Actix RT in Telemetry (#7265)
* Deadlock on Actix RT

* Spawn blocking task for sync lock
2025-09-17 10:57:00 +02:00
Andrey Vasnetsov
bff9cb1a50 fix strict mode validation for nested filters (#7264)
* fix strict mode validation for nested filters

* fix test
2025-09-16 12:03:54 +02:00
Arnaud Gourlay
e79100a2f2 Fix phrase match to not match with unknown tokens (#7252)
* Fix phrase match to not match with unknown tokens

* add tests

* spelling
2025-09-16 11:16:17 +02:00
Arnaud Gourlay
bc13f01680 [strict-mode] Add max number of payload index count (#7222)
* [strict-mode] Add max number of payload index count

* improve docs and validation for StrictModeConfig

* fix from coderabbit

* test blocking access to payload indexes

* polish test
2025-09-15 13:12:40 +02:00
Arnaud Gourlay
04d1c194e6 Fix RecommendExample conversion to support new format (#7234) 2025-09-11 17:36:29 +02:00
Roman Titov
a3ce716590 Fix deadlock during streaming shard snapshot (#7241)
* Add `test_shard_snapshot_deadlock` integration test

* Fix deadlock during streaming shard snapshot
2025-09-11 14:55:03 +02:00
Arnaud Gourlay
f30e7f7273 Fix missing REST API validation for point batch update (#7219)
* Fix missing REST API validation for point batch update

* use a valid vector
2025-09-05 18:25:37 +02:00
tellet-q
23a20261cf Add e2e test for snapshots (#7066)
* Add e2e test that mimics 1 crasher's scenario
2025-09-05 09:14:59 +02:00
Tim Visée
d78290b08c build(deps): bump h2 from 4.2.0 to 4.3.0 in /tests (#7167)
Bumps [h2](https://github.com/python-hyper/h2) from 4.2.0 to 4.3.0.
- [Changelog](https://github.com/python-hyper/h2/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/python-hyper/h2/compare/v4.2.0...v4.3.0)

---
updated-dependencies:
- dependency-name: h2
  dependency-version: 4.3.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-28 15:36:21 +02:00
tellet-q
5936926c88 Refactor bash test for snapshots recovery (#7008)
* Refactor bash test for snapshots recovery

* Address review: minio check

* Address review
2025-08-27 15:25:27 +02:00
tellet-q
6c75566ac4 Add tests for immutable text index (#7153)
* Add unit test

* Flush text index deletes before reopening them

* Fix unused warning if compiling without RocksDB

* Add e2e test

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-27 12:18:58 +02:00
Andrey Vasnetsov
e744bd6014 implement custom collection metadata (#7123)
* implement custom collection metadata

* persist metadata change

* Also unset a key in the test

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-26 11:49:17 +02:00
Andrey Vasnetsov
bf60e96619 filter text any (#7100)
* internal implementation for filter any

* implement api for match `text_any`

* allow naive implementation of text match any

* dedup after kmerge + renames

* improve `expected_should_estimation`

* congruence test

* openapi test

* setup index in each test

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-08-24 12:09:05 +02:00
Andrey Vasnetsov
c188f5b7c4 Update if (#7006)
* implement condition parameter for upsert operation

* fmt

* update api schema

* implement conditional update for update-vectors op

* rename to update_filter for consistency

* add tests

* explicilty ignore strict mode for conditional updates

* rabbit review fix

* Also assert vector element before, confirm we don't normalize

* Simplify filter creation using new helper

* Rename merge_with_ids to with_point_ids

---------

Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
2025-08-15 15:47:43 +02:00
Jojii
290f9e2675 Always load InferenceService (#7050)
* Always load InferenceService

* Add BM25 openapi test

* Add warning to development.yaml config (#7051)
2025-08-13 16:59:42 +02:00
tellet-q
f0612d6680 Refactor bash tests: tls (#6995)
* Refactor tls bash test

* Update codespell

* Address review
2025-08-11 10:05:47 +02:00
Roman Titov
a08c89082c Track payload index schema version (#6819) 2025-08-06 16:35:43 +02:00
Roman Titov
563194febf Fix test_create_shard_key_read_availability flakiness (#6902) 2025-08-06 16:14:13 +02:00
Kumar Shivendu
a4220fdd84 Partial snapshots update recovery ts (#6980)
* Update recovery ts despite empty partial snapshots

* fmt

* apply suggestions with some improvements
2025-08-06 10:48:03 +02:00
Arnaud Gourlay
33d15fd452 Bump Python integration test deps for Dependabot (#6979) 2025-08-05 18:05:36 +02:00
tellet-q
f87e505439 Refactor bash tests: low-ram (#6911)
* Refactor low-ran bash test

* Address review
2025-08-05 13:05:39 +02:00
Roman Titov
e8625ba903 Avoid creating partial snapshot if snapshot manifest contains no changes (#6924) 2025-07-29 14:18:57 +02:00
Roman Titov
22e15b9f40 Update collection-level payload index schema after recovering partial snapshot (#6886) 2025-07-29 12:25:24 +02:00
Andrey Vasnetsov
2c8c0a91fd wait-for instead of validate (#6899) 2025-07-24 08:37:23 +02:00
Tim Visée
6f4069e711 Preprocess query vectors, sort sparse vectors (#6895)
* Add test to assert behavior of unsorted sparse query

* Preprocess sparse vectors

* Fix typo
2025-07-17 16:11:44 +02:00
Tim Visée
cd1a31448d Fix broken is_empty filter on new points (#6882)
* Add test to catch bug

Bug: <https://github.com/qdrant/qdrant/issues/6880>

* Fix loading null index from wrong path

* Simplify test

* Also bump total point count when removing a point

* Add debug assertion, ensure we don't create null index in selector

* Update bug link

* Remove unused import

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-07-16 14:25:09 +02:00
tellet-q
f84e1f1f4a Refactor bash tests: low-disk (#6845)
* Add e2e_tests pytest test suite

* Remove old bash test
2025-07-14 12:42:06 +02:00
Luis Cossío
6307510476 [MMR] expose as query variant in rest and grpc (#6797)
* expose mmr in rest and grpc

* fix referenced vectors in mmr queries

* rename to MmrInput in grpc for consistency

* mmr is now an optional parameter of nearest query

* fix score_threshold at local shard

* fix conversion of rest -> collection

* fix and refactor planned query

* nits

* rename `lambda` to `diversity` in interface

* handle score_threshold within mmr calculation

* add openapi test

* fix candidate limit at collection level

* candidate_limit -> candidates_limit

* finish rename to candidates_limit

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-11 10:39:12 -04:00
Andrey Vasnetsov
f47e499825 fix check for pure primary condition filter (#6840)
* fix check for pure primary condition filter

* avoid cloning has_id for primary condition

* fix test

* add test

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2025-07-10 11:40:11 +02:00
Roman Titov
68a653649e Wait until replicas are activated, when creating custom shards (#6817)
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: timvisee <tim@visee.me>
2025-07-08 12:18:53 +02:00
Roman Titov
3ff0a34821 Create custom shards in Initializing state instead of Active (#6778) 2025-07-02 17:53:42 +02:00
Arnaud Gourlay
5c54f437ab Deleted leftover test openapi collections (#6788) 2025-07-02 10:50:23 +02:00
Luis Cossío
6919e3ae5b [phrase matching] expose phrase condition (#6670)
* add `"match": { "phrase": ... }` condition

* gen grpc and openapi

* [phrase matching] expose `phrase_matching` flag in rest and grpc (#6620)

* expose setting in rest and grpc

* phrase matching openapi test

* regen openapi

* [phrase matching] Text index fixes (#6730)

* allow rocksdb-based immutable text index

* fix repeated-token phrases

* fmt

* Update OpenAPI spec

---------

Co-authored-by: timvisee <tim@visee.me>

* add repeated word case in openapi test

* [phrase match | strict mode] Allow phrase condition when enabled in index (#6749)

* allow phrase filter when index is present

* prettier error message

* clippppppy

---------

Co-authored-by: timvisee <tim@visee.me>
2025-06-24 16:56:36 -04:00
Tim Visée
f79ad06ae9 build(deps): bump urllib3 from 2.4.0 to 2.5.0 in /tests (#6721)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.4.0...2.5.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.5.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-19 10:07:19 +02:00
Roman Titov
a87a4e0d79 Track version of mutable files in vector and payload storage (#6652) 2025-06-17 19:00:11 +02:00