Commit Graph
445 Commits
Author SHA1 Message Date
Roman Titov 31738d3b85 Track payload index schema version (#6819) 2025-08-11 13:17:19 +02:00
Roman Titov 2938660e92 Fix test_create_shard_key_read_availability flakiness (#6902) 2025-08-11 13:17:11 +02:00
Kumar Shivendu 8be66adfc3 Partial snapshots update recovery ts (#6980)
* Update recovery ts despite empty partial snapshots

* fmt

* apply suggestions with some improvements
2025-08-11 13:16:49 +02:00
Arnaud Gourlay 416d82a554 Bump Python integration test deps for Dependabot (#6979) 2025-08-11 13:16:39 +02:00
tellet-q cad14965e3 Refactor bash tests: low-ram (#6911)
* Refactor low-ran bash test

* Address review
2025-08-11 13:16:20 +02:00
Roman Titov bf319a0b40 Avoid creating partial snapshot if snapshot manifest contains no changes (#6924) 2025-08-11 13:15:39 +02:00
Roman Titov e6bdcb2520 Update collection-level payload index schema after recovering partial snapshot (#6886) 2025-08-11 13:15:23 +02:00
Andrey Vasnetsov b279aefde6 wait-for instead of validate (#6899) 2025-07-24 08:46:44 +02:00
Tim Visée 60c216be92 Preprocess query vectors, sort sparse vectors (#6895)
* Add test to assert behavior of unsorted sparse query

* Preprocess sparse vectors

* Fix typo
2025-07-17 16:13:01 +02:00
Tim Viséeandcoderabbitai[bot] 877adc6748 Fix broken is_empty filter on new points (#6882)
* Add test to catch bug

Bug: <https://github.com/qdrant/qdrant/issues/6880>

* Fix loading null index from wrong path

* Simplify test

* Also bump total point count when removing a point

* Add debug assertion, ensure we don't create null index in selector

* Update bug link

* Remove unused import

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-07-17 14:01:38 +02:00
tellet-q cf9862949c Refactor bash tests: low-disk (#6845)
* Add e2e_tests pytest test suite

* Remove old bash test
2025-07-17 13:57:56 +02:00
Luis Cossíoandgenerall e8daf9c79f [MMR] expose as query variant in rest and grpc (#6797)
* expose mmr in rest and grpc

* fix referenced vectors in mmr queries

* rename to MmrInput in grpc for consistency

* mmr is now an optional parameter of nearest query

* fix score_threshold at local shard

* fix conversion of rest -> collection

* fix and refactor planned query

* nits

* rename `lambda` to `diversity` in interface

* handle score_threshold within mmr calculation

* add openapi test

* fix candidate limit at collection level

* candidate_limit -> candidates_limit

* finish rename to candidates_limit

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-07-17 13:56:46 +02:00
Andrey VasnetsovandArnaud Gourlay 6bc7f9c650 fix check for pure primary condition filter (#6840)
* fix check for pure primary condition filter

* avoid cloning has_id for primary condition

* fix test

* add test

---------

Co-authored-by: Arnaud Gourlay <arnaud.gourlay@gmail.com>
2025-07-17 13:54:43 +02:00
eceefa780e Wait until replicas are activated, when creating custom shards (#6817)
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: timvisee <tim@visee.me>
2025-07-17 13:49:48 +02:00
Roman Titov bda74ceae2 Create custom shards in Initializing state instead of Active (#6778) 2025-07-17 13:39:59 +02:00
Arnaud Gourlay afe4daa38d Deleted leftover test openapi collections (#6788) 2025-07-17 13:38:30 +02:00
Luis Cossíoandtimvisee ad936a463c [phrase matching] expose phrase condition (#6670)
* add `"match": { "phrase": ... }` condition

* gen grpc and openapi

* [phrase matching] expose `phrase_matching` flag in rest and grpc (#6620)

* expose setting in rest and grpc

* phrase matching openapi test

* regen openapi

* [phrase matching] Text index fixes (#6730)

* allow rocksdb-based immutable text index

* fix repeated-token phrases

* fmt

* Update OpenAPI spec

---------

Co-authored-by: timvisee <tim@visee.me>

* add repeated word case in openapi test

* [phrase match | strict mode] Allow phrase condition when enabled in index (#6749)

* allow phrase filter when index is present

* prettier error message

* clippppppy

---------

Co-authored-by: timvisee <tim@visee.me>
2025-07-17 13:25:12 +02:00
Tim Viséeanddependabot[bot] 125d9b5f40 build(deps): bump urllib3 from 2.4.0 to 2.5.0 in /tests (#6721)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.4.0...2.5.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.5.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-17 13:14:54 +02:00
Roman Titov 082f101988 Track version of mutable files in vector and payload storage (#6652) 2025-07-17 13:14:45 +02:00
Roman Titov 7497d01bf1 Tweak partial snapshot recover and recover_from response (#6610) 2025-07-17 13:07:13 +02:00
Tim Viséeanddependabot[bot] d6abef04ed build(deps): bump requests from 2.32.3 to 2.32.4 in /tests (#6662)
Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-17 13:05:40 +02:00
Roman Titov 9dd3b6ff9e Simpler partial snapshots implementation based on immutable_files (#6599) 2025-07-17 13:04:15 +02:00
Roman Titovandgenerall 2aafd0174d Add partial snapshot create requests count and recovery timestamp to telemetry (#6545)
Co-authored-by: generall <andrey@vasnetsov.com>
2025-05-22 23:27:14 +02:00
Roman Titovandgenerall 86d7c02528 Add partial snapshot recovery and search locks (#6497)
Co-authored-by: generall <andrey@vasnetsov.com>
2025-05-22 23:27:01 +02:00
Arnaud Gourlayandgenerall dd97253fe9 Strict mode allows fullscan for multitenant payload index (#6498)
* Strict mode allows fullscan for multitenant payload index

* different error for missing payload index in multitenant case

* handle payload_m

* add simple test

* handle hnsw.m not set

* add another simple test

* fix test

* fallback to global HSNW config

* new error status code

* do not block on global HNSW and improver error reporting

* clearer error reporting

* review fixes

* fix test error messages

---------

Co-authored-by: generall <andrey@vasnetsov.com>
2025-05-22 23:26:16 +02:00
Kumar Shivendu 5e7d9aa4fc Fix force delete peer flaky test (#6568)
* Fix force delete peer flaky test

* Temporarily only run force delete peer test

* Run till failure

* Add mutable transfer field

* Introduce wait_with_return and use it to solve TOCTOU

* Revert "Run till failure"

This reverts commit 53319a19c8.

* Revert "Temporarily only run force delete peer test"

This reverts commit 5ea47e30a6.

* Simplify the test instead of introducing wait_with_return

* revert changes to utils
2025-05-22 23:25:50 +02:00
Arnaud GourlayandLuis Cossío bbc4beb9ec Fix UnindexedField infra to handle lookup&range index requirements (#6496)
* Fix UnindexedField infra to handle lookup&range index requirements

* more generic approach

* better test

* restore

* small unit test for sanity

* review nits

* Do not recommend parametrized index

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
2025-05-22 23:24:10 +02:00
Andrey Vasnetsov 7872c182f1 Strict mode error is now 400 instead of 403 (#6560)
* strcit mode error is now 400 instead of 403

* fix test
2025-05-22 23:22:16 +02:00
Tim Visée b81c87d0e0 Wait when putting collection metadata (#6559) 2025-05-22 23:22:08 +02:00
Tim Visée 2c7f54584e Don't wait on shard transfer progress, just wait on transfer count (#6550) 2025-05-22 23:18:48 +02:00
Arnaud Gourlay 9b99408529 Always overwrite existing payload on upsert (#6533)
* Always overwrite existing payload on upsert

* Add REST tests
2025-05-22 23:17:18 +02:00
Kumar Shivendu 72fd3b98c3 Abort transfer on force peer delete and add test for bug (#6505)
* Test for reproducing force delete peer bug

* drop seed port

* use larger transfers to ensure killing before transfer is completed

* simplify test cases

* kill and wait for same commit before force delete

* Don't kill to avoid breaking consesnsus before force delete peer

* Abort transfer on force peer delete (#6507)

* Abort transfer on force peer delete

* ToC changes to abort internally

* Abort shard transfer using consensus not just locally

* fmt

* fix fmt

* Abort transfers internally when shard is being removed

* remove unused imports
2025-05-22 23:15:34 +02:00
n0x29aandjojii 5393fc8a6b Refactor: move HardwareUsage to the Usage map (#6532)
---------

Co-authored-by: jojii <jojii@gmx.net>
2025-05-22 23:15:15 +02:00
Roman Titov 168ba380e2 Add recover_partial_snapshot_from endpoint (#6468) 2025-05-22 23:13:35 +02:00
Kumar Shivendu 27d54c8d4d Add a test for recovery mode (#6389)
* Add a test for recovery mode

* use fewer points for faster test

* fix failing test

* Sleep to ensure that qdrant does trigger a transfer
2025-05-22 23:03:40 +02:00
Arnaud Gourlay eaaa4e4ee1 Validate IntegerIndexParams (#6494)
* Validate IntegerIndexParams

* focus
2025-05-22 23:00:52 +02:00
Arnaud Gourlay 00e7ffa9b3 Strict mode detects full scan on query (#6473) 2025-05-22 22:59:16 +02:00
Arnaud Gourlay 1da478c897 Fix strict mode unindexed group_by path (#6363)
* Fix strict mode unindexed group_by path

* check index schema for matching support

* Handle disabled lookup on integer index
2025-05-22 22:56:24 +02:00
Kumar Shivendu 04846d3be9 Fix point comparison in test (#6474)
* Fix point comparison in test

* keep using scroll

* drop unused imports
2025-05-22 22:56:11 +02:00
Kumar Shivendu 7d5675690c Fix snapshot test flakiness (#6440)
* Temporarily run fewer tests in CI to reproduce flakiness

* disable other integration test

* log collection topology if request fails

* Run in CI until failure

* update tests

* Catch failures and log

* improve logging for search result comparison flakiness

* fix test

* Simplify test

* Use scroll instead of search and simplify existing code

* Trigger CI

* new util function

* debug again

* print collection topology on mismatch

* wait for deletion of flag after recovery is done

* use wait_for

* Revert integration-tests.yml
2025-05-22 22:53:47 +02:00
Arnaud Gourlay 4c7c730c97 Log context of openapi validation crash (#6448) 2025-05-22 22:52:50 +02:00
Arnaud Gourlay 1e7eed5b69 Regen and update poetry lock (#6435) 2025-05-22 22:51:05 +02:00
Arnaud Gourlay 15d674d0c0 Fix warning regarding missing wait param in openapi tests (#6434) 2025-05-22 22:50:58 +02:00
Arnaud Gourlay 63cb49f055 Debug assert a missing column family or not loaded payload index (#6193)
* assertion on non built index

* Remove old custome storage for gridstore
2025-05-22 22:50:36 +02:00
Arnaud Gourlay 377b614e1e Fix flaky test formula (#6429) 2025-05-22 22:49:44 +02:00
Roman Titov 5483581064 Add partial snapshot API endpoints (#6304) 2025-05-22 22:49:26 +02:00
Arnaud Gourlay cee53ca300 Storage compatibility tests v1.14.0 (#6422) 2025-05-22 22:48:56 +02:00
Luis Cossío a304a51f84 [rate limits] consider length of multivectors for query cost (#6356)
* consider length of multivectors for rate limiter

* add openapi test

* organize imports

* better cost estimation for sparse vectors

* restore groups openapi test

* address @timvisee's review

* Fix consensus test
2025-05-22 22:46:39 +02:00
Kumar Shivendu 0389060955 Introduce coverage reports for integration tests (#6414)
* Introduce coverage reports for integration tests

* Install cargo-llvm-cov

* Use multiline script

* Explicityl setup COVERAGE env var

* fix integration tests and log generated data

* fix ls path

* upload lcov file to GH artifacts

* integration profraw dynamic filename

* Fix llvm profile filename template

* Use interrupt instead of kill and merge consensus test results into same file

* Drop upload artifact stage

* install llvm-cov

* upload as artifact and export coverage files

* try simplifying workflow

* Migrate coverage generation to existing dedicated gh workflow

* trigger on coverage related branches

* Build only if qdrant binary with cov doesnt exist

* Use valid yaml

* include mode in profraw filename

* split coverage workflow into parallel jobs

* add poetry version to env

* log poetry version to install

* clean up integration test workflow

* Simplify comments
2025-05-13 11:16:09 +02:00
dependabot[bot] 7558f67a47 build(deps): bump h11 from 0.14.0 to 0.16.0 in /tests (#6430)
Bumps [h11](https://github.com/python-hyper/h11) from 0.14.0 to 0.16.0.
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

---
updated-dependencies:
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-25 10:30:17 +02:00