upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-08-06 18:10:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,465 Commits 346 Branches 118 Tags
94de0e2d2dcf997bf26bec34dad407d8de9a9abb
Commit Graph

3618 Commits

Author SHA1 Message Date
Arnaud Gourlay
5c6ccf50a5 Fix duplicate ids in upsert_points (#7194) 2025-09-01 15:29:08 +02:00
xzfc
febf72a47e Generic AccessPattern (#7166) 2025-09-01 11:16:20 +00:00
tellet-q
ef6e926826 Add hw usage to Facet response (#7189) 
* Add usage stats to Facet response
 2025-09-01 11:25:17 +02:00
Tim Visée
094fb02e6e Revert "Fix flaky save_on_disk test race condition (#7126)" (#7182) 
* Revert "Fix flaky save_on_disk test race condition (#7126)"

This reverts commit 80f0b53cea.

Related flakiness reports:
- https://github.com/qdrant/qdrant/issues/7156
- https://github.com/qdrant/qdrant/issues/7181

* Fix comment
 2025-09-01 10:51:23 +02:00
Andrey Vasnetsov
2c6cc22325 do not anonymize peer id (#7177) 2025-08-29 20:09:51 +02:00
Ivan Pleshkov
b03853c33e Quantization storage builder for chunked mmap (#7174) 
* Quantization storage builder for chunked mmap

* fix after rebase

* Update lib/segment/src/vector_storage/quantized/quantized_chunked_mmap_storage.rs

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
 2025-08-29 16:10:30 +02:00
Ivan Pleshkov
3be4ca8805 Populate multivector quantization offsets (#7173) 2025-08-29 14:05:48 +02:00
Ivan Pleshkov
4b52affe52 Dont use quantization in exact plain search (#7179) 2025-08-29 14:05:06 +02:00
Tim Visée
bb51da71ea Use locked segment by reference without cloning Arc (#7176) 
* Use locked segment by reference without cloning Arc

* Remove unnecessary clone

* Return list of lock references
 2025-08-29 13:26:51 +02:00
Arnaud Gourlay
71a8d96372 Fix potential deadlock in ProxySegment all_vectors (#7172) 
* Fix potential deadlock in ProxySegment all_vectors

* Add comment

---------

Co-authored-by: timvisee <tim@visee.me>
 2025-08-29 11:06:10 +02:00
Roman Titov
126a3cf558 Move CoreSearchRequest and QueryEnum from collection into shard crate (#7169) 2025-08-29 10:15:58 +02:00
Tim Visée
894692919c Remove chunked mmap vectors mlock flag (#7165) 2025-08-28 15:40:13 +02:00
Roman Titov
6111bd2634 Move segments_updater from collection into shard crate (#7163) 2025-08-28 15:01:54 +02:00
Roman Titov
1327604648 Make update_worker_fn a blocking task, instead of async (#7015) 2025-08-27 17:26:33 +02:00
Roman Titov
d17f54c853 Fixes to UpdateHandler::flush_worker (#7151) 
- rename `wal_flash_job` to `wal_flush_job`
- correctly handle/report WAL flush errors
 2025-08-27 14:16:57 +02:00
tellet-q
6c75566ac4 Add tests for immutable text index (#7153) 
* Add unit test

* Flush text index deletes before reopening them

* Fix unused warning if compiling without RocksDB

* Add e2e test

---------

Co-authored-by: timvisee <tim@visee.me>
 2025-08-27 12:18:58 +02:00
Fabrício Bracht
80f0b53cea Fix flaky save_on_disk test race condition (#7126) 
* fix flaky save_on_disk tests with barrier synchronization

* apply suggestions from coderabbitai bot

* fix timeout in test_wait_for_multiple_notifications

* add TEST_IO_SLACK for disk write latency in tests

* make concurrent readers count explicit in test

* relax timing assertions and use timeout variable for consistency

* refactor wait_for to use deadline-based loop to prevent false timeouts

* fix stale variable name in comment

* relax immediate test timing bound to reduce CI flakes

* add overflow protection for deadline calculation

* apply "checked_sub here and unwrap" suggestion

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* track remaining time instead of deadline to avoid overflow issues

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-27 10:55:25 +02:00
Tim Visée
931b467dde Bump dev version to v1.15.5-dev (#7154) 2025-08-27 10:54:15 +02:00
Roman Titov
6fa24e1ee3 Move SerdeWal from collection into shard crate (#7146) 2025-08-26 16:27:35 +02:00
dependabot[bot]
9b1bbac534 build(deps): bump url from 2.5.4 to 2.5.7 (#7144) 
Bumps [url](https://github.com/servo/rust-url) from 2.5.4 to 2.5.7.
- [Release notes](https://github.com/servo/rust-url/releases)
- [Commits](https://github.com/servo/rust-url/commits)

---
updated-dependencies:
- dependency-name: url
  dependency-version: 2.5.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-26 13:20:57 +02:00
Andrey Vasnetsov
e744bd6014 implement custom collection metadata (#7123) 
* implement custom collection metadata

* persist metadata change

* Also unset a key in the test

---------

Co-authored-by: timvisee <tim@visee.me>
 2025-08-26 11:49:17 +02:00
dependabot[bot]
bef3e88340 build(deps): bump tinyvec from 1.9.0 to 1.10.0 (#7139) 
Bumps [tinyvec](https://github.com/Lokathor/tinyvec) from 1.9.0 to 1.10.0.
- [Changelog](https://github.com/Lokathor/tinyvec/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Lokathor/tinyvec/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: tinyvec
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-26 10:43:09 +02:00
dependabot[bot]
0c937a3801 build(deps): bump io-uring from 0.7.9 to 0.7.10 (#7142) 
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.7.9 to 0.7.10.
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-version: 0.7.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-26 10:34:39 +02:00
dependabot[bot]
8434547690 build(deps): bump charabia from 0.9.6 to 0.9.7 (#7140) 
Bumps [charabia](https://github.com/meilisearch/charabia) from 0.9.6 to 0.9.7.
- [Release notes](https://github.com/meilisearch/charabia/releases)
- [Commits](https://github.com/meilisearch/charabia/compare/v0.9.6...v0.9.7)

---
updated-dependencies:
- dependency-name: charabia
  dependency-version: 0.9.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-26 10:34:29 +02:00
Andrey Vasnetsov
55b0f86122 restore missed index laoding (#7133) 2025-08-26 09:48:52 +02:00
xzfc
937cd477ac Misc fixes (#7134) 
* PrimitiveVectorElement: use zerocopy

* PrimitiveVectorElement: also use zerocopy::FromBytes

* serialize_graph_links: back_index is Vec<PointOffsetType>

* for_each_link_with_vector: loosen lifetimes
 2025-08-26 09:43:37 +02:00
xzfc
0bcdcffb69 Clean up GraphLayers::search_entry_on_level a bit (#6946) 
review remarks

are you happy fmt

don't do unnecessary scoring
 2025-08-25 19:07:35 +02:00
Arnaud Gourlay
0781ea8893 Segment holder has deterministic segment iteration (#7131) 2025-08-25 15:40:21 +02:00
Arnaud Gourlay
1e81d4f3b3 Make points upsert ordering deterministic (#7130) 
* Make points upsert ordering deterministic

* use Iterator magic (thx Tim)
 2025-08-25 14:49:03 +02:00
Arnaud Gourlay
db2489ba9f Test for segment double proxies (#7127) 2025-08-25 14:20:33 +02:00
Arnaud Gourlay
59257681c9 Log optimizer name running (#7129) 2025-08-25 14:20:14 +02:00
Ivan Pleshkov
a2089f6a71 chunked mmap as a quantization storage (#7116) 2025-08-25 12:27:29 +02:00
xzfc
f2ce816fbe Add memory::madvise::will_need_multiple_pages() (#7124) 2025-08-25 09:22:11 +00:00
xzfc
902916f524 Remove unused GraphLinks::on_disk method (#7125) 2025-08-25 11:19:18 +02:00
Andrey Vasnetsov
5235d86435 measure disk size of directories if available (#7120) 
* measure disk size of directories if available

* review suggestions

* Update comment

---------

Co-authored-by: xzfc <xzfcpw@gmail.com>
 2025-08-25 00:17:27 +02:00
Andrey Vasnetsov
bf60e96619 filter text any (#7100) 
* internal implementation for filter any

* implement api for match `text_any`

* allow naive implementation of text match any

* dedup after kmerge + renames

* improve `expected_should_estimation`

* congruence test

* openapi test

* setup index in each test

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
 2025-08-24 12:09:05 +02:00
xzfc
95d7936687 Graph links serializer: avoid large intermediate allocations (#7118) 
* Introduce io::file_operations::atomic_save

* Replace GraphLinksSerializer with serialize_graph_links
 2025-08-22 17:08:35 +00:00
Ivan Pleshkov
879da486af Remove quantization load from trait (#7113) 
* remove quantization load

* refactor load

* fix fmt error

* fix clippy
 2025-08-22 14:37:32 +02:00
Roman Titov
bfdc779203 Move SegmentHolder into shard crate (#7115) 2025-08-22 12:39:06 +02:00
Jojii
4012ea9e51 Mirgate from old shard key format on load (#7110) 
* Migrate shard-key format

* Reduce visibility of was_old_format attribute

* Add debug message when we migrate persisted shard key format

---------

Co-authored-by: timvisee <tim@visee.me>
 2025-08-22 10:47:28 +02:00
Roman Titov
5eb40111a5 Move UpdateTracker from SegmentHolder into UpdateHandler/CollectionUpdater (#7112) 2025-08-21 15:56:14 +02:00
Tim Visée
e02d3dcb38 In proxy segment, don't return deleted point versions (#7109) 
* In proxy segment, don't return deleted point versions

More specifically, don't return a point version from the wrapped segment
if that point is already marked for deletion within the proxy.

* Do get wrapped segment point version if it's newer than soft delete

* Flatten function
 2025-08-21 15:48:18 +02:00
Luis Cossío
a2d3270eb8 Custom RRF k parameter (#7065) 
* allow custom K parameter for RRF

* generate grpc docs and openapi

* use tagged type approach for parametrized fusions

* use params approach in grpc

* simplify api structure

* upd schema

* nits

* rest: parameterized rrf as query variant

* consistency in doc comments

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
 2025-08-20 15:05:27 -04:00
Roman Titov
5d826c583d Move PayloadIndexSchema into shard crate (#7108) 2025-08-20 18:58:03 +02:00
Roman Titov
0bfa8f61d5 Move SaveOnDisk into common crate (#7105) 2025-08-20 17:06:47 +02:00
Tim Visée
1d97e9a415 Keep segment ID mapping when proxying all segments (#7106) 
* When swapping existing segment, remove first to allow segment ID reuse

* Share original segment ID with proxies, keeps original mapping

* Add replace function, that directly replaces existing segment
 2025-08-20 15:18:06 +02:00
Ivan Pleshkov
f3b61bbf4a Retrieve files list from quantization storage (#7095) 
* Get files list from quantization

* fix ci
 2025-08-20 14:15:09 +02:00
Roman Titov
933c7c8ac1 Refactor ShardHolder to use SegmentConfig instead of CollectionParams (#7098) 2025-08-20 14:01:04 +02:00
Roman Titov
8a84c1abb0 Remove async from SegmentHolder::deduplicate_points (#7094) 2025-08-20 13:26:16 +02:00
Ivan Pleshkov
cb6958f56c using quantization in plain segment (#7060) 2025-08-20 13:08:58 +02:00