upstream-archive/qdrant
1
0
Fork 0
mirror of https://github.com/qdrant/qdrant.git synced 2026-09-21 13:37:46 -05:00
Code Issues Packages Projects Releases Wiki Activity
6,642 Commits 438 Branches 119 Tags
84b2738823d460450bcef800f23bd76c0463e71e
Commit Graph
6642 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Luis Cossío 84b2738823 flush after prefilling deleted vectors (#9992) 2026-08-04 11:18:44 +02:00
dependabot[bot] 56e25fd7e1 build(deps): bump thiserror from 2.0.18 to 2.0.19 (#10003) 
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 2.0.18 to 2.0.19.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/2.0.18...2.0.19)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-version: 2.0.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:44 +02:00
dependabot[bot]Cursorroot
 		5c51522081 build(deps): bump prettyplease from 0.2.37 to 0.3.0 (#9999) 
* build(deps): bump prettyplease from 0.2.37 to 0.3.0

Bumps [prettyplease](https://github.com/dtolnay/prettyplease) from 0.2.37 to 0.3.0.
- [Release notes](https://github.com/dtolnay/prettyplease/releases)
- [Commits](https://github.com/dtolnay/prettyplease/compare/0.2.37...0.3.0)

---
updated-dependencies:
- dependency-name: prettyplease
  dependency-version: 0.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: upgrade macros to syn 3 for prettyplease 0.3

prettyplease 0.3 depends on syn 3, so macros must use the same syn
version to avoid File type mismatches in tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:44 +02:00
dependabot[bot] a3b875a991 build(deps): bump pyroscope from 2.1.0 to 2.1.1 (#9994) 
Bumps [pyroscope](https://github.com/grafana/pyroscope-rs) from 2.1.0 to 2.1.1.
- [Release notes](https://github.com/grafana/pyroscope-rs/releases)
- [Changelog](https://github.com/grafana/pyroscope-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/grafana/pyroscope-rs/compare/lib-2.1.0...lib-2.1.1)

---
updated-dependencies:
- dependency-name: pyroscope
  dependency-version: 2.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:44 +02:00
dependabot[bot] 34ccca32a4 build(deps): bump uniffi from 0.31.2 to 0.32.0 (#10001) 
Bumps [uniffi](https://github.com/mozilla/uniffi-rs) from 0.31.2 to 0.32.0.
- [Changelog](https://github.com/mozilla/uniffi-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mozilla/uniffi-rs/compare/v0.31.2...v0.32.0)

---
updated-dependencies:
- dependency-name: uniffi
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:44 +02:00
qdrant-cloud-botandCursor d5d7011cab build(deps): bump quinn-proto from 0.11.14 to 0.11.16 (#10009) 
Same dependency bump as #10006, applied on top of dev.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:44 +02:00
Ivan PleshkovandClaude Opus 5 8b0d092c01 Fix TQ quantized vector layout alignment (#10005) 
`EncodedVectorsTQ::layout()` declared `align_of::<f32>()`, but the size it
reports is the packed dimensions plus a 4-byte-multiple extras trailer, which
is not a multiple of 4 for three quarters of all dimensions (e.g. dim=756,
Bits4: 378 + 4 = 382 bytes).

The claim was never true — the encoded storage packs vectors at
`id * quantized_vector_size` with no per-vector padding — and nothing relies on
it: packed dimensions are read through unaligned SIMD loads (`loadu` / `vld1`)
and the extras trailer through `f32::from_le_bytes` on a byte slice.

It is also actively harmful. Inline HNSW storage packs link vectors
back-to-back using this layout and rejects one whose size is not a multiple of
its alignment, so building an index with `inline_storage` enabled fails for
those dimensions — and retries forever as an optimization crashloop.

Use `align_of::<u8>()`, matching scalar and product quantization. Old links
files stay readable: both layouts are persisted in the file header and the
reader takes size and alignment from there, never from the live quantizer.

Add a test covering the `size % align == 0` invariant across awkward
dimensions, bit widths, distances and modes — `layout()` had no coverage, which
is why the mismatch went unnoticed on the multiple-of-32 dimensions everyone
uses in practice.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-04 11:18:44 +02:00
dependabot[bot] 81a0e43c73 build(deps): bump regex from 1.13.0 to 1.13.1 (#10002) 
Bumps [regex](https://github.com/rust-lang/regex) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.13.0...1.13.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-version: 1.13.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:43 +02:00
dependabot[bot] 7ed581ee7e build(deps): bump tokio from 1.52.3 to 1.53.1 (#9996) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.3 to 1.53.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.52.3...tokio-1.53.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.53.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:43 +02:00
Sasha Denisov acbdbe334f test(edge-ffi): assert Match.Any/Except land in the correct AnyVariants arm (#9991) 
The AnyVariants redesign's tests only checked `is_ok()`. Strengthen them to
pin the actual contract now that the strings-XOR-integers constraint is
type-enforced:

- destructure the converted `SegmentMatch` and assert the values land in the
  matching engine `AnyVariants` arm, in order (catches a mis-wired `From`);
- cover the previously-missing `Except` x `Strings` corner;
- assert `into_iter().collect()` into the engine's `IndexSet` dedups
  (first-seen order), matching the engine's own construction;
- pin empty-set validity for both `Any` (matches nothing) and `Except`
  (matches everything) — the one semantic the docstring promises and the old
  XOR guard used to reject.

Uses `let`-else rather than a wildcard match arm to satisfy the crate's
`wildcard_enum_match_arm` lint on engine enums. 3 tests -> 7; suite 122 -> 126.
 2026-08-04 11:18:43 +02:00
dependabot[bot] eabd17612b build(deps): bump bytemuck from 1.25.1 to 1.25.2 (#10000) 2026-08-04 11:18:43 +02:00
dependabot[bot] 82630ef54a build(deps): bump serde_json from 1.0.150 to 1.0.151 (#9997) 2026-08-04 11:18:43 +02:00
dependabot[bot] 84c89e8214 build(deps): bump quote from 1.0.46 to 1.0.47 (#9998) 2026-08-04 11:18:43 +02:00
dependabot[bot] efb8891480 build(deps): bump async-trait from 0.1.89 to 0.1.91 (#9995) 
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.89 to 0.1.91.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.89...0.1.91)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-version: 0.1.91
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:43 +02:00
dependabot[bot] e0cb3f0856 build(deps): bump astral-sh/setup-uv from 8.3.2 to 9.0.0 (#9993) 
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.2 to 9.0.0.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/11f9893b081a58869d3b5fccaea48c9e9e46f990...c771a70e6277c0a99b617c7a806ffedaca235ff9)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-08-04 11:18:43 +02:00
Andrey VasnetsovClaude Opus 5Cursor
 		73e6c16f9c Remove deprecated search endpoints from OpenAPI, deprecate them in gRPC (#9982) 
* Remove deprecated search/recommend/discover endpoints from OpenAPI

Remove deprecated REST API endpoint definitions from the OpenAPI
generator. These endpoints were deprecated in v1.13.3 (`f4ced2567`,
#5907, 2025-01-30) in favor of the universal `/points/query` endpoint:

- POST /points/search
- POST /points/search/batch
- POST /points/search/groups
- POST /points/recommend
- POST /points/recommend/batch
- POST /points/recommend/groups
- POST /points/discover
- POST /points/discover/batch

Also removes the corresponding request types from the schema generator
and updates the expected API count in the consistency check.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Migrate OpenAPI integration tests to /points/query

The deprecated /points/search, /points/recommend and /points/discover
endpoints (along with their /batch and /groups variants) were removed
from the OpenAPI spec, which caused validation failures in the Python
integration test harness.

This commit migrates the affected tests to the universal /points/query
endpoint:

- Delete tests dedicated to the deprecated endpoints:
  test_recommend.py, test_discover.py, test_multicollection_reco.py,
  test_recommendation_multivector.py
- Refactor remaining tests to call /points/query (and /query/batch,
  /query/groups), translating request bodies (vector -> query / using,
  positive/negative -> query.recommend, target/context -> query.discover)
  and unwrapping the new result.points response shape.
- Drop equivalence assertions against the now-removed legacy endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Relax non-empty assertions in migrated recommend/discover tests

The previous migration added `len(...) > 0` assertions to tests that
previously only checked equivalence between the deprecated and new
API. These assertions are too strict because the parametrized
`query_filter` cases legitimately produce empty result sets.

Drop the `> 0` assertion and rely on `request_with_validation` to
verify the response is well-formed and HTTP OK.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Migrate remaining OpenAPI tests off deprecated search endpoints

Tests added to dev after the original migration was written still call
/points/search and /points/recommend/groups through
`request_with_validation`, which resolves the endpoint against the
OpenAPI spec and therefore breaks once the endpoint is not in the spec:

- test_turbo4_storage.py, test_sparse_idf_corpus.py, test_validation.py:
  translate /points/search to /points/query (vector{name,vector} ->
  query + using, result -> result.points).
- test_group.py: drop the /points/recommend/groups half of the
  lookup_from validation test in favour of the query equivalent.

test_sparse_idf_corpus.py's test_query_api_supports_idf_corpus goes
away: with the helper on /points/query every test in the file now
exercises what it asserted.

Also record why test_recommend_group cannot assert on its groups: it
uses every point in the collection as a recommend example, so all of
them are excluded and the result is legitimately empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Regenerate openapi.json without the deprecated search endpoints

Drops the 8 deprecated paths and the request schemas that only they
referenced: Search/Recommend/Discover request (+Batch, +Groups) types
and their exclusive dependencies (NamedVector, NamedSparseVector,
NamedVectorStruct, UsingVector, RecommendExample, ContextExamplePair).

Regenerated output is a strict subset of the previous spec, and every
remaining $ref still resolves.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Deprecate the search/recommend/discover RPCs in gRPC

The REST counterparts have carried `deprecated: true` since v1.13.3 and
are now gone from the OpenAPI spec, while the gRPC RPCs never got any
deprecation annotation at all. Mark all 8 with `option deprecated = true`
so generated clients warn, and point each doc comment at its `Query`
replacement.

tonic puts `#[deprecated]` on the generated client methods only; the
server trait gets the doc comment alone, so our own `impl` is unaffected.
The RPCs keep serving traffic — this is annotation only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Restore the deleted recommend/discover suites on /points/query

The earlier migration deleted these four files outright, but the
query-side tests it left behind are all shallow smoke tests
(`len(result) > 0`, `"points" in result[0]`). The deleted ones carried
invariants with no query-API equivalent anywhere, so deleting them was a
real loss of coverage rather than de-duplication:

- test_recommend.py: default strategy equals average_vector; batch
  results identical to sequential singles across six request shapes;
  best_score with only negatives yields all-negative scores; best_score
  with a single positive orders identically to a nearest query; raw
  vectors as examples equal ids as examples.
- test_discover.py: context-only scores are all <= 0; target-only orders
  identically to a nearest query but scores differently; with a fixed
  context the integer part of the score is stable while the decimal part
  moves, and vice versa with a fixed target; batch equals singles;
  lookup_from by id equals by vector.
- test_multicollection_reco.py: cross-collection lookup_from, plus
  wrong-vector-size, unknown-collection and unknown-vector rejections.
- test_recommendation_multivector.py: the same recommend invariants over
  a max_sim multivector collection, which the query suite never covered.

Only test_recommend_missing_lookup_from_collection_with_raw_vector is
dropped as genuinely redundant — test_query.py's
test_query_missing_lookup_from_collection covers query, query/batch and
prefetch.

Two request-shape differences the translation had to absorb:

- Giving no examples at all is 422 (a RecommendInput validation rule),
  where the legacy API reported 400 from the query itself. A malformed
  example, such as an empty vector, is still 400.
- DiscoverInput requires the `context` key and accepts only an explicit
  null to mean "no context", so target-only discover must spell it out.
  The legacy API let it be omitted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-04 11:18:43 +02:00
Andrey VasnetsovClaude Opus 5xzfc
 		5819d224da fix: evict page cache of files that are mapped twice (#9984) 
* fix: evict page cache of files that are mapped twice

`clear_cache()` on a `memory: cold` vector storage was a silent no-op: after
optimization the whole storage stayed resident in the page cache.

`MmapFile` opened with `need_sequential` holds two mappings of the same file
(`MADV_RANDOM` + `MADV_SEQUENTIAL`), and `MADV_PAGEOUT` skips any page carrying
more than one page-table reference. Any page faulted through both mappings was
therefore never reclaimed, no matter which mapping was advised. Quantization is
one way to get there: it reads the raw vectors through the sequential mapping
while `populate_vector_storages()` populated the random one, so with
quantization enabled `matrix.dat` stayed 100% cached after the build, and
without it the same build evicted down to ~1%.

`POSIX_FADV_DONTNEED` alone does not help either, as it skips pages with any
page-table reference. So zap the page tables of both mappings first
(`MADV_DONTNEED` on a shared file mapping only drops the PTEs; the data stays
in the page cache and refaults on the next access) and then evict through the
file. Dirty pages are still kept, exactly as before — `MADV_PAGEOUT` did not
write back filesystem pages either — so callers that flush first, like
`SegmentBuilder::build`, get a complete eviction.

Also affects the payload storage (gridstore pages), the disk id tracker reader
and quantized multivector offsets, which open with `need_sequential` too.

Measured on a 200k x 256 build with quantization: `matrix.dat` 100% -> 0.0%
resident, whole segment 67% -> 2.4%.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: satisfy clippy::cast_lossless in the eviction test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* MADV_DONTNEED should not be safe

* Document why Madviseable::clear_cache might not be enough

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
 2026-08-04 11:18:43 +02:00
Jojii 98c1ed0fae Make folder structure consistent (#9990) 2026-08-04 11:18:43 +02:00
Sasha Denisov 8e54ad254b refactor(edge-ffi): model Match.Any/Except as a typed AnyVariants one-of (#9988) 
Match::Any/Except took `{ strings: Option<Vec<String>>, integers: Option<Vec<i64>> }`
with a runtime XOR check — two of the four representable states were invalid
(both-none, both-set -> InvalidArgument), and the prior field defaults made the
all-invalid empty call the easiest thing to type in Kotlin/Swift.

Replace with a typed sum type `AnyVariants { Strings | Integers }`, matching the
engine's own `AnyVariants`, the gRPC `oneof`, the crate's existing
`ValueVariants`, and every official Qdrant client (Python union, Java/Go/Rust
typed constructors, JS `string[] | number[]`). "Exactly one" is now enforced at
compile time; the two InvalidArgument paths and the field defaults are gone.
Empty sets remain valid (Any -> matches nothing, Except -> matches everything),
mirroring the engine.

122 crate tests pass; clippy clean; regenerated bindings expose
`Match.Any(AnyVariants.Strings(...))`.
 2026-08-04 11:18:43 +02:00
Sasha Denisov ea6688f00a fix(edge-ffi): default every Option field to None so optionals are skippable (#9987) 
`#[uniffi(default = None)]` was applied to request Record fields but not to
enum-variant fields (Query / Match / Fusion, payload-index params) or the
response Records — an oversight, not a UniFFI limitation: uniffi 0.31 fully
supports defaults on enum-variant fields (verified end-to-end by regenerating
the bindings and confirming `= null` lands, e.g. `Query.Nearest.using`).

Annotate every remaining Option<T> field (enum variants + Records) so the
generated Kotlin/Swift bindings default them to null/nil. Consumers can now omit
any optional field — e.g. `Query.Nearest(vector)` instead of
`Query.Nearest(vector, using = null)` — and adding a new optional field to a
Record or enum variant stays source-compatible (named-argument callers).

Constructor/method arguments already use
`#[uniffi::constructor/method(default(...))]` and are unchanged.

Verified: 123 crate tests pass; regenerated bindings have zero optional fields
without a default.
 2026-08-04 11:18:43 +02:00
Tim ViséeClaude Opus 5generallLuis Cossío
 		71d99b144c Add Logstore and Blobstore wrapper (#9673) 
* Gridstore: introduce storage operating mode in config

Add a mode field to the gridstore config, selecting between the dynamic
mode (current behavior, the default) and the upcoming serverless mode.
The mode is specified through StorageOptions on creation, persisted in
config.json, and read back first when opening so the correct variant can
be selected automatically. Configs written before this field existed
deserialize as dynamic.

For now, selecting the serverless mode returns an error; the variant
itself is added in follow-up commits.

* Gridstore: move dynamic implementation into dedicated module

Mechanical move of the current Gridstore implementation into
gridstore/dynamic.rs as DynamicGridstore. The public Gridstore struct
becomes a thin wrapper holding a mode variant enum, propagating every
call into the selected variant. For now the enum only has the dynamic
variant; the serverless variant is added in follow-up commits.

No logic changes to the dynamic implementation itself: only visibility,
the config parameter now passed into open (the wrapper reads it first to
select the mode), and open_or_create staying on the wrapper.

* Gridstore: add serverless tracker

Add the append-only mapping tracker for the serverless storage mode.

The tracker file is a plain array of 16-byte mapping entries without any
header: the number of mappings is defined by the exact file length, and
the entry index is the point offset. The file starts empty and only ever
grows by appending, existing bytes are never rewritten. Mappings must be
set in monotonically increasing point offset order; skipped offsets are
backfilled as zeroed entries which decode as None.

New mappings are buffered in memory and appended with a single write per
flush. A flush with a stale target is a no-op so bytes are never written
twice. A torn trailing entry (file length not a multiple of the entry
size) is ignored when reading and truncated away when opening writable.

Unlike the dynamic tracker, the file is read and written directly with
positional file IO instead of memory mapping, as serverless environments
do not handle memory mapped files well.

* Gridstore: add serverless storage variant

Add the append-only gridstore variant for serverless deployments, which
restrict IO to appending to files: existing bytes can never be
rewritten, and IO is expensive so as few files as possible are used.

The variant stores all value data in a single page file next to the
serverless tracker and the storage config, three files in total. Both
data files start empty and only ever grow by appending; there is no
preallocation, no used-block bitmask and no gap/region bookkeeping.
Values are appended at put time at the next block aligned offset, with
the zero padding included in the write so it lands exactly at the end
of the file. Mappings are buffered and appended to the tracker with a
single write per flush, after the page file is synced, so a mapping on
disk never points at data that is not durable.

Values cannot be updated or deleted, and must be put at monotonically
increasing point offsets; violations are rejected before any data is
written. Files are read and written directly, never memory mapped.

The mode is selected through StorageOptions on creation and picked up
automatically from the persisted config when opening.

* Gridstore: serverless support in reader and view

Extend the read-only GridstoreReader and the GridstoreView with the
serverless mode, keeping both public types unchanged: like the writable
Gridstore they now hold a mode variant internally, selected
automatically from the persisted config when opening.

The serverless reader holds the tracker and page directly and reads the
files positionally, without memory mapping. A live reload re-reads the
mapping count from the exact tracker file length (there is no size
header), ignoring a torn trailing entry, and never truncates as it is
read-only. Value reads always go directly to the file, so newly
appended data is readable without remapping anything.

* Gridstore: document storage operating modes

* Gridstore: review fixes for the serverless mode

Hardening and cleanup from a review pass over the new serverless
storage variant:

- Batch the reader side iteration like the writer already did, instead
  of materializing tracker mappings for the full range in one go, which
  could transiently allocate gigabytes on large storages.
- Recover the append cursors when a positional write fails partway:
  truncate the file back to the tracked length so a retried append or
  flush never rewrites bytes that already landed in the file.
- Validate page addressability before appending value data, a rejected
  put must not grow the page file.
- Cross-check tracker and page consistency when opening: mappings that
  reference value data past the end of the page file (e.g. after a
  partial copy or restore) now fail fast instead of surfacing as
  opaque read errors per point.
- Reject value pointers into any page other than page 0 on the
  serverless read path with PageNotFound, matching the dynamic mode
  contract, instead of silently reading from a wrong location.
- Refresh the reported storage size on reader live reload even when no
  new mappings were flushed, unflushed value data may have grown the
  page file already.
- Validate configs read from disk: a corrupt config with zero sized
  blocks, pages or regions is now rejected when opening instead of
  panicking on a division by zero later.
- Classify rejected serverless puts as UnsupportedOperation, consistent
  with rejected deletes, so they don't surface as user-facing
  validation errors at the segment level.
- Deduplicate the compression dispatch into Compression::compress and
  Compression::decompress, and the serverless file create/open patterns
  into shared direct IO helpers, so the two modes and files can't
  silently drift apart.

* Gridstore: cover both operating modes in mode-agnostic tests

Parameterize the gridstore tests that exercise mode-agnostic behavior
over both the dynamic and serverless mode with rstest, using a
single and bulk put/get roundtrips, storage files, basic persistence,
corrupt config rejection, batched read congruence, reader live reload,
and the different block sizes.

Mode specific expectations branch inside the tests: expected file
names, storage size semantics (whole blocks vs exactly packed bytes),
value pointer layout (page spill over vs a single packed page), and
gaps (created by deletes in dynamic mode, by skipped puts in serverless
mode). Dynamic-only internals assertions are kept behind a mode check.

Tests around updates, deletes, page spanning, block reuse and other
dynamic-only behavior intentionally stay dynamic; the serverless
specific format invariants remain covered by the dedicated serverless
tests.

* Gridstore: port serverless specific tests from sibling branch

Source the serverless specific test cases that the
serverless-gridstore-updates branch added, adapted to the dedicated
variant implemented here (distinct file names, headerless tracker
with 16 byte entries, a single packed page without trailing padding,
and rejected re-puts):

- writes only ever append: tracker and page files only grow and
  previously written bytes stay byte-for-byte untouched
- new mappings land exactly at the end of the tracker file, which
  always covers the exact number of mappings
- mapping gaps are zero-padded on disk and survive reopening
- values are packed back to back at block aligned offsets, the page
  file ends exactly at the last value
- serverless mode never creates nor reports block flag files
- a flusher persists exactly the mappings that existed at its
  creation, later puts stay pending
- a config claiming the wrong mode fails loudly in both directions
  instead of loading the incompatible file format of the other mode

Tests around their mode switching, page spanning and tolerated deletes
don't apply to this design and are intentionally not ported.

* Gridstore: test serverless production risk scenarios

Add tests for the operational aspects that matter before serverless
mode goes to production, each covering a scenario that wasn't
evaluated yet:

- Replayed puts of already persisted offsets (a WAL redo after a
  crash where the flush completed but was never acknowledged) are
  rejected without appending anything, and max_point_offset is the
  exact offset a replay must resume at.
- The accepted crash case of a tracker file extended with zeroed
  bytes: the entries count as permanent None mappings, can never be
  put again, and the storage stays consistent and writable past them.
- The read-only reader never modifies the files: opening over a torn
  tracker tail, reading, iterating and live reloading leave both
  files byte-for-byte untouched.
- A multi-round put/flush/reopen cycle always exposes exactly the
  flushed prefix, with the mapping count matching the exact tracker
  file length and unflushed offsets reusable.
- An append beyond the maximum addressable block offset is rejected
  before writing anything, keeping retried puts from growing the page
  file unboundedly.

* Gridstore: rename serverless mode to append-only, split into module

Rename the mode after its defining characteristic instead of its
deployment target: files only ever grow, existing bytes are never
rewritten. Renames Mode::Serverless to Mode::AppendOnly (persisted as
"mode": "append_only") and the on-disk file names to
append_only_tracker.dat and append_only_page_0.dat. The serverless
deployment motivation stays in the documentation.

Also split the single 2300 line serverless.rs into an append_only
module with dedicated files for the storage, page, view, reader and
tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Use universal IO in Gridstore

* Include upstream preopen logic in new Gridstore variant

* Gridstore: buffer append-only value writes until flush

In append-only mode, put previously wrote the value data to the page
file right away, one write operation per put, while mappings were
already buffered and batch persisted on flush. Buffer value writes the
same way: both the value and its mapping now only land on disk once a
flush cycle executes.

This batches all new value data into a single write operation per
flush, which is significantly more efficient on S3 based storage where
every write is a costly operation. A flush now performs exactly two
writes: one appending all buffered value data to the page file, one
appending all pending mappings to the tracker file, in that order, so
a mapping on disk never points at value data that is not durable.

The page mirrors the tracker's pending mechanism: an in-memory buffer
that is byte for byte the next append (zero padding between block
aligned values included), a watermark captured at flusher creation so
puts made during a flush stay buffered, a stale-flush no-op guard so
appended bytes are never written twice, and truncate-back recovery on
failed writes. Reads transparently serve buffered values from memory.

As a side effect, a crash between flushes now leaves nothing on disk
at all, where the write-through approach left orphaned value bytes in
the page file. The buffered data is held in memory until the next
flush, bounded by the flush cadence.

Universal IO filesystem handles are now required to be Send + Sync, so
the flusher closure can carry one to grow the page file at flush time;
all existing backends already satisfied this.

* Gridstore: rename inner DynamicGridstore to Gridstore

The dynamic variant keeps the Gridstore name; the outer dispatching type
will be renamed to Blobstore in a follow-up. Until then the inner type is
referred to as dynamic::Gridstore to distinguish it from the outer type.

* Gridstore: rename append-only variant to Arenastore

The append-only variant stores all value data in a single ever-growing
page, allocating space by appending, hence: arena store.

* Gridstore: rename outer storage type to Blobstore

The outer type dispatching between the two storage variants is now called
Blobstore, being more generic than Gridstore. This frees up the Gridstore
name, which now exclusively refers to the dynamic mode variant, next to
Arenastore for the append-only variant. Storage components keep using the
outer type, so they now use Blobstore.

The gridstore crate name, GridstoreError, and the persisted names
(config.json mode, payload config storage_type) are unchanged.

* Gridstore: split Gridstore and Arenastore into dedicated modules

The outer module is now blobstore, matching the Blobstore type it
defines. The two storage variants each get their own submodule: the
dynamic Gridstore moves from dynamic.rs into gridstore/ with its reader
and view extracted from the shared files, mirroring the arenastore/
module (previously append_only/) which already had this layout.

* Rename gridstore crate to blobstore

The crate is named after the outer Blobstore storage type it provides.
The gridstore name lives on in the dynamic mode variant. GridstoreError
and the persisted names (config.json mode, payload config storage_type)
are unchanged.

* Arenastore: pack values back to back across multiple pages

Drop the block alignment from the append-only mode: values are packed
byte to byte, without blocks, and the tracker offset is now a plain byte
offset within the page. Blocks and regions are dynamic mode concepts;
their page size constraints no longer apply to append-only configs.

Bring back support for multiple pages. Once appending a value would
grow the current page beyond the configured page size, a new page is
started, bounding the size of and the number of appends to each file:
object stores like S3 Express limit the number of appends per object.
A value larger than the page size gets a page of its own; values never
span pages.

A rollover creates the new, empty page file at put time; the value data
itself stays buffered until the next flush, which appends to each
touched page with a single write, using per-page watermarks captured at
flusher creation. The reader scans for consecutively numbered page
files when opening, validates the most recent mappings against them,
and adopts pages created since on a live reload.

* Blobstore: rename dynamic mode to mutable

Rename Mode::Dynamic to Mode::Mutable, and the persisted config value
with it: config.json now writes "mode": "mutable". There is no
compatibility alias for "dynamic", released versions never wrote the
mode field (a missing field still defaults to mutable), only unreleased
storages did.

The Gridstore type and module names for the mutable variant are
unchanged.

* Fix Edge compilation due to package rename

* Review remarks

* Extract Gridstore preopen into module

* Rename Arenastore files

* Use universal IO for append operations

* Rename GridstoreError to BlobstoreError

The error type belongs to the Blobstore crate and is shared by both the
Gridstore and Arenastore variants, so it follows the crate naming. Also
update the user-facing error messages that referred to the old name.

* Split config into per-variant types

* Rename Arenastore to Logstore

Rename the Arenastore type to Logstore, including the reader, view,
config, module and variant names. The storage file names follow:
log_page_{n}.dat and log_tracker.dat. The persisted mode tag stays
"append_only".

* Move bitmask module into the Gridstore variant

The bitmask tracks free blocks, which only exists in the mutable mode.
Move the module from the crate root into the Gridstore variant that
owns it. It stays re-exported at the crate root because the bitmask
benchmark needs a public path.

* Move pages module into the Gridstore variant

Like the bitmask, the block based pages module is only used by the
mutable mode. Move it from the crate root into the Gridstore variant
that owns it. The Logstore variant has its own page implementation.

* Use universal IO for every Logstore operation

Replace the direct_io module with universal IO in the append-only
tracker, making the whole Logstore go through a universal IO backend
bounded by UniversalRead and UniversalAppend:

- The tracker is generic over the backend now. Reads go through
  UniversalRead with the caller's access pattern, flushes land as one
  atomic append with the same offset compare-and-swap recovery as the
  pages: a retried append after a lost acknowledgement is adopted
  instead of appended twice. A torn trailing entry is still truncated
  away on writable open, through a fresh handle since shrinking is not
  supported through an open one.
- The reader now schedules a prefetch for the tracker file too, it no
  longer bypasses the backend.
- The config write, clear and wipe use the backend file operations
  instead of local filesystem calls, matching the Gridstore variant.

* Batch reads in Logstore read_values

Apply the same batching logic as the Gridstore variant: resolve all
mappings first, then fetch the value data, both through the backend's
read pipeline so async backends can serve the reads in parallel.

The tracker gains a batched lookup mirroring the mutable tracker's
iter, serving pending mappings and out of range point offsets directly
from memory. The pages gain a batched value read; unflushed values are
served from the in-memory buffers, and since values never span pages
each value is a single read without reassembly.

Like in the Gridstore variant, the callback may now be invoked in a
different order than the requested point offsets.

* Better describe logstore live reload ordering

* use enum for options, swap `*Options`<->`*Config` naming

* don't wrap enum in struct

* ditch unused `StorageConfig`, make deserialization more ergonomic

* rename `*Options`->`*Config`

* make `preopen` non-blocking

* fixup! ditch unused `StorageConfig`, make deserialization more ergonomic

* fixup! use enum for options, swap `*Options`<->`*Config` naming

* fixup! don't wrap enum in struct

* fix rebase

* use `populate` param in Logstore

* test: failing repro of stale page after live reload across rollover

A reader that live-reloads between a page rollover and the following
flush adopts the new, still empty page. The previous page is then no
longer the last one and is never reloaded again, so the tail that the
next flush appends to it stays invisible to the reader forever:

    value pointer at byte 100 with length 100 is out of range

AppendOnlyPages::live_reload only reloads the last held page, assuming
earlier pages never change once a newer page exists. But the rollover
creates the new page file eagerly at put time, while the previous
page's buffered tail only lands at the next flush (see
test_rollover_writes_no_value_data_before_flush), so a page can keep
growing on disk after its successor exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: reload all pages that grew

* use Fs in `open_or_create`

* fix: publish tracker mappings only after the pages reload

`AppendOnlyTracker::live_reload` observed the mapping count and made it
visible in one step, before `LogstoreReader::live_reload` reloaded the
pages. Every failure path in the page reload -- `list_files`, reopening a
grown page, opening an adopted one, the truncation check -- therefore left
the reader with mappings referencing value data it never loaded, so reads
in the new offset range fail until a later reload happens to succeed. The
edge refresh loop keeps a segment whose reload failed, expecting it to keep
serving its pre-refresh state, which it then does not.

Split observing from publishing: `reload_count` refreshes the handle and
returns the count as a `PendingReload` token, `commit_reload` publishes it.
The reader still observes the tracker first, as the writer persists pages
before the mappings referencing them, but only commits once the pages are
loaded. Reopening without committing is harmless: reads stay bounded by the
unchanged count, and the bytes below it never change.

A partial failure inside the page reload needs no unwinding, pages running
ahead of the tracker is the safe direction.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf: batch the value reads in Logstore iteration

`LogstoreView::iter_range`, the path behind `Logstore::iter` and
`LogstoreReader::iter`, fetched the mappings for the whole range with a
single read but then read the values themselves one at a time, serially.
Gridstore routes its `iter` through `read_values` and pipelines both stages,
so a full scan of an append-only storage was the one read path without
batching -- one blocking round trip per value on the object store backends
this variant exists for. It is reached by payload storage iteration and by
the payload index build, which scans every payload.

Feed the pointers into `read_batch_values` instead, keeping the single
contiguous tracker read, which is better than the per-offset pipeline
scheduling Gridstore does on that side.

Values are now delivered through the read pipeline, so the callback may be
invoked out of order, as it already could be for Gridstore's `iter` and for
`read_values` in both variants. Both segment callers are order independent.
Tests that happened to rely on the mmap backend completing reads in
scheduling order now sort before comparing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: don't run the failed-page-reload test on Windows

The test shrinks a page file out of band to make the page reload fail, but
Windows refuses to resize a file while the reader holds it mapped, which it
does by construction here: "the requested operation cannot be performed on a
file with a user-mapped section open". The panic is on the injection itself,
the code under test never runs.

There is no portable injection. Truncating a page the reader holds is what
the check under test detects, so the mapping cannot be avoided; failing the
adopted page open instead needs a listed but unopenable file, and
`local_list_files` descends into matching directories rather than listing
them; failing the directory listing needs the storage directory removed,
which Windows also refuses while pages are mapped.

The storage itself is fine on Windows, its append path grows mapped pages
there and every other Logstore test passes. The logic under test is platform
independent and stays covered elsewhere, with the tracker half of the
guarantee pinned by `test_live_reload`, which runs on every target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
 2026-08-04 11:18:43 +02:00
qdrant-cloud-botandCursor 127cb23bf0 build(deps): bump pyasn1 from 0.6.3 to 0.6.4 in /tests (#9976) 
Security release addressing CVE-2026-59884, CVE-2026-59885, and CVE-2026-59886.
Equivalent to #9973 for the dev branch.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:43 +02:00
qdrant-cloud-botandCursor 34b58b45a3 test: wait for raft leader before collection recovery (#9972) 
POST /cluster/recover can return 200 while raft silently drops the
snapshot request when no leader is known yet, leaving the test stuck
on a missing collection until timeout.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:43 +02:00
Ivan PleshkovandClaude Opus 4.8 f91cbbcf4d [TQDT] TQ dense/multi vector storage consistency (#9953) 
* [TQDT] Align TQ vector storage layout with the reference dense storage

Make the TurboQuant vector storage structurally mirror the reference dense
(and multi_dense) storages, down to file names and their contents:

- Rename files + structs to the dense convention:
  - immutable.rs -> turbo_vector_storage.rs (ImmutableTurboVectorStorage ->
    TurboVectorStorageImpl)
  - appendable.rs -> appendable_turbo_vector_storage.rs
    (AppendableTurboVectorStorage -> AppendableMmapTurboVectorStorage)
  - multi.rs -> multi_turbo/appendable_mmap_multi_turbo_vector_storage.rs
    (TurboMultiVectorStorage -> AppendableMmapMultiTurboVectorStorage)
  - ReadOnlyTurboMultiVectorStorage -> ReadOnlyChunkedMultiTurboVectorStorage
- Thin out turbo/mod.rs to module declarations + re-exports: open_* fns move
  into their storage files, consts + turbo_storage_roundtrip into shared.rs,
  and TurboScoring / TurboMultiScoring join the other TQ traits in
  vector_storage_base.rs.
- Split read_only/ into the chunked storage (read_only/) and the single-file
  storage (read_only/immutable/), each with the mod/lifecycle/live_reload/
  read_ops 4-file layout, mirroring dense/read_only/.
- Introduce multi_turbo/ mirroring multi_dense/, with its own read_only/
  submodule holding ReadOnlyChunkedMultiTurboVectorStorage.
- Relocate the storage test suites to
  tests/test_appendable_turbo_vector_storage.rs and
  tests/test_appendable_multi_turbo_vector_storage.rs, paralleling the
  dense/multi_dense integration test files (tests moved verbatim, no new
  tests added).
- Fix a gpu-gated VectorStorageEnum match that referenced stale DenseTurbo /
  DenseTurboAppendable variant names.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* are you happy fmt

* fix after rebase

* [TQDT] Address review feedback on TQ vector storage split

- gpu tests: use the real `VectorStorageEnum::DenseTurboAppendableMemmap`
  variant (the old `DenseTurboAppendable` name never existed post-rename, so
  the gpu-feature test failed to compile — missed because `cargo build
  --features gpu` does not compile the `#[cfg(test)]` code).
- memory_reporter: report `DenseTurboUring` files as `FileStorageIntent::OnDisk`
  like the other io_uring variants; io_uring never mmap-caches, so delegating
  to `is_on_disk()` could wrongly report `Cached` for a populated backend.
- turbo_vector_storage: fix the misleading `insert_tq_bytes` doc comment — the
  single-file backend rejects the upsert via `?`, so `set_deleted` is never
  reached.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [TQDT] Fix clippy::wildcard_enum_match_arm in read-only routing test

Spell out the non-routing `VectorStorageType` variants instead of `_`, so a
future added variant fails the match rather than silently mapping to `false`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [TQDT] Fix stale Turbo4 storage-variant assertion in quantization test

The segment is built with the default (appendable/chunked) storage type, so a
Turbo4 datatype now lands in `DenseTurboAppendableMemmap`, not the single-file
`DenseTurboMemmap`. The assertion was left on the pre-split variant; align it
with the non-turbo branch, which already expects the appendable variants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* are you happy fmt

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-04 11:18:43 +02:00
Andrey VasnetsovandClaude Fable 5 8b7411e8d8 Support memory placement in service-level storage config defaults (#9950) 
* Support memory placement in service-level storage config defaults

Follow-up to #9684: `storage.payload.memory` and
`storage.collection.vectors.memory` set service-wide placement defaults
for newly created collections, deprecating `storage.on_disk_payload` and
`storage.collection.vectors.on_disk`.

Defaults resolve as: request `memory` > request legacy flag > service
`memory` > service legacy flag; exactly one level is filled to avoid
spurious memory-vs-legacy mismatch warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Mark hnsw_index.on_disk deprecated in config.yaml, document memory option

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:18:43 +02:00
Daniel Boros 3c55c33beb fix: fold appended points' deletion into read-only live_reload (#9948) 
* fix: fold appended points' deletion into read-only live_reload

* fix: batch appended-deletion reads in read-only live_reload

* fix: clamp appended-deletion reload to persisted flag length
 2026-08-04 11:18:43 +02:00
Luis Cossío f853a93a12 [CI] Enforce no default impl for batch methods (#9939) 
* [AI] Add ast-grep rule to avoid default batch trait methods

* reword

* fix `FullTextIndexRead::check_match_batch`

* move to `tools/ast-grep/`

* Add tests

* pin ast-grep version

* fix spelling
 2026-08-04 11:18:42 +02:00
qdrant-cloud-botandCursor d15819a815 test: fix flaky test_partial_snapshot optimizer race (#9951) 
Wait for green on write (and read after recover_read) so collection and
partial snapshots are not taken mid-indexing. Otherwise a leftover
appendable segment survives partial merge and breaks manifest equality.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
Sasha DenisovClaude Fable 5Ivan AfanasievrootCursorgenerall
 		785f10f7de feat: shared qdrant-edge-ffi crate (UniFFI boundary for mobile SDKs) (#9374) 
* Add shared qdrant-edge-ffi crate with UniFFI bindings

Introduce a shared FFI crate that wraps Qdrant Edge's core types with
UniFFI attributes so the same Rust source can power both the Swift and
Kotlin bindings. The crate lives at lib/edge/ffi/ and exposes ~60 public
types (EdgeShard, Point, Query, Filter, UpdateOperation, …) plus their
enum / record / sealed-union variants.

- lib/edge/ffi/src/       UniFFI-wrapped domain types (config, types,
                          filter, query, update, error, lib)
- lib/edge/ffi/bindgen/   Separate crate housing the uniffi-bindgen CLI
                          (needed so consumers of the `uniffi` runtime
                          don't have to depend on its CLI feature)
- lib/edge/ffi/uniffi.toml Sets the generated Kotlin package to
                          tech.qdrant.edge.ffi (Swift uses the crate
                          name verbatim)

Every public type carries Rust doc comments that UniFFI propagates to
Swift Quick Help and Kotlin KDoc, so the generated bindings ship with
first-class IDE documentation.

Workspace changes:
- Cargo.toml          Adds the new crates as workspace members and
                      introduces a `release-mobile` profile
                      (thin LTO, codegen-units=1, strip=symbols,
                      panic=abort) for size-conscious mobile builds
- Cargo.lock          Pins uniffi 0.31 and its transitive dependencies

Made-with: Cursor

* fix(edge-ffi): harden FFI boundary, add tests, quantization parity, optimize/HNSW

Builds on @ivan-afanasiev's qdrant-edge-ffi foundation (preceding commit) — takes
it to a tested, safe, reviewable state. Split out of #9359 per maintainer request
so the FFI crate can be reviewed in isolation; Swift/Android SDK PRs stack on top.

Boundary safety (host input → catchable error, never a process abort):
- release-mobile profile switched to `panic = "unwind"` so UniFFI's catch_unwind
  turns a panic into a catchable error (abort would risk WAL/segment consistency
  on an on-device DB).
- Fallible boundary conversions reject bad input (UUID, geo, JSON path, payload
  JSON, contradictory match filters) with InvalidArgument instead of panicking.
- Host-supplied counts bounded: limit/offset (bounded_limit, 1 Mi cap), vector
  size (1..=65536), HNSW params (m/payload_m ≤ 2048, ef_construct 4..=100000,
  max_indexing_threads ≤ 1024) — these drive eager allocation / thread spawning
  at optimize(), so unbounded values would abort uncatchably.

API:
- Quantization parity with the Python Edge SDK: all four strategies
  (Scalar/Product/Binary/Turbo) accepted; HnswIndexConfig + optimize() exposed
  (without optimize() search is brute-force).
- config() is an honest "as-requested" read-back (HNSW + quantization round-trip).
- EdgeError stays branchable (ShardClosed / InvalidArgument / OperationError;
  field is `reason`, not `message`, to avoid the Kotlin Throwable collision).

edge core (required by the boundary):
- EdgeShard::flush is fallible (OperationResult) instead of panicking on lock
  contention; Drop logs a flush error instead of aborting; python flush()? updated.
- scroll.rs drops a with_capacity(limit) pre-alloc a huge limit could turn into an
  allocator abort (defense-in-depth alongside bounded_limit).

Tests (CI: cargo +nightly test -p qdrant-edge-ffi): 4 unit + 22 conversion +
18 integration — persistence, crash-recovery, payload round-trip, concurrency,
delete-reload, search ranking, scroll pagination, quantization accept + config
round-trip, HNSW optimize, boundary rejection.

* fix(edge-ffi): validate geo radius/rings and reject empty field conditions

Three boundary-validation gaps surfaced in review of #9374, all rejected
now with EdgeError::InvalidArgument instead of producing wrong results or
reaching a panic in the geo index:

- GeoRadius: a negative or non-finite radius passed straight through to
  the geo index. Reject !is_finite() || < 0.0.
- GeoLineString rings (exterior + interiors of a GeoPolygon): the segment
  type was built by direct struct literal, bypassing the engine's
  validate_line_string (which only runs on the serde path). A malformed
  ring (<4 points or unclosed) could panic in the geo index on indexed
  payloads. Mirror validate_line_string at the single GeoLineString
  conversion chokepoint, covering both exterior and interior rings.
- FieldCondition: a condition with no predicate set is a silent no-op
  (matches every point). Reject it, mirroring the engine's
  validate_field_condition. This is the engine/gRPC/REST/Python contract
  of "at least one" predicate -- NOT "exactly one"; multiple predicates
  remain valid and AND together. The doc comment is corrected accordingly.

Adds 9 conversion tests (geo radius negative/NaN/infinite/valid, ring
too-few/unclosed/bad-interior, field-condition no-predicate/multiple).
cargo +nightly test -p qdrant-edge-ffi: 53 green.

* fix(edge-ffi): adapt to memory/idf API and fallible info after rebase

Keep FlushMode::Sync from recent segment-holder changes while preserving
fallible flush. Fill newly required memory/idf fields (matching the Python
edge bindings) and propagate EdgeShard::info()'s OperationResult.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci(edge): disable checkout credential persistence in edge-test

actions/checkout persists GITHUB_TOKEN into .git/config by default. This
workflow runs on pull_request from any branch and then builds and runs
repository-controlled code (Rust/Python examples), which could read or
exfiltrate that token. The job never pushes, so drop the persisted
credentials with persist-credentials: false.

Addresses a CodeRabbit security finding on the edge-ffi PR.

* fix(edge): block on lock in flush instead of failing on contention

flush() used try_lock()/try_read() and returned a 'lock busy' OperationError
when a concurrent update/optimize held the WAL or segment lock. That branch is
the wrong trade-off: callers of flush() expect their data persisted, and the
one spot it would fire on the direct Rust path is exactly when an in-flight
update is holding the WAL lock across its whole operation — i.e. when there is
unflushed data most worth persisting. At the FFI boundary it is moreover dead
code, since the outer Mutex<Option<EdgeShard>> already serializes every call.

Switch to blocking lock()/read(), matching the semantics update() and optimize()
already use on these same locks. flush() stays fallible so a genuine WAL/segment
I/O error is still surfaced rather than panicking. Drop cannot contend (it needs
&mut self, so no &self borrow can hold the locks), so it will not hang.

Addresses a CodeRabbit review nitpick on the edge-ffi PR.

* fix(edge-ffi): harden vector/query boundary from multi-agent review

Addresses findings from a multi-agent review of the FFI boundary:

- Multivector conversions were infallible: an empty outer Vec panicked in
  release (MultiDenseVectorInternal::new_unchecked only debug_asserts) and a
  ragged matrix was silently reshaped against row[0].len(), storing data the
  host never sent. Make NamedVector/Vector -> persisted conversions TryFrom and
  validate the matrix (non-empty, uniform non-zero row width) with the same
  rules as the engine's try_from_matrix, returning InvalidArgument.
- Reject non-finite (NaN/inf) vector components at ingest, mirroring the geo
  is_finite guard. The engine validates only dimensionality, so a poisoned
  component would be stored and later serialized back as JSON null silently.
- unload() now returns Result: it flushes explicitly and surfaces a final fsync
  failure instead of only reaching Drop's log line (no default log sink exists).
  On error the shard stays loaded so the host can retry.
- Cap filter/prefetch nesting depth (MAX_QUERY_NESTING_DEPTH). Condition::Filter
  and nested Prefetch are self-recursive; an unbounded host tree would overflow
  the stack — a SIGABRT that panic=unwind cannot catch. Reject deeper trees as
  InvalidArgument via depth-threaded conversion helpers.
- Replace the '""'-on-serialization-failure fallback in payload/vector JSON
  encoding with .expect (serialization is infallible; fail loud, not silent
  invalid JSON).
- Docs: correct the flush() # Errors (can return OperationError), the edge-core
  flush() caller enumeration, upsert_points/update_vectors # Errors (vector
  validation), and reword the fictional lib/edge/VERSION / version-sync comment
  in ffi/Cargo.toml to reflect that no automated check exists yet.

* test(edge-ffi): add behavior coverage for vectors, filters, query, updates

Adds 18 integration tests closing gaps a multi-agent review flagged (the suite
proved type conversion but not behavior):

Safety (back the new boundary validation):
- multi_vector_invalid_matrices_rejected — empty/ragged/zero-dim multi-vectors
- non_finite_vector_components_rejected — NaN/inf across single/named/multi/sparse
- finite_vectors_accepted_by_upsert_constructor — over-rejection guard
- deeply_nested_filter_rejected_shallow_accepted, deeply_nested_prefetch_rejected
  — depth cap rejects >64, accepts shallow

Behavior:
- filter_restricts_count_scroll_and_search — a filter actually narrows the result
  set across count/scroll/search (not just that conversion succeeds)
- flush_under_concurrent_upserts — flush() blocks under a concurrent update loop,
  no panic, final count == successful upserts
- vector_content_round_trips_through_retrieve_and_search
- cosine_distance_ranks_by_direction, euclid_and_manhattan_rank_nearest_first
- delete_points_by_filter / update_vectors / delete_vectors / delete_payload /
  clear_payload — the five previously-untested update ops
- multivector_round_trips, sparse_vector_round_trips
- rrf_fusion_over_prefetches_returns_fused_set

Suite: 4 unit + 32 conversion + 36 integration = 72, all green.

Not covered (blocked by FFI surface, tracked for follow-up): facet() and OrderBy
scroll both need a payload index, and UpdateOperation exposes no create-index
constructor.

* fix(edge): avoid lost-update TOCTOU in set_vector_hnsw_config

set_vector_hnsw_config read().clone()'d the config, mutated the clone, then
write(|c| *c = cfg) overwrote the whole config. The read lock is released before
the write, so a concurrent config update between the two is silently discarded
(lost-update TOCTOU). Use SaveOnDisk::write_optional to run the fallible mutation
on a clone inside the held lock, returning None on failure to abort persist+swap
without overwriting — the atomic pattern the repo's SaveOnDisk learning
prescribes for fallible config mutations.

Addresses a CodeRabbit critical finding on the edge-ffi PR.

* fix(edge-ffi): adapt read requests to edge::* structs after #9901 rebase

#9901 (Edge: request-specific structures for EdgeShardRead) moved the read API
off the shard/core request types onto edge-owned request structs. Retarget the
FFI conversions accordingly:

- QueryRequest/SearchRequest/CountRequest/ScrollRequest/FacetRequest/Prefetch
  now convert into edge::{QueryRequest,SearchRequest,CountRequest,ScrollRequest,
  FacetRequest,Prefetch} (were ShardQueryRequest/CoreSearchRequest/*Internal).
  Field shapes are identical except score_threshold, which is a plain ScoreType
  (f32) on the edge structs, not OrderedFloat — drop the wrap. Nesting-depth
  guard and bounded_limit validation preserved.
- retrieve() builds an edge::RetrieveRequest and calls the new single-argument
  EdgeShardRead::retrieve (was a 3-arg call).
- Cargo.lock reconciled onto dev's lockfile + the FFI/uniffi deps (dev advanced
  23 commits incl. dependency bumps).

cargo +nightly test --locked -p qdrant-edge-ffi: 4 unit + 32 conversion + 18
integration all green.

* feat(edge-ffi): full engine coverage — restructure, all update ops, all scoring queries, missing shard methods

Restructure the crate to mirror the edge crate's layout: shard.rs owns the
EdgeShard object and lifecycle, ops/ has one file per read operation
(request/response records + conversions + exported method together), and
update construction stays in update.rs. Multiple #[uniffi::export] impl
blocks merge cleanly in the generated bindings.

Interface modernization:
- retrieve() takes a RetrieveRequest record mirroring edge::RetrieveRequest
- config surface moves off the deprecated always_ram/on_disk booleans to a
  Memory placement enum (cold/cached/pinned); read-back resolves legacy
  flags via memory_placement(), None-preserving for HNSW
- EdgeShard.inner is RwLock<Option<...>>: operations take the read half and
  run in parallel; unload/update_from_snapshot take the write half and
  drain in-flight requests
- request/config records carry #[uniffi(default = ...)], so generated
  Swift/Kotlin constructors get default arguments (count exact=true, facet
  limit=10, HNSW 16/100/10000/0, everything optional defaults to nil/null)
- all conversions destructure their source exhaustively; intentionally
  unexposed internal fields are named `field: _` with a why-comment

Full update-operation coverage (Python SDK parity):
- upsert_points gains condition/update_mode (conditional upsert),
  update_vectors gains condition, set_payload gains a JSON-path key
- new: delete_vectors_by_filter, set/overwrite/delete/clear payload
  by-filter forms, overwrite_payload, create/delete_field_index (with a
  PayloadSchemaType enum), create_dense_vector, create_sparse_vector,
  delete_vector_name

Full scoring-query coverage:
- Query::Nearest takes a NamedVector (dense/sparse/multi-vector search)
- new Query variants: Recommend (BestScore/SumScores), Discover, Context,
  Feedback; new ScoringQuery variants: Formula (recursive Expression
  object with validating, depth-capped constructors) and Mmr;
  Fusion::Rrf gains weights

Missing shard methods: query_groups, search_matrix, create, path,
snapshot_manifest, update_from_snapshot (full + partial recovery),
set_hnsw_config, set_vector_hnsw_config, set_optimizers_config.

Two compile-time coverage maps (update.rs, ops/query.rs) exhaustively
match the engine's operation/query enum trees with no wildcard arms, so a
new engine variant fails compilation in this crate until the FFI surface
decision is recorded. A staging passthrough feature keeps them exhaustive
when shard/staging is enabled. The scoring-query map immediately caught
the otherwise-missed Mmr variant.

Tests: 82 total (4 unit + 32 conversion + 46 integration), including new
end-to-end coverage for field-index-enabled facet, conditional upsert,
overwrite/clear-by-filter, recommend/discover/context/MMR, formula
re-scoring, grouped queries, search matrix, vector-name ops, and the
lifecycle additions. Kotlin+Swift binding generation verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(edge-ffi): cover the remaining enum boundaries — filter tree, formula, schema, selections

Add coverage maps for the four construction-only enum families that had no
exhaustiveness guard, and expose everything they revealed as missing:

Filter surface (the big one — filter.rs map over Condition/Match/
RangeInterface/AnyVariants/ValueVariants):
- Condition::Slice — deterministic id-space slice filter, with the
  serde-path total/index validation re-applied at the boundary
- Condition::Nested — nested-object array filters, depth-counted against
  the recursion cap
- Match::TextAny / Match::Phrase / Match::Prefix — the three previously
  unreachable text-match modes
- FieldCondition.datetime_range — RFC 3339 datetime ranges (mutually
  exclusive with the float range, matching the engine's single
  RangeInterface slot)
- Filter.min_should
- WithPayload::Exclude — exclude-style payload selection (types.rs map
  over WithPayloadInterface/PayloadSelector/WithVector)
- OrderBy.start_from — Integer/Float/Datetime cursor (mapped in the
  scoring-query coverage map)
- CustomIdChecker recorded as intentionally unexposed (serde-skipped,
  runtime-internal)

Formula (formula.rs map over ExpressionInternal + DecayKind): all 17
expression variants were already constructible; the map now forces a
decision when the engine grows a new one.

Field-index schema and vector-name config (update.rs map extended):
PayloadFieldSchema::FieldType covered per schema type; the per-type
FieldParams forms recorded as not exposed yet; VectorNameConfig
Dense/Sparse tied to their constructors.

Tests: 91 total (+7 conversion, +2 integration: slice partitioning and
payload-exclude retrieval). Kotlin binding generation verified for the
new types.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(edge-ffi): resolve clippy warnings (inline format arg, large enum variant)

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(edge-ffi): address multi-agent review of the full-coverage expansion

Fixes findings from a multi-agent review (each independently re-verified),
targeting the new surface added in the full-engine-coverage expansion. All
verdicts CONFIRMED except query_groups (refuted → no change). 100 tests green.

- formula: cap Expression node count (MAX_FORMULA_NODES = 10_000), not just
  depth. Expression is a host-held Arc; combinators eagerly deep-clone their
  children's inner tree, so reusing one handle as several children
  (sum([e, e]) repeated) grows node count as 2^depth while depth stays under
  the 64 cap — an eager multi-GB clone that aborts the process (uncatchable
  under panic=unwind). Track a saturating size in node() and decay(); the
  depth guard stays (it protects the stack for narrow-but-deep chains).
- sparse: validate host sparse vectors at the boundary via
  validate_sparse_vector_impl (indices.len == values.len, unique indices).
  Without it a length mismatch reached an out-of-bounds panic at insert/scoring
  (opaque caught-panic, not a typed error) and duplicate indices silently
  double-counted. Route both sparse arms through a fallible helper; the
  infallible From<SparseVector> is removed so no path can bypass validation.
- order_value: surface it on ScoredPoint/Record (new OrderValue { Int, Float },
  matching the Python edge SDK). It was dropped behind a comment claiming "no
  order-by-scored surface" — false: order-by query/scroll and OrderBy.start_from
  are exposed, and ordered results carry a constant score and no next_offset, so
  order_value is the only way to resume ordered pagination when payload is off.
- search_matrix: drop from v1. It is an O(n^2) analytics op the reference Python
  edge SDK does not expose; its flat 1-Mi bounds are the wrong shape (one call
  hangs/OOM-crashes the device on a normal shard). Removing pre-publish is free;
  re-adding later with proper caps is non-breaking, the reverse is not.
- tests: node-count reject, sparse len/dup reject, order_value populated +
  start_from resume, and snapshot negative tests (bad path / corrupt archive /
  post-unload surface OperationError, not a panic, and the shard survives).

* fix(edge-ffi): reject oversized formula before the eager clone; close test gaps

Follow-up from a re-review of the fix commit.

- formula: the node-count check ran AFTER the eager `inner` deep-clone, because
  the clone was an argument to `node()` (evaluated before the function body). A
  host could build one accepted handle and reuse it as many children in one call
  (sum(vec![e; N])), materializing N x e.size nodes before the size check could
  reject it — the same uncatchable OOM abort the cap was meant to prevent, in two
  calls. `node()` now takes the children handles plus a build closure and runs
  both caps BEFORE invoking it, so a rejected tree never clones (decay() already
  did this). All combinators route through it.
- tests: decay node-count reject + happy path (decay has its own guard, was
  untested); wide-fanout reject (regression for the ordering fix); query-side
  ScoringQuery::OrderBy -> ScoredPoint.order_value (only the scroll/Record side
  was covered); real multi-page order-by continuation via StartFrom (was a
  single-page degenerate case); sparse values-longer reject; float OrderValue;
  with_payload=false omits payload. Suite: 107 green (4 + 39 + 64).

* feat(edge-ffi): keep search_matrix behind an off-by-default `matrix` feature

Reconsidered the outright removal: the FFI crate is a general UniFFI boundary,
not mobile-only, so non-mobile consumers (desktop/server/Rust) may want the
distance-matrix op. Instead of deleting it, gate the whole `ops/matrix.rs`
module behind a new off-by-default `matrix` Cargo feature.

- The mobile Swift/Kotlin bindings build without the feature, so search_matrix
  stays off the mobile surface (verified: default `cargo test` excludes it and
  its test; `--features matrix` includes both).
- The O(n^2) DoS is documented, not capped here: the op is opt-in and off the
  constrained mobile surface, so bounding sample_size/limit_per_sample for a
  given deployment is the enabling consumer's / SDK layer's responsibility. The
  module doc spells this out; the per-field bounded_limit only stops a lone
  u64::MAX value, not the quadratic compute.
- CI: add an `--all-features` test leg so the feature-gated surface (matrix +
  the pre-existing staging passthrough) can't bit-rot — this also closes the
  build-publish review note that `staging` had no CI coverage.

* docs+test(edge-ffi): tighten matrix doc wording; pin formula node-count boundary

Non-blocking polish from a final all-reviewer pass (6/6 APPROVE):

- matrix.rs / Cargo.toml docs: (1) the per-field bounded_limit caps at
  MAX_RESULT_COUNT (1,048,576), not merely a lone u64::MAX — say so, since 1 Mi
  is itself catastrophic for an O(n^2) op; (2) the DoS bound is owned by the
  opting-in consumer, not an "SDK/wrapper layer" that need not exist for a raw
  UniFFI consumer; (3) the mobile bindgen (a follow-up PR) must build with
  default features to keep the op off the mobile surface — stated as intent, not
  present-tense fact (no swift/android dirs exist yet).
- formula_node_count_exact_boundary: pins the exact MAX_FORMULA_NODES threshold
  (10_000 accepted, 10_001 rejected) — the existing tests jumped to ~16k, so the
  precise cap edge was unverified.

Suite: 108 default / 109 --all-features, all green.

* feat(edge-ffi): make the matrix feature on by default

Flip `matrix` to on-by-default (`default = ["matrix"]`). General (desktop /
server-side / Rust) UniFFI consumers now get `search_matrix` without opting in;
the mobile Swift/Kotlin bindgen builds with `--no-default-features` to drop the
O(n^2) analytics op from the mobile binding surface.

CI now covers all three shapes: default (matrix on), `--no-default-features`
(mobile surface, matrix excluded — guards the crate still compiles/passes
without it), and `--all-features` (matrix + staging). Verified: default 66 /
no-default 65 / all-features 66 integration tests, all green; no Cargo.lock
drift.

* fix(edge-ffi): rustfmt the sparse validator; correct stale matrix-feature docs

From a full all-reviewer pass of the review fixes:

- types.rs: rustfmt the `to_internal_sparse` `.map_err` chain. It failed
  `cargo +nightly fmt --all -- --check` (the rust-lint.yml CI gate) — the
  edge-test legs only run `cargo test`, so it slipped through locally.
- ops/mod.rs + integration.rs: fix two doc comments that still said the `matrix`
  feature is 'off-by-default' after it was flipped on-by-default. Left uncorrected
  they'd mislead the follow-up mobile-bindgen author into skipping
  `--no-default-features` and shipping the O(n^2) op to phones.

Design decisions (recorded): matrix stays on-by-default; the O(n^2) DoS is
documented, not capped — a static cap can't know the target device (compute cost
is device-dependent; the caller owns it via async/timeout). No cap added.

Reviewers: 3 APPROVE, 3 CHANGES-REQUESTED, all CR items were these two doc/fmt
misses plus the recorded design calls. All 3 feature configs green (66/65/66).

* fix(edge-ffi): resolve clippy errors in FFI tests

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(edge): reject conflicting vector-name re-create instead of desyncing config

The segment-level `create_vector_name` is idempotent: re-creating an existing
name is a no-op that leaves storage untouched. `EdgeShard::update` treated any
`Ok` from that no-op as success and unconditionally re-applied the requested
params to the shard config, so a second `create_dense_vector("v", 8, Cosine)`
after `("v", 4, Dot)` left `config()` advertising 8/Cosine while storage kept
4/Dot — a shape the shard then rejects on upsert. The empty name "" (the
default vector) hit the same path against a single-vector shard's primary field.

Reject a conflicting re-create up front (before the WAL, so it can't brick
replay) with a clear error, accept an identical re-create as idempotent, and
only re-apply config when storage actually changed.

* fix(edge-ffi): harden FFI boundary from an adversarial per-search-type review

Boundary-validation and doc fixes surfaced by fuzzing each query type with
executed repros:

- Reject non-finite floats that JSON cannot represent but the raw-f64 FFI can:
  order-by `StartFrom::Float` (a NaN panicked on a float-indexed field and
  silently truncated an integer scan), and formula `decay` midpoint/scale +
  `div` by_zero_default (a NaN evaded the engine's comparison-based range
  checks → debug panic across the boundary / silent all-zero rescore).
- Drop the `key` parameter from `overwrite_payload`/`overwrite_payload_by_filter`:
  the engine has no payload selector on the overwrite path (the server discards
  it too), so a keyed overwrite silently replaced the whole payload.
- Reject a `FieldCondition` carrying more than one predicate: the engine has no
  defined semantics for it (it evaluates one, index-dependent), so passing
  several through diverged silently from a Qdrant server. Callers AND predicates
  via separate `must` conditions.
- Doc fixes: FeedbackCoefficients b/c (exponent/multiplier, not weight/margin),
  RecommendStrategy::BestScore default, div-by-zero behavior, retrieve
  duplicate-ID collapsing, non-atomic partial-snapshot recovery.

Also fix the clippy `--all-targets -D warnings` lints in the test files that
reddened CI's `lint` job (uninlined_format_args, err_expect, disallowed
std::fs::write) and add regression tests for the validations above.

* fix(edge): compare only vector identity fields when detecting re-create conflicts

The conflicting-re-create guard added in the previous commit compared the full
`EdgeVectorParams`/`EdgeSparseVectorParams` via `PartialEq`, but a vector
declared in the initial `EdgeConfig` is stored with `on_disk: Some(false)` (from
`from_vector_data_config`) while a `CreateVectorName` op leaves it `None`. So an
identical re-create of a construction-defined vector — or of any vector after
`set_vector_hnsw_config` — was falsely rejected as a conflict, a regression on
the idempotent path.

Compare only the identity fields the op actually defines (dense: size, distance,
multivector_config, datatype; sparse: modifier, datatype); the storage/tuning
fields it cannot express are set from the config, the optimizer, or
`set_*_config` and must not trigger a false conflict. Adds a regression test
re-creating the config-defined "vec".

* style: apply cargo fmt

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(edge-ffi): expose payload_schema in info() and parameterized index creation

Close the last interface gap in the FFI surface: `info()` previously
elided the engine's `payload_schema`, and `create_field_index` only
accepted a bare type.

- New `payload_index` module mirroring the full `PayloadSchemaParams`
  family (all 8 index types incl. tokenizer/stopwords/stemmer options)
  with bidirectional conversions.
- `ShardInfo.payload_schema` reports each index's type, creation params,
  and indexed point count.
- New `UpdateOperation::create_field_index_with_params` constructor;
  coverage map updated accordingly.
- Boundary normalizations, following the quantization-config precedent:
  deprecated `on_disk` folds into the reported `memory` placement, and
  contradictory integer params (lookup+range both off) reject with
  `InvalidArgument`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Ivan Afanasiev <ivan.afanasiev@yahoo.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
qdrant-cloud-botandCursor c6d86fdf1a test: tolerate transfer race in corrupted snapshot recovery (#9947) 
#9013 skipped the manual replicate_shard when a transfer was already
visible, but the recovery loop can still start one between that check
and the POST. Accept 400 "already involved in transfer" as success so
the remaining wait assertions still cover recovery.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
Andrey VasnetsovandClaude Opus 4.8 a66b898dfc docs: describe StartResharding fields in OpenAPI (#9946) 
Add doc comments to `StartResharding` fields so the generated OpenAPI
spec explains what a user has to pass, and regenerate the spec.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
Jojii b2194f2109 Remove wrong debug assertion (#9945) 2026-08-04 11:18:42 +02:00
Ivan PleshkovCursorClaude Opus 4.8root
 		35b9aa0e2b [TQDT] Support Turbo4 in the read-only vector storage path (#9925) 
* feat(segment): support Turbo4 in the read-only vector storage path

Read-only segments (UniversalRead-backed: mmap / cache / remote) could not
open a `Turbo4`-typed vector storage — every dispatch arm returned "not yet
supported". This adds the missing read-only TurboQuant storages so read-only
segments can retrieve and score TQ-typed vectors.

- Reuse the existing, correct TQ scoring: extract `TurboScoring` /
  `TurboMultiScoring` traits and generalize `TurboQueryScorer` /
  `TurboCustomQueryScorer` (+ multi) and `raw_turbo_*_scorer_impl` from the
  concrete `TurboVectorStorage` to `&impl TurboScoring`. Scoring logic is
  untouched.
- Split `DenseTQVectorStorage` / `MultiTQVectorStorage` into read-only
  (`*Read`) + write supertraits, mirroring the dense storages, so the
  read-only storage need not implement the ingest path.
- Add `ReadOnlyTurboVectorStorage<S>` (single-file + chunked backends) and
  `ReadOnlyTurboMultiVectorStorage<S>` over `QuantizedStorage<S>` /
  `QuantizedChunkedStorageRead<S>` + `InMemoryBitvecFlags`; the quantizer is
  rebuilt from `(dim, distance)`, so nothing beyond the encoded bytes and
  flags is read from disk.
- Wire `DenseTurbo` / `MultiDenseTurbo` variants into `VectorStorageReadEnum`
  (dispatch, scorer, live-reload) and replace the `Turbo4` stubs in the
  read-only `open` / `preopen`.
- Round-trip tests: writable TQ -> reopen read-only, asserting byte-exact raw
  TQ bytes, identical dequantization, deletion flags and per-point scores
  matching the writable scorer (dense single-file + chunked, and multi).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* review remarks

* Split turbo read_only.rs into a module

Mirror the layout of the other read-only vector storages: split the
oversized read_only.rs into mod.rs (struct definitions + shared encoded
backend enum), lifecycle.rs (preopen/open), read_ops.rs (retrieval +
scoring trait impls) and live_reload.rs.

Co-authored-by: Cursor <cursoragent@cursor.com>

* review remarks

* fix after rebase

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
Arnaud GourlayandClaude Fable 5 9aecd0f6f5 Tests: use SmallRng for RNG-bound test data generation in segment (#9889) 
The turbo model tests, HNSW graph properties tests, and id-tracker
mapping tests generate their datasets with StdRng (ChaCha12 in rand
0.10). Switching to SmallRng (Xoshiro256++) makes the turbo model tests
~10-14% faster and the 400k-mapping id-tracker test ~30% faster.

All affected tests pass with the changed seeded sequences, including
the tolerance-carrying turbo model comparisons.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
qdrant-cloud-botandCursor d2946b8508 test: harden test_routing_token_sticky_reads against post-recovery flakiness (#9937) 
Right after the no_sync snapshot recovery, the recovered replica serves local
reads immediately, but a remote read to it can transiently fail for a short
window. The read path then falls back to the other replica in hash order on
just the requesting peer, so a single routing token momentarily resolves to
different replicas across peers (observed as {A, B, B}), failing the
determinism assertion.

Wait until token-routed reads are stable across all peers for every token the
test asserts on before measuring, so the transient post-recovery fallback
window is passed. Pure test-side change; routing behaviour is unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
xzfc 4645c498a9 Cleanup UniversalRead methods interface (#9934) 
* use common::generic_consts::{Random, Sequential};

* UniversalRead::read_batch: generic over E

* UniversalRead::read_batch: pass `AccessPattern` as ZST arg

* UniversalRead::read_bytes_iter: pass `AccessPattern` as ZST arg

* UniversalRead::read_iter: pass `AccessPattern` as ZST arg

* UniversalRead::read: pass `AccessPattern` as ZST arg

* UniversalRead::read_bytes: pass `AccessPattern` as ZST arg
 2026-08-04 11:18:42 +02:00
qdrant-cloud-botandCursor 0477976f7d test: de-flake test_upload_snapshot (robust to shard placement) (#9938) 
* test: make test_upload_snapshot robust to shard placement balance

The final assertion in recover_from_uploaded_snapshot assumed a perfectly
balanced shard placement (peer 0 having exactly 2*n_replicas remote shards).
Shard placement across peers is not guaranteed to be balanced, so this made
the test flaky (e.g. peer 0 ended up hosting all shards locally, leaving only
3 remote replicas instead of 4).

Instead, verify the full replica layout is healthy: peer 0 observes every
replica through its local + remote shards, so assert that all replicas are
Active and every shard has exactly n_replicas copies across the cluster.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: fetch cluster info once for shard validation

Read local and remote shards from a single /cluster response so both lists
come from the same cluster revision, per review feedback.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
xzfc a4a805acac UioResult (#9933) 2026-08-04 11:18:42 +02:00
Luis CossíoandClaude Fable 5 9a224a1d67 Mandatory batch impls (#9935) 
* make `EncodedStorage::for_each_batch` mandatory

* make `DenseVectorStorageRead::for_each_in_dense_batch` mandatory

* make `DenseTQVectorStorage::for_each_in_dense_batch` mandatory

* make `DenseTQVectorStorage::read_dense_tq_bytes` mandatory

* make `QueryScorer::score_stored_batch` mandatory

...and implement for tq multivectors

* [AI] make `IdTrackerRead::internal_versions_batch` mandatory

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* [AI] make `IdTrackerRead::external_ids_batch` mandatory

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* [AI] make `DiskMappingsSource::resolve_internal_batch` mandatory

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
Arnaud GourlayandClaude Opus 4.8 8cfa487e90 Honor applied_seq during WAL replay only under prevent_unoptimized (#9930) 
`load_from_wal` splits WAL recovery in two: it replays
`[first_index, applied_seq + APPLIED_SEQ_SAVE_INTERVAL + 1)` synchronously and
hands the remaining tail to the update worker, which applies it in the
background *after* `LocalShard::load` has returned and the shard has started
serving reads.

That split was introduced by #8008 and applies to every collection, so up to
`update_queue_size - 1` operations already acknowledged to a client with
`wait=true` can be missing from reads right after a restart, reappearing one by
one as the worker catches up.

Only `prevent_unoptimized` needs that routing: the update worker signals the
optimizer per operation, and optimization is the only thing that makes deferred
points visible. Everywhere else the synchronous replay is sufficient, so gate
the use of `applied_seq` on the flag -- the same condition that already gates
the worker's deferred-points wait -- and replay the whole WAL before load
returns, as it did before #8008.

Found by the crasher: after a crash-restart cycle it reported 72 missing points
out of a confirmed 3202, with the shard counting 3202 points while 3930 had been
acknowledged and 332 WAL entries were still queued across two shards.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
xzfc 5e2bba7148 ConditionChecker::check_batched: use in hnsw (#9845) 2026-08-04 11:18:42 +02:00
xzfcandLuis Cossío 8b7b7faee6 Batched ConditionChecker (#9740) 
* ConditionChecker::check_batched: trait + ConditionCheckerEnum

* check_batched for OptimizedFilter

* OnDiskPointToValues::values_iter_batch: improve performance

* OnDiskPointToValues::values_iter_batch: update interface

Accept bitvec, call on every point, pass UserData.

* check_batched for geo index

* geo_index tests: add same_geo_index_between_points_with_dups_test

Catches broken load_from_on_disk/for_all_points_values.

* check_batched for map index

* check_batched for numeric index

* check_batched for full-text index

* tests for check_batched

* Review fixups

- default_check_batched: avoid running pred twice at boundary
- condition_checker: use explicit match over matches!/if-else
- Partitioner: use assert! over debug_assert!

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>

* ConditionChecker::check_batched: &mut self -> &self

* ConditionChecker::check_batched: make mandatory

---------

Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
 2026-08-04 11:18:42 +02:00
Andrey VasnetsovandClaude Fable 5 8c11f488ac Clean up stale shard transfers when applying consensus snapshot (#9928) 
A transfer source that misses the transfer abort (e.g. while partitioned
or paused) keeps its local shard wrapped in a proxy. When such a peer can
only catch up via consensus snapshot, snapshot application re-creates
payload indexes with an update operation that the stale forward proxy
forwards to a transfer target which may no longer have the shard. The
resulting precondition error fails snapshot application and stops the
consensus thread ("No target shard N found for update"), leaving the
peer unable to ever catch up.

Snapshot application now explicitly cleans up transfers that are no
longer registered in consensus: the transfer task is stopped and the
proxy is reverted via the new `ShardReplicaSet::discard_proxy_local`,
which is infallible, never contacts the remote, and forgets queued
updates (replica states in the same snapshot already reflect the
transfer outcome).

The consensus test reproduces the incident: pause the transfer source
mid-transfer, restart the other peers so the aborted transfer can only
be learned via snapshot, and verify the source recovers.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:18:42 +02:00
xzfc cd01cabbfb deps: drop proc-macro-error2 (#9931) 2026-08-04 11:18:42 +02:00
Luis Cossío eaa505ecdc [io-bridge] split large reads into unordered chunks (#9896) 
* read large files in unordered chunks

* use S3 error

* use a vec of ranges to track scattering

* self nits

* use less concurrent chunks
 2026-08-04 11:18:42 +02:00
Jojii 4efc5ead9f Rename WrongVectorBytesSize -> MalformedVectorBlob (#9929) 2026-08-04 11:18:42 +02:00
Jojii f7ab9103c5 [TQDT] upsert raw malformed blob multi + sparse vectors (#9904) 
* Fix malformed vector upsertion for multivecs and sparse-vecs too

* Shorten comment

* Fix after rebase
 2026-08-04 11:18:42 +02:00
dependabot[bot]Cursorroot
 		3e1545b7cc build(deps): bump constant_time_eq from 0.4.2 to 0.5.0 (#9920) 
* build(deps): bump constant_time_eq from 0.4.2 to 0.5.0

Bumps [constant_time_eq](https://github.com/cesarb/constant_time_eq) from 0.4.2 to 0.5.0.
- [Changelog](https://github.com/cesarb/constant_time_eq/blob/main/CHANGES)
- [Commits](https://github.com/cesarb/constant_time_eq/compare/0.4.2...0.5.0)

---
updated-dependencies:
- dependency-name: constant_time_eq
  dependency-version: 0.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: bump constant_time_eq floor in Cargo.toml to 0.5.0

Cargo.toml still constrained to ^0.4.2 (<0.5.0), so --locked builds failed
and unlocked builds downgraded the lockfile back to 0.4.2.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
dependabot[bot]Cursorroot
 		48ac84a8ab build(deps): bump lz4_flex from 0.13.1 to 0.14.0 (#9922) 
* build(deps): bump lz4_flex from 0.13.1 to 0.14.0

Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.13.1 to 0.14.0.
- [Release notes](https://github.com/pseitz/lz4_flex/releases)
- [Changelog](https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pseitz/lz4_flex/compare/0.13.1...0.14.0)

---
updated-dependencies:
- dependency-name: lz4_flex
  dependency-version: 0.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: enable lz4_flex alloc feature for Vec-returning APIs

lz4_flex 0.14.0 gates compress_prepend_size/decompress_size_prepended
behind the new alloc feature when default-features are disabled.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-04 11:18:42 +02:00
Ivan Pleshkov 7b14104111 [TQDT] Fix/upsert raw malformed blob badinput (#9886) 2026-08-04 11:18:41 +02:00