Replace the mixed read interface (internal types, explicit parameter
enumeration, ad-hoc custom types) with edge-owned request structs, one
file per request in src/requests/. Each has a new() constructor taking
only the required parameters, a no-macro fluent builder in src/builders/,
and a From conversion into the internal request type in
requests/conversions/, grouped by request type. Conversions construct
and destructure with full field lists, so a parameter added on either
side fails compilation instead of being silently dropped.
The old reexport aliases (ScrollRequest = ScrollRequestInternal, etc.)
are replaced by the edge types under the same names; retrieve() takes a
RetrieveRequest instead of a parameter triple. Python bindings wrap the
edge types, and the published Rust examples use the builders.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add per-query IDF corpus for sparse vector search
Let the caller choose, per query, which population sparse IDF statistics
are computed over. `params.idf` is either `"global"` (default, unchanged
behavior) or `{"corpus": <filter>}`, where the corpus filter is
independent of - and usually broader than - the retrieval filter.
Decoupling the two keeps the score scale stable when the retrieval
filter tightens: term importance is measured against a population the
user names, not against whatever subset the filter happens to select.
Design decisions:
- Corpus grammar is restricted to a conjunction (`must`) of `match`
conditions on payload fields; loosening later is backward compatible.
- Strict mode validates the corpus filter like a read filter
(unindexed fields rejected).
- `idf` on a vector without the IDF modifier is a validation error,
never silently ignored.
- An empty corpus yields degenerate but corpus-scoped scores (smoothed
IDF over N=0), never a fallback to global statistics - in multi-tenant
collections a fallback would leak term statistics across tenants.
Implementation:
- QueryContext IDF stats are keyed by corpus, so one batch can mix
requests with different corpora.
- Statistics come from the sparse index: df(term) is counted over the
query terms' posting lists only, never by scanning stored vectors.
Small corpora (under ~1/32 of the segment, by cardinality estimate)
are kept as a sorted id list galloping through posting lists via
skip_to; large ones as a dense membership mask filled streaming from
the filtered-points iterator. A misestimated small corpus degrades
into the mask.
- Exposed uniformly: REST (`params.idf`), gRPC (`IdfParams` message),
edge python bindings; OpenAPI schema regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Apply rustfmt
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix clippy manual_is_multiple_of in sparse IDF corpus test.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Allow any filter as IDF corpus
Drop the must+match grammar restriction on the corpus filter. A
restriction enforced only as a validation step over the full Filter
type buys nothing; if a narrower corpus syntax is ever wanted, it
should be a dedicated API-level type instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix build: add memory field to SparseIndexConfig in idf corpus test
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: root <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Implement `__repr__` for `PyJsonPath`
* Implement `__repr__` for `PyFilter`
* Implement `__repr__` for `PyQuery`
* Implement `__repr__` for `PyQueryRequest`
* Implement `__repr__` for `PySearchRequest`
* Implement `__repr__` for `PyFilter` using `pyclass_repr` attribute
* Implement `__repr__` for `PyQuery` using `pyclass_repr` attribute
* Implement `__repr__` for `PyQueryRequest` using `pyclass_repr` attribute
* Implement `__repr__` for `PySearchRequest` using `pyclass_repr` attribute
* Refactor `PyUpdateOperation` constructors
* Add default parameters to `PyVectorDataConfig::new`
* Add `Repr` trait and `WriteExt` helper
* Implement `__repr__` for config types
* fixup! Implement `__repr__` for config types
Use `Copy` instead of `Clone`
* fixup! Implement `__repr__` for config types
Add basic test
* Implement `__repr__` for `PyPointId`
* Implement `__repr__` for `PyVector`
* Implement `__repr__` for `PyVectorInternal`
* Implement `__repr__` for `PyPayload`
* Implement `__repr__` for `PyValue`
* Implement `__repr__` for `PyPoint`
* Implement `__repr__` for `PyPointVectors`
* Implement `__repr__` for `PyRecord`
* Move `PyScoredPoint` into a separate file
* Implement `__repr__` for `PyScoredPoint`
* Cleanup examples
* fixup! Implement `__repr__` for `PyScoredPoint`
* Move `PyOrderValue` into separate file
* Add `PyScoredPoint::order_value`
* Implement `pyclass_repr` proc-macro attribute
* Implement `__repr__` for config types using `pyclass_repr` attribute
* Implement `__repr__` for `PySparseVector` using `pyclass_repr` attribute
* Implement `__repr__` for `PyPoint` using `pyclass_repr` attribute
* Implement `__repr__` for `PyPointVectors` using `pyclass_repr` attribute
* Implement `__repr__` for `PyRecord` using `pyclass_repr` attribute
* Implement `__repr__` for `PyScoredPoint` using `pyclass_repr` attribute
* Minor fixes and cleanups
* fixup! Minor fixes and cleanups
* rollback copy for quantization config
* rollback copy for quantization config
---------
Co-authored-by: generall <andrey@vasnetsov.com>
