Add prefix matching option to keyword index (#9683)

* Add prefix matching option to keyword index

Introduce an opt-in `prefix` option for the keyword payload index and a
new `match: { "prefix": ... }` filter condition, enabling efficient
byte-wise prefix filtering over keyword values (e.g. URL prefixes,
web-ui value autocompletion via facet + prefix filter).

Index side: a new `prefix_index.bin` file stores a sorted, front-coded
key dictionary with a resident block index (cumulative counts per
block); it is an ordered view over the keys of `values_to_points.bin`
and stores no postings. Presence of the file signals prefix support at
load time, so legacy segments load unchanged and enabling the option
goes through the standard incompatible-schema rebuild. The mutable
variant keeps an in-RAM ordered key set (not persisted), the immutable
variant builds a sorted key vector at load, and the on-disk variant
reads the dictionary lazily (block index resident, 1-2 block reads per
prefix lookup; reader is generic over UniversalRead).

Query side: prefix conditions are served from the dictionary when
available (filter + cardinality estimation from per-block aggregates),
from the forward index as per-point checks, and degrade to the payload
full-scan fallback otherwise - same execution model as other match
conditions. Strict mode (`unindexed_filtering_*`) rejects prefix
queries on fields without a prefix-enabled keyword index via a new
KeywordPrefix capability.

HNSW payload blocks: prefix-enabled indexes additionally emit prefix
blocks for heavy branching trie nodes (single-child chains collapsed to
their longest common prefix, one block per distinct point set, emitted
largest-first) so filtered search with prefix conditions gets navigable
subgraphs without rebuilding the same subset repeatedly.

API: `prefix` flag on KeywordIndexParams (REST bool, gRPC empty
message for extensibility), `prefix` variant in the Match oneof, edge
python bindings, regenerated OpenAPI spec.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Split prefix index into a dedicated module, fix clippy in tests

Reorganize the flat prefix_index.rs / prefix_read.rs into a
map_index/prefix_index/ module: format.rs (on-disk layout primitives),
writer.rs, reader.rs (PrefixIndex), map_read.rs (StrMapIndexPrefixRead
with per-variant impls) and tests.rs, with a file-format diagram and a
read-path walkthrough in the module docs. No logic changes.

Also fix clippy --all-targets complaints in test code: replace a
wildcard Match arm with an exhaustive list and a field-reassign-with-
default with a struct literal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add OpenAPI test for prefix match and snapshot file-tracking test

- tests/openapi/test_prefix_match.py: index-less fallback, prefix index
  creation with schema echo, scroll/count parity against ground truth,
  facet + prefix filter (the autocompletion flow), strict-mode rejection
  without the prefix capability.
- test_prefix_index_file_tracking: `prefix_index.bin` is listed in
  `files()` / `immutable_files()` exactly when built with the option.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Replace hand-rolled varint parsing with bytemuck Pod records

Per review: the prefix index format now uses fixed-size little-endian
Pod records (BlockEntry 24 B, KeyEntry 12 B, Header 40 B) written with
bytemuck::bytes_of and read back by copy via pod_read_unaligned — no
manual varint encode/decode, no alignment requirement, one shared
read_record helper. Costs ~9 bytes per key on disk versus LEB128; the
raw key bytes dominate dictionary size, so the simplification wins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fetch the whole candidate block range with a single storage read

Candidate key blocks of a prefix lookup are contiguous in the file, so
enumerate them from one ranged read instead of one read per block; the
over-read versus the exact key range is bounded by the two boundary
blocks. Block decoding is split into a storage-free helper reused by
the per-block path of stats estimation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Align prefix payload blocks with the geo index granularity principle

Geo's large_hashes emits only the smallest geohash regions above the
threshold — a disjoint antichain, never a parent nested with its
children. Prefix payload blocks now follow the same rule: a heavy
collapsed trie node is emitted only if nothing heavy is nested inside
it, counting both deeper qualifying prefixes and single heavy values
(which already get their own exact-match blocks). Emitted blocks are
therefore mutually disjoint and disjoint from exact-value blocks; no
near-collection-sized ancestor subgraphs, no reliance on the HNSW
connectivity check to skip nested duplicates.

Implemented as a `covered` flag propagated through the existing
LCP-interval scan, still one O(total key bytes) pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Document block wire format and unaligned-read rationale in decode_block

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

lib/edge/python/src/lib.rs

@@ -71,8 +71,8 @@ mod qdrant_edge {
     use super::types::filter::{
         PyFieldCondition, PyFilter, PyGeoBoundingBox, PyGeoPoint, PyGeoPolygon, PyGeoRadius,
         PyHasIdCondition, PyHasVectorCondition, PyIsEmptyCondition, PyIsNullCondition, PyMatchAny,
-        PyMatchExcept, PyMatchPhrase, PyMatchText, PyMatchTextAny, PyMatchValue, PyMinShould,
-        PyNestedCondition, PyRangeDateTime, PyRangeFloat, PyValuesCount,
+        PyMatchExcept, PyMatchPhrase, PyMatchPrefix, PyMatchText, PyMatchTextAny, PyMatchValue,
+        PyMinShould, PyNestedCondition, PyRangeDateTime, PyRangeFloat, PyValuesCount,
     };
     #[pymodule_export]
     use super::types::formula::{PyDecayKind, PyExpressionInterface, PyFormula};

lib/edge/python/src/types/filter/match.rs

@@ -24,6 +24,7 @@ impl FromPyObject<'_, '_> for PyMatch {
             Text(PyMatchText),
             TextAny(PyMatchTextAny),
             Phrase(PyMatchPhrase),
+            Prefix(PyMatchPrefix),
             Any(PyMatchAny),
             Except(PyMatchExcept),
         }
@@ -34,6 +35,7 @@ impl FromPyObject<'_, '_> for PyMatch {
                 Match::Text(_) => {}
                 Match::TextAny(_) => {}
                 Match::Phrase(_) => {}
+                Match::Prefix(_) => {}
                 Match::Any(_) => {}
                 Match::Except(_) => {}
             }
@@ -44,6 +46,7 @@ impl FromPyObject<'_, '_> for PyMatch {
             Helper::Text(text) => Match::Text(MatchText::from(text)),
             Helper::TextAny(text_any) => Match::TextAny(MatchTextAny::from(text_any)),
             Helper::Phrase(phrase) => Match::Phrase(MatchPhrase::from(phrase)),
+            Helper::Prefix(prefix) => Match::Prefix(MatchPrefix::from(prefix)),
             Helper::Any(any) => Match::Any(MatchAny::from(any)),
             Helper::Except(except) => Match::Except(MatchExcept::from(except)),
         };
@@ -63,6 +66,7 @@ impl<'py> IntoPyObject<'py> for PyMatch {
             Match::Text(text) => PyMatchText(text).into_bound_py_any(py),
             Match::TextAny(text_any) => PyMatchTextAny(text_any).into_bound_py_any(py),
             Match::Phrase(phrase) => PyMatchPhrase(phrase).into_bound_py_any(py),
+            Match::Prefix(prefix) => PyMatchPrefix(prefix).into_bound_py_any(py),
             Match::Any(any) => PyMatchAny(any).into_bound_py_any(py),
             Match::Except(except) => PyMatchExcept(except).into_bound_py_any(py),
         }
@@ -76,6 +80,7 @@ impl Repr for PyMatch {
             Match::Text(text) => PyMatchText::wrap_ref(text).fmt(f),
             Match::TextAny(text_any) => PyMatchTextAny::wrap_ref(text_any).fmt(f),
             Match::Phrase(phrase) => PyMatchPhrase::wrap_ref(phrase).fmt(f),
+            Match::Prefix(prefix) => PyMatchPrefix::wrap_ref(prefix).fmt(f),
             Match::Any(any) => PyMatchAny::wrap_ref(any).fmt(f),
             Match::Except(except) => PyMatchExcept::wrap_ref(except).fmt(f),
         }
@@ -267,6 +272,36 @@ impl PyMatchPhrase {
     }
 }
 
+#[pyclass(name = "MatchPrefix", from_py_object)]
+#[derive(Clone, Debug, Into, TransparentWrapper)]
+#[repr(transparent)]
+pub struct PyMatchPrefix(pub MatchPrefix);
+
+#[pyclass_repr]
+#[pymethods]
+impl PyMatchPrefix {
+    #[new]
+    pub fn new(prefix: String) -> Self {
+        Self(MatchPrefix { prefix })
+    }
+
+    #[getter]
+    pub fn prefix(&self) -> &str {
+        &self.0.prefix
+    }
+
+    pub fn __repr__(&self) -> String {
+        self.repr()
+    }
+}
+
+impl PyMatchPrefix {
+    fn _getters(self) {
+        // Every field should have a getter method
+        let MatchPrefix { prefix: _ } = self.0;
+    }
+}
+
 #[pyclass(name = "MatchAny", from_py_object)]
 #[derive(Clone, Debug, Into, TransparentWrapper)]
 #[repr(transparent)]

lib/edge/python/src/types/payload_schema/mod.rs

@@ -208,13 +208,19 @@ pub struct PyKeywordIndexParams(KeywordIndexParams);
 #[pymethods]
 impl PyKeywordIndexParams {
     #[new]
-    #[pyo3(signature = (is_tenant = None, on_disk = None, enable_hnsw = None))]
-    pub fn new(is_tenant: Option<bool>, on_disk: Option<bool>, enable_hnsw: Option<bool>) -> Self {
+    #[pyo3(signature = (is_tenant = None, on_disk = None, enable_hnsw = None, prefix = None))]
+    pub fn new(
+        is_tenant: Option<bool>,
+        on_disk: Option<bool>,
+        enable_hnsw: Option<bool>,
+        prefix: Option<bool>,
+    ) -> Self {
         Self(KeywordIndexParams {
             r#type: Default::default(),
             is_tenant,
             on_disk,
             enable_hnsw,
+            prefix,
         })
     }
 
@@ -232,6 +238,11 @@ impl PyKeywordIndexParams {
     pub fn enable_hnsw(&self) -> Option<bool> {
         self.0.enable_hnsw
     }
+
+    #[getter]
+    pub fn prefix(&self) -> Option<bool> {
+        self.0.prefix
+    }
 }
 
 impl PyKeywordIndexParams {
@@ -242,6 +253,7 @@ impl PyKeywordIndexParams {
             is_tenant: _,
             on_disk: _,
             enable_hnsw: _,
+            prefix: _,
         } = self.0;
     }
 }