mirror of
https://github.com/qdrant/qdrant.git
synced 2026-09-25 23:47:38 -05:00
Tim Visée
9f01222ccf
* Add `FlagsMode::from_feature_flags`, the mode for newly created flags Compact in serverless-compatible deployments, dynamic otherwise. Only creation consults it; opening existing flags detects their mode from disk. * Support the compact mode in the read-only flags types Add `ReadOnlyFlags`, the mode-dispatching union of the two read-only counterparts, serving the shared `RoaringFlagsRead` surface. Teach `InMemoryBitvecFlags` to detect the mode it opens; its compact `reload_appended` decodes the whole (small) file, as the format has no random access. * Create flags through mode selection in storages and indexes Vector storage deleted flags and the bool/null indexes now open through `open_or_create` with the mode from the feature flags: serverless deployments create compact flags, dedicated ones keep creating dynamic flags, and existing flags are opened in their detected mode either way. * Read flags in either mode in the read-only bool and null indexes `ReadOnlyFlags` shares the `RoaringFlagsRead` surface and the lifecycle signatures of the roaring type it replaces, so the swap is a type rename. * Add TODO to not lock bitmask structure during flush * `MutableStoredBitmask::save` returns the number of bytes written Zero when the skip-clean save wrote nothing. Lets wrappers charge the actual write to a hardware counter. * Refuse to open compact flags in a dynamic-mode directory Creating the compact file next to dynamic files would leave a directory of both modes behind, which every later open rejects — refuse up front instead. Both production callers already rule the case out through `FlagsMode::detect`, so this only removes a foot-gun for future callers. The open-or-eagerly-create logic moves into `open_or_create_compact_mask`, shared with the update-only writer next. * Rewrite `UpdateOnlyStoredFlags` onto the compact bitmask The update-only flags writer now writes the compact mode — a single roaring-encoded `compact_flags.dat` through `MutableStoredBitmask` — instead of rewriting the whole padded dynamic file pair every batch. A flush with no effective changes now writes nothing at all, where the old writer rewrote the full mask on any `set`. This also fixes opening serverless-created segments: the old open eagerly wrote a `status.dat` into directories the writable side had created in the compact mode, leaving files of both modes behind and poisoning the directory for every later open. A directory already holding dynamic-mode flags is refused loudly rather than kept current or migrated; rebuild the segment to migrate its flags. Migration may come later. Drops the now-dead `InMemoryBitvecFlags::into_bitvec` and `DynamicFlagsStatus::new`, and demotes `file_size_for` to private. * Run edge tests with serverless feature flags The edge fixtures ran with default feature flags, building leader shards with dynamic-mode flags — a configuration edge never serves in production, and one the update-only flags writer now refuses. It also hid that the writer poisoned compact directories: no test exercised update-only writes over a serverless-created shard. Feature flags are process-global and first-init-wins, so every fixture in the binary initializes the same serverless set; the manifest test folds into it, since serverless implies `write_segment_manifest`. * Don't use sequencial mode for one shot reads
This dir contains a Justfile with recipes to build/check/run examples for edge packages. Install https://github.com/casey/just to use it.
Rust Qdrant Edge package
Rust Qdrant Edge workspace lives in the publish directory. It's a separate workspace, not tied to the main workspace.
The qdrant-edge package is autogenerated by the amalgamate.py script and placed in the ./publish/qdrant-edge directory (gitignored).
If you need to make changes in the qdrant-edge package, edit the original packages in this repo (/lib in the repo root).
just rs-examples
# Or, manually:
./amalgamate.py
cargo check -p examples
cargo run -p examples --bin demo
cargo run -p examples --bin …
(see full binary list in publish/examples/src/bin)
How to publish Rust Package to crates.io
- Update the
VERSIONin publish/amalgamate.py. - Run Qdrant Edge Rust Release workflow.
Python Qdrant Edge package
It's in the python directory. It's part of the main workspace, unlike the Rust package.
just py-build
just py-examples
Or, manually:
# Setup environment
cd lib/edge/python
python -m venv .venv
source .venv/bin/activate
pip install --user maturin
# Build and install the package:
cd lib/edge/python
maturin develop --no-default-features
# Run example:
python examples/demo.py