mirror of
https://github.com/qdrant/qdrant.git
synced 2026-07-26 20:51:08 -05:00
9977421331
* perf(edge): load ReadOnlyEdgeShard segments in parallel Open each segment on a dedicated thread during initial open and refresh, reducing follower startup time for shards with many segments. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(edge): resolve clippy type_complexity in parallel segment load Co-authored-by: Cursor <cursoragent@cursor.com> * perf(edge): run per-segment reads on a configurable thread pool Replace the per-segment sequential read loops and the spawn-a-thread-per-segment loader with a single fixed-size rayon thread pool owned by each shard. - Add EdgeConfig::max_search_threads (Option<usize>, None = CPU-derived default matching the core search runtime via common::defaults::search_thread_count). - Build a long-lived pool in EdgeShard and ReadOnlyEdgeShard; reuse it for parallel segment loading on open/refresh instead of std::thread::spawn. - Add EdgeReadView::par_map_segments as the single seam that runs per-segment work on the pool; use it in search, scroll, count, facet and rescore-formula. Each task mints its own HardwareCounterCell from the shared accumulator. - Expose max_search_threads through the builder and the Python binding (+ stub). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(edge): return error instead of panicking on search pool creation ThreadPoolBuilder::build() can fail (e.g. thread spawn / resource exhaustion). This runs during EdgeShard open/load and ReadOnlyEdgeShard follower open, so a transient failure must not abort the process. Propagate it as an OperationError through the existing OperationResult-returning constructors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This dir contains a Justfile with recipes to build/check/run examples for edge packages. Install https://github.com/casey/just to use it.
Rust Qdrant Edge package
Rust Qdrant Edge workspace lives in the publish directory. It's a separate workspace, not tied to the main workspace.
The qdrant-edge package is autogenerated by the amalgamate.py script and placed in the ./publish/qdrant-edge directory (gitignored).
If you need to make changes in the qdrant-edge package, edit the original packages in this repo (/lib in the repo root).
just rs-examples
# Or, manually:
./amalgamate.py
cargo check -p examples
cargo run -p examples --bin demo
cargo run -p examples --bin …
(see full binary list in publish/examples/src/bin)
How to publish Rust Package to crates.io
- Update the
VERSIONin publish/amalgamate.py. - Run Qdrant Edge Rust Release workflow.
Python Qdrant Edge package
It's in the python directory. It's part of the main workspace, unlike the Rust package.
just py-build
just py-examples
Or, manually:
# Setup environment
cd lib/edge/python
python -m venv .venv
source .venv/bin/activate
pip install --user maturin
# Build and install the package:
cd lib/edge/python
maturin develop --no-default-features
# Run example:
python examples/demo.py