fe5a063dc2
* Add UpdateOnlyAppendableIdTracker, the append-only writer The write counterpart of `ReadOnlyAppendableIdTracker`, producing the two files that tracker already consumes — `mutable_id_tracker.mappings`, an append-only log of mapping changes, and `mutable_id_tracker.versions`, a dense array of one version per slot — through `UniversalAppend`, so the same code drives a local file and an object store. `insert_operations` records a batch of `MappingOperation`s in order: an insert claims the next slot above the highest one in use and reports it, a delete retires an external id and claims nothing. Nothing is rewritten in place, so re-inserting a live id moves it to a fresh slot and supersedes the old one — the update-only shape of an update. `set_internal_versions` extends the versions array. Ids may come in any order but must be exactly the slots the array does not cover yet: a slot below the end would need an in-place overwrite, and a hole would have to be zero-filled — and since "covered by the versions file" *is* the commit signal for readers, that would publish a slot as a live point of version 0 before its data exists. Both are rejected rather than written. Both methods append at an offset they probed for, never at an implicit end: the offset is a compare-and-swap token, so a file that has moved on since the probe is rejected instead of being written twice or in the wrong place. Both have persisted what they wrote when they return `Ok` — append, then run the handle's flusher — and nothing is buffered across calls. The order of the two calls, claim the slot then commit the version, is what makes a crash in between safe: readers ignore slots the versions array does not cover. Cleaning up the slots such a crash abandons is left to the opener, along with repairing a torn tail; the writer fails loudly rather than guessing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Share the versions file format between both write paths `set_internal_versions` was reimplementing what `versions_storage` already knew: entries are `SeqNumberType`-sized, slot `n` lives at `n * VERSION_ELEMENT_SIZE`, a file length that is not a whole number of entries has a torn tail. Every one of those facts existed in two to four places, spelled as inline `/`, `%` and `write_u64::<FileEndianess>`, so the append-only writer could drift away from the in-place one silently. Move them into `versions_storage`, which now owns the format for both writers and both readers: - `write_version` / `read_version`, the entry codec, with a static assertion tying its `u64` to `VERSION_ELEMENT_SIZE` so a change to `SeqNumberType` cannot silently shrink every offset; - `version_offset` and `versions_byte_len`, the slot arithmetic; - `VersionsLayout`, which splits a file length into committed entries and a partial tail. The two writers still react differently — the in-place one truncates the tail, the append-only one refuses it, because an append cannot — but they no longer each work out what the tail is. `store_version_changes`, `load_versions`, `set_internal_versions` and the read-only tracker's live reload all go through it. The write loops themselves stay separate: one seeks to sparse offsets, the other emits a validated consecutive run, and merging them would obscure both. What they share is where the bytes go, which is the part that must not diverge — and a new test pins it down by writing the same versions through both writers and comparing the files byte for byte. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Append mapping and version entries as batches Both writers built one concatenated buffer and handed it to `append`. `append_batch` takes the entries as separate buffers and places them in a single operation — a vectored write locally, one request on an object store — so the entry boundaries reach the backend instead of being flattened away first. Versions are fixed-size, so the entries are the payload's `chunks_exact`. Mapping changes are variable-length, so their bounds are recorded as they are serialized. Both keep the compare-and-swap offset, which `append_batch` validates the same way `append` does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Test the writer against MutableIdTracker end to end The suite leaned on round-trips through the storage loaders, which only restated what the writer had just written. Four such tests are replaced by one that checks the property actually worth having: drive the append-only writer and `MutableIdTracker` with the same points, versions and deletes, open each segment through `ReadOnlyAppendableIdTracker`, and require the two views to be indistinguishable — counts, deleted state, external ids, live points' versions, and id resolution. Versions are compared for live points only. `MutableIdTracker::drop` overwrites the slot with `DELETED_POINT_VERSION`, which an append cannot do, so the append-only writer leaves the point's original version there; neither is observable for a point that is gone. The remaining tests keep what a round-trip cannot show: slot allocation across calls, instances and deletes (three tests folded into one), the rejection of holes and rewrites, and the byte-for-byte agreement of the two writers on the versions file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * better buffer names * fmt * Heal a torn versions tail instead of refusing to write A writer that dies mid-entry leaves the versions file ending inside a slot. The in-place writer already truncated that tail before writing; the append-only writer refused, which left the file unwritable forever since an append cannot truncate. Share the decision — what counts as torn, the healthy length, the warning — in `heal_versions_tail`, and let each writer supply the shrink its backend can do: `set_len` in place, or reading the committed prefix back and putting it in place as a whole file where there is no truncate. Dropping the tail loses nothing: the array covers a slot only once its whole entry is there, so a partial entry belongs to a slot no reader ever saw and no writer counted as committed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Append mappings at the end of the log, not the end of the file Mapping entries vary in length, so no length tells you whether the log ends on an entry boundary. Appending at the file's end therefore could not fail: a torn entry was silently appended after, and every entry from there on was framed off the stray bytes. Carry the boundary instead. `new` takes the offset just past the last complete entry — `ReadOnlyAppendableIdTracker::mappings_read_to`, from the same view that supplies `max_internal_id` — and appends there, which turns a file ending elsewhere into an append offset conflict. On that conflict `heal_mappings` cuts the file back to the log's end, the same read-prefix-and-rewrite the versions file heals with, and writes the batch again. A torn entry and a batch that landed unacknowledged are indistinguishable without parsing, and need not be told apart: neither `max_internal_id` nor `mappings_end` moves before an append is durable, so the retry writes the same bytes at the same offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Move the healing functions into their own file `update_only/mod.rs` had grown to hold the writer, its two public write paths and the two repair routines they fall back on. Split the latter out: `heal_versions` and `heal_mappings` move verbatim into `update_only/heal.rs`, as a second impl block, following the layout the read-only half already uses (`lifecycle.rs`, `live_reload.rs`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Retire inherited pending inserts when opening the writer A slot is spoken for from the moment the mappings log claims it — components may already have written data at it — so the writer must resume above every slot the log ever handed out. Deriving that bound from a reader's point set undercounts it twice over: a claimed slot whose version was never committed is not in the mapping, and one whose external id was deleted afterwards is not among the pending inserts either. Track it in the log instead, as `ReadOnlyAppendableIdTracker::max_claimed_internal_id`, bumped on every insert entry regardless of what becomes of the point, and take it as `UpdateOnlyAppendableIdTracker::new`'s bound. The points on those claimed-but-unversioned slots are the other half. They cannot be adopted: a writer stopped partway through them, so some components hold their data and others do not, and which is unknowable here. They cannot be left alone either, the versions array being dense — covering any slot above one of them publishes it, half-written. So `new` now takes the pending inserts explicitly and retires them, recording a `Delete` per id before the writer can be used at all, which is what makes it fallible. Doing it at construction rather than lazily on the first write means no write path can be added later that forgets to. `set_internal_versions` accordingly stops rejecting holes: it writes the whole run from the end of the array through the highest id given, covering skipped slots with `DELETED_POINT_VERSION` as the in-place writer's seek already does. It gains an upper bound in exchange — publishing a slot means covering every slot below it, so an id the log never claimed is refused. Live-reload semantics are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Skip UpdateOnlyAppendableIdTracker heal tests on Windows Healing replaces the file via atomic_save while an mmap handle is still open, which Windows denies with os error 5. * Drop mmap handles before healing ID tracker files atomic_save cannot replace a path while an mmap is still open on Windows. Copy the committed prefix, drop the handle, then rewrite and reopen. * Trim ID tracker docs and drop redundant helpers Condense the doc comments on the update-only tracker to the style of the sibling modules, merge a duplicate impl block, and remove `read_version` and a debug assert that restates its own operands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Drop the versions layout helpers and inline the arithmetic VersionsLayout, versions_byte_len and version_offset wrapped one divmod and one multiplication between them, and adopting the struct made the live-reload hunk longer than the line it replaced. Compute the committed length where it is needed instead, and let heal_versions_tail and heal_versions return unit, since no caller used the layout they handed back. Also drop the writer's unused max_claimed_internal_id accessor, and fold retires_inherited_pending_inserts_at_construction into retires_inherited_pending_inserts: the merged test asserts the retirement happened before the writer did anything, and commits a real version to the retired slot rather than letting it take the filler, so it still shows the Delete is what hides the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: qdrant-cloud-bot <111755117+qdrant-cloud-bot@users.noreply.github.com>
Vector Search Engine for the next generation of AI applications
Qdrant (read: quadrant) is a vector similarity search engine and vector database. It provides a production-ready service with a convenient API to store, search, and manage points—vectors with an additional payload. Qdrant is tailored for extended filtering support, making it useful for all sorts of neural-network or semantic-based matching, faceted search, and other applications.
Qdrant is written in Rust 🦀, which makes it fast and reliable even under high load. See benchmarks.
With Qdrant, embeddings or neural network encoders can be turned into full-fledged applications for matching, searching, recommending, and much more!
Qdrant is also available as a fully managed Qdrant Cloud ⛅ including a free tier.
Quick Start • Agent Skills • Client Libraries • Demo Projects • Integrations • Contact
Getting Started
Agent Skills
Qdrant provides a collection of ready-to-use agent skills that bring Qdrant's vector search capabilities directly into your AI coding assistant. Install these skills to empower your agent in making critical engineering decisions for optimal vector search performance, such as quantization, sharding, tenant isolation, hybrid search, model migration, and more.
Client-Server
To experience the full power of Qdrant locally, run the container with this command:
docker run -p 6333:6333 qdrant/qdrant
Note that this starts an insecure deployment without authentication, open to all network interfaces. Please refer to secure your instance.
Now you can connect to the server with any client. For example, using Python:
from qdrant_client import QdrantClient
client = QdrantClient(url="http://localhost:6333")
Before deploying Qdrant to production, be sure to read our installation and security guides.
Clients
Qdrant offers the following client libraries to help you integrate it into your application stack:
- Official:
- Community:
Qdrant Edge
Qdrant Edge is a lightweight version of Qdrant designed for edge devices and resource-constrained environments. Unlike Qdrant Server, which uses a client-server architecture, Qdrant Edge runs inside the application process. Data is stored and queried locally and can be synchronized with a Qdrant server. It offers the same powerful vector search capabilities as the client-server version but with a smaller footprint, making it ideal for applications that require low latency and offline functionality.
To get started with Qdrant Edge from Python or Rust, initialize an instance of EdgeShard, which exposes methods to manage data, query it, and restore snapshots. For example:
from qdrant_edge import Distance, EdgeConfig, EdgeVectorParams, EdgeShard, Point, UpdateOperation
shard = EdgeShard.create("./shard", EdgeConfig(
vectors={"my-vector": EdgeVectorParams(size=4, distance=Distance.Cosine)}
))
shard.update(UpdateOperation.upsert_points([
Point(id=1, vector={"my-vector": [0.1, 0.2, 0.3, 0.4]}, payload={"color": "red"})
]))
Where Do I Go from Here?
- Quick Start Guide
- Detailed Documentation
- Take the Qdrant Essentials course
- Follow this tutorial to create a semantic search engine with Qdrant
Demo Projects
Discover Semantic Text Search 🔍
Unlock the power of semantic embeddings with Qdrant, transcending keyword-based search to find meaningful connections in short texts. Deploy a neural search in minutes using a pre-trained neural network, and experience the future of text search. Try it online!
Explore Similar Image Search - Food Discovery 🍕
There's more to discovery than text search, especially when it comes to food. People often choose meals based on appearance rather than descriptions and ingredients. Let Qdrant help your users find their next delicious meal using visual search, even if they don't know the dish's name. Check it out!
Master Extreme Classification - E-Commerce Product Categorization 📺
Enter the cutting-edge realm of extreme classification, an emerging machine learning field tackling multi-class and multi-label problems with millions of labels. Harness the potential of similarity learning models, and see how a pre-trained transformer model and Qdrant can revolutionize e-commerce product categorization. Play with it online!
API
REST
Qdrant provides a REST API with an OpenAPI 3.0 specification, enabling client generation for virtually any framework or programming language.
You can also download the raw OpenAPI definitions.
gRPC
For faster, production-tier searches, Qdrant also provides a gRPC interface.
Features
Dense, Sparse, and Multi Vector Search
Qdrant supports dense vectors for semantic similarity, sparse vectors for full-text search, and multivector search for objects with multiple embeddings or late interaction models like ColBERT.
Filtering on Payload
Attach any JSON payload to your vectors and filter on it using a rich set of conditions—keyword matching, full-text, numeric ranges, geo-locations, and more—combined with should, must, and must_not clauses.
Hybrid Search
Combine multiple vectors in a single query to get the best of semantic understanding and keyword precision, with results merged via configurable fusion strategies, such as Reciprocal Rank Fusion (RRF) and Distribution-Based Score Fusion (DBSF).
Vector Quantization and On-Disk Storage
Built-in quantization cuts RAM usage by up to 97% and lets you tune the trade-off between search speed and precision.
Distributed Deployment
Scale horizontally with sharding and replication, and update or resize collections with zero downtime.
Highlighted Features
- Faceting - aggregate search results by payload values.
- Recommendation - use positive and negative examples to find similar points.
- Discovery - constrain search to a specific region of the vector space.
- Search Relevance Tuning - tools for adjusting search results, such as Maximal Marginal Relevance (MMR) and the Relevance Feedback Query.
- Multitenancy - scalable partitioning of data for multi-user environments.
- Observability - comprehensive metrics, telemetry, and audit logging for monitoring and debugging.
- Query Planning and Payload Indexes - leverages stored payload information to optimize query execution strategy.
- SIMD Hardware Acceleration - utilizes modern CPU x86-x64 and Neon architectures to deliver better performance.
- GPU Support - for accelerated indexing, with support for NVIDIA and AMD GPUs.
- Async I/O - uses
io_uringto maximize disk throughput utilization even on network-attached storage. - Write-Ahead Logging - ensures data persistence with update confirmation, even during power outages.
Web UI
Web UI provides a visual way to interact with your data and monitor the health of your deployment. It enables you to explore your collections, manage data, interact with the REST API, and more.
Integrations
Qdrant integrates with the tools you're already using across every stage of your AI stack. You can connect to embedding providers, AI application frameworks, and data pipeline tools, as well as observability platforms for monitoring and tracing your vector search in production. No-code and low-code automation platforms are supported too. Refer to the Ecosystem page for the complete list.
Contributing
We are happy to receive your contributions! Before opening a pull request, please read our Contributing Guide.
Important
Our development branch is
dev, notmaster. Please fork the repo, branch fromdev, and open your pull request againstdev. PRs targetingmasterwill be asked to retarget.
Contacts
- Have questions? Join our Discord channel or mention @qdrant_engine on X
- Want to stay in touch with the latest releases? Subscribe to our Newsletters
- Looking for a managed cloud? Check pricing. Need something personalized? We're at info@qdrant.tech
License
Qdrant is licensed under the Apache License, Version 2.0. View a copy of the License file.