mirror of
https://github.com/qdrant/qdrant.git
synced 2026-08-01 07:30:54 -05:00
5e109bbecc
* feat: add io_bridge * fix: cached dispatcher * feat: add open_with_handle * fix: naming * fix: wording * fix: rebase io_bridge onto split BorrowedReadPipeline/OwnedReadPipeline * fix: linter * feat: add S3 backend * chore: remove io_bridge * fix: linter * fix: linter * fix: read handle * chore: simplify S3Source * fix: s3 test * feat: support multi runtime * fix: clippy errors * fix: review comments * feat: add io design * feat: add S3 backend * chore: fix docs * fix: dev changes * chore: add some docs * chore: remove explicit type * feat: add new methods * [WIP] review refactor * fmt * fix: bytes alignment * fix: linter * feat: remove Bytes * fix: ci/cd * fix: tests * fix: tests * refactor: simplify io_bridge pipeline to Handle-based dispatch Replace the BridgeRuntime worker thread + request channel + boxed BridgeRequest with direct tokio Handle usage: - BridgeRuntime is now just an Arc<Runtime>; schedule() spawns the read future via Handle::spawn instead of routing it through a dispatcher thread. Removes BridgeRequest and the now-unreachable S3RuntimeShutDown error variant. - Guard against a panicking read task hanging wait() forever: the spawned task catches unwinds and converts them into a TaskPanicked error reply, so every scheduled slot is always answered. - Encapsulate slot bookkeeping in PendingSlots, exposing only the needed operations instead of a public map + counter. - Split the grown pipeline.rs into a pipeline/ module (slots / inner / borrowed / owned), de-duplicating the shared read-future construction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: move pipeline buffer ownership into the read future Instead of the pipeline owning the destination Vec<T> in a slot map and the future writing through a SendBytePtr raw pointer, let the future allocate the buffer itself and return it through BridgeResponse. The buffer crosses the worker-thread boundary as a normal move via the reply channel, wrapped in a SendableVec<T> newtype that asserts Send for T: bytemuck::Pod only. This removes the entire unsafe SendBytePtr apparatus from the pipeline: no raw pointer, no unsafe fn, no per-call-site unsafe blocks, no heap-stability invariants. The only remaining unsafe in the crate is one bounded `unsafe impl<T: Pod> Send for SendableVec<T>` with a trivially true invariant (Pod types are plain bytes). PendingSlots collapses back to PendingSlots<U>: slots no longer carry buffers. AlignedBufWriter::from_raw_bytes (used only by the SendBytePtr path) and its test are removed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: drop SendableVec wrapper now that Item: Send UniversalRead's element type is now bound to `Item` (`Pod + Send`), so the io_bridge pipeline no longer needs a hand-rolled `Send` wrapper around `Vec<T>` to ship buffers through the reply channel. Replace `SendableVec<T>` with `Vec<T>` end-to-end and tighten the local impls to `T: Item`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: implement UniversalRead::reopen for BlobFile BlobFile has no cached file metadata or mapping — `len()` queries the object store fresh on each call — so reopen is a no-op, matching the io_uring impl. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add new fs impl for Blob * fix: is_in_ram_or_mmap for S3 --------- Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collection
Crate, which implements all functions required for operations with a single collection of points. Points within a collection should share the same payload schema and have same vector size. So that search requests could be performed over all points of a single collection.
Structure
Update process
