* IoUringState: generic `RequestId`
* UniversalRead: generic `RequestId`
* Simplify `gridstore::Pages::get_page_value_ranges`
Now we don't need two separate `SmallVec`s as we can put `buffer_offset`
into `RequestId`.
* Better doc comment
* Rename `RequestId` -> `Meta`
* Skip broken tests
* Add rocksdb dropper
This is a temporary tool. It will be removed later in this PR.
* [automated] Drop rocksdb
This commit is made by running tools/rocksdb/drop.sh
* Touch-up after ast-grep
The previous automated commit removed items, but not their comments.
Also, some blocks are left with only one item.
Also, ast-grep can't handle macros like `vec![]`.
This commit completes the job.
* Remove RocksDB dropper
* Remove mentions of rocksdb feature in CI
* Fix clippy warnings
These `FIXME` comments added previously in this PR by ast-grep by
"peeling" cfg_attr like this:
-#[cfg_attr(not(feature = "rocksdb"), expect(...))]
+#[expect(...)] // FIXME(rocksdb): ...
This commit removes these allow/expect attributes and fixes clippy
lints.
* Remove leftover rocksdb-related code
Removed:
- Cargo.toml: rocksdb cargo feature flag and dependencies.
- code: rocksdb-related items that was not under feature flag.
- flags.rs: rocksdb-related qdrant feature flags.
Disabled:
- Some benchmarks because these do not compile now.
* Print warning if rocksdb leftovers found in snapshots
* Remove Clone from tokenizer
* Regenerate openapi.json
* [AI] Implement low-level iterator for universal-io implementation of io_uring
* [AI] io_uring pool
* fmt
* [AI] refactor io_uring into multiple files
* [AI] implement IoUringReadMultiIter
* [AI] replace file_indices Vec -> AHashMap to prevent memory growth
* fmt
* Guard populate() when file uses O_DIRECT
O_DIRECT bypasses the page cache, so reading the whole file to warm
it is pointless — return early as a no-op.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Update comment about O_DIRECT in concurrent read iter test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: apply submit_and_wait retry loop to read iterators
On older kernels, submit_and_wait may return before completions are
available. Apply the same retry loop to the iterator step() methods
to prevent premature iterator termination.
Made-with: Cursor
* do not step every time
* [AI] unify iterators
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
Loop `submit_and_wait(1)` until the completion queue is non-empty in
`UniversalRead::read` and `UniversalWrite::write`. On some earlier
kernel versions, `submit_and_wait` may return before the requested
number of events have completed, which would cause the subsequent
`.expect()` to panic.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
* Remove UniversalRead::is_empty
Reason: Less methods to override in wrappers. And I don't think this
method makes much sense.
* Move common::universal_io::{ => wrappers}::read_only
* Update ReadOnly wrapper
Implement missing methods, and follow the code style of the upcoming
TypedStrorage wrapper.
* Add TypedStorage
* Use TypedStorage
* Add reminder comments
* Clarify TypedStorage use-case
* Add `#[must_use]` to RAII guard and task handle types
These types rely on being held for a scope — dropping them immediately is
almost always a bug:
- `StoppingGuard`: sets `is_stopped` flag on drop
- `UpdateGuard`: decrements update counter on drop
- `UpdatesGuard`: releases mutex preventing concurrent updates on drop
- `ClockGuard`: marks clock as inactive on drop
- `IsAliveGuard`: releases liveness lock on drop
- `ScopeTrackerGuard`: decrements scope counter on drop
- `CancellableAsyncTaskHandle`: detaches task on drop
- `StoppableTaskHandle`: may abort task on drop (AbortOnDropHandle)
Made-with: Cursor
* Remove redundant function-level #[must_use] now covered by type
With ScopeTrackerGuard marked #[must_use] at the type level, the
function-level attributes on measure_scope(), measure(),
track_create_snapshot_request(), and count_snapshot_creation() are
redundant and trigger clippy::double_must_use.
Made-with: Cursor
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
* UIO traits: require Sized
I don't think we ever going to add unsized implementation. So, lets,
require it on the whole trait rather than on individual methods.
* Move `AccessPattern` to the `common` package
Was: segment::vector_storage::vector_storage_base::AccessPattern
Now: common::generic_consts::AccessPattern
* UIO traits: replace `bool` with `AccessPattern`
* feat(universal_io): add storage-agnostic BitSliceStorage with read/write support`
Add BitSliceStorage<S> generic over
UniversalRead<u64>/UniversalWrite<u64>,
providing bit-level read and write operations over u64-element storage.
Read operations (S: UniversalRead<u64>):
- read_all: returns Cow<BitSlice> (zero-copy for mmap, owned for others)
- get_bit: single bit read via u64 element fetch
- read_bit_range: arbitrary bit range read
Write operations (S: UniversalWrite<u64>):
- set_bit / replace_bit: single bit write (skips write if unchanged)
- write_bit_range: arbitrary bit range write from BitSlice source
- fill_bit_range: fill range with a value
- set_bits_batch: batch individual bit updates coalesced by element
- flusher: flush underlying storage
* refactor: replace MmapBitSlice with BitSliceStorage in MmapBitSliceBufferedUpdateWrapper
Migrate all deleted-flag bitslice storage from the legacy MmapBitSlice
(Deref-based mmap wrapper) to BitSliceStorage<MmapUniversal<u64>>
(storage-agnostic universal IO backend).
Updated consumers:
- MmapMapIndex
- MmapNumericIndex
- MmapGeoMapIndex
- MmapInvertedIndex (fulltext)
- ImmutableIdTracker
- Benchmark
Additionally:
- Add BitSliceStorage::create(path, num_bits) to encapsulate
file creation + sizing + open (eliminates duplicated size math
across 5 call sites)
- Add MmapBitSliceStorage type alias for
BitSliceStorage<MmapUniversal<u64>>
- Add count_ones() convenience method
- Use set_bits_batch() in wrapper flusher and all build paths
instead of per-bit set_bit() loops (coalesces u64 read-modify-writes)
- Fix silent error swallowing in wrapper get(): log + debug_assert
on I/O errors instead of .ok().flatten()
- Remove dead bitmap_mmap_size function from immutable_id_tracker
* use bitvec's approach for offset
* less api
* misc improvements
* refactor write method
* move to segment crate
* handle creation of file outside of StoredBitSlice logic
* fix codespell
* document bitwise calculations
* coalesce more updates into a single write, batch all writes
* use native `extend_from_bitslice` instead of iterating.
* clarity refactor
* clippyyy
* use `u64` as BitStore everywhere
* assume iterator is sorted
* only use chunk_by for generating runs
* fix update wrapper flusher
* review fixes
* larger set bits batch test
* remove reintroduced file
* oops, fix new test
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* refactor(edge): split EdgeShard load into new() and load()
- new(path, config): create edge shard only when path has no existing segments
- load(path, config?): load from existing files; config optional (load from
edge_config.json or infer from segments)
- Helpers: has_existing_segments, ensure_dirs_and_open_wal, resolve_initial_config,
load_segments, ensure_appendable_segment
- Python: EdgeShard.create_new(path, config); __init__ still calls load()
- Examples use new() for creation and load(path, None) for reopen
- Error instead of panic on config mismatch; explicit load/create in Python
- fix: use OptimizerThresholds.deferred_internal_id for dev compatibility
Made-with: Cursor
* rollback threshold changes
* fix(edge): address dancixx review comments
- Persist inferred config in load() when edge_config.json does not exist
(SaveOnDisk::new only saves when file exists)
- Python: add #[new] delegating to load() for backward compatibility
- qdrant_edge.pyi: Optional return types for deleted_threshold,
vacuum_min_vector_number, default_segment_number; add __init__ doc
Made-with: Cursor
* Revert "fix(edge): address dancixx review comments"
This reverts commit 370e21a6c74c41210e1658f89814395cb86ed81c.
* fix: optional returns
* fix: save config it is not exist
* fix: save data on disk always
* fix: python examples
* chore: remove edge.close()
* fix: wal lock
* fix: rename of EdgeShardConfig -> EdgeConfig
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: Daniel Boros <dancixx@gmail.com>
* Log individual payload index load time per field
Add per-field timing to `load_all_fields()` in `StructPayloadIndex`,
using the existing `LOAD_TIMING_LOG_TARGET` infrastructure. Each
payload field index that takes >= 5ms to load now gets its own log
line, making it easy to identify slow-loading fields.
Made-with: Cursor
* Move `log_load_timing` to common module and use everywhere
Move the `log_load_timing` helper from `segment_constructor_base` into
`common::defaults` so it can be reused across crates. Convert all raw
`log::debug!(target: LOAD_TIMING_LOG_TARGET, ...)` call sites to use
the shared function, giving consistent formatting and min-duration
suppression everywhere.
Made-with: Cursor
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Add a `log_load_timing` helper that checks elapsed time against a 5ms
threshold before logging. Sub-component loads faster than this round to
"0.00s" in the {:.2} format and are pure noise. The per-segment "total
loaded" line remains unconditional.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
* Improvements for `IoUringFile`
- handle read and write requests more explicitly
- assert against partial reads/writes
* Implement `read_multi`/`write_multi`
* Implement `populate` and `clear_ram_cache`
* Rename internal structures as `IoUringSomething`
* [ai] Make `UniversalRead`/`UniversalWrite` impl generic over `T`
* Handle `io_uring` initialization error
* Check that `io_uring` is initialized and supported, when opening `IoUringFile`
* fixup! Check that `io_uring` is initialized and supported, when opening `IoUringFile`
Fix typo
* test for reading u64 from file with uring
* fmt
* clippy
* fix(io_uring): allocate Vec<MaybeUninit<T>> for reads to fix alignment for T (#8353)
* fix(io_uring): allocate Vec<MaybeUninit<T>> for reads to fix alignment for T
Refactor IoUringState::read to take generic T and item_offset/items_length
instead of byte_offset/byte_length. Allocate Vec<MaybeUninit<T>> so the
kernel writes into correctly aligned memory, then convert to Vec<T> in
finalize. Fixes bytemuck::cast_vec alignment panic for types like u64.
Made-with: Cursor
* refactor(io_uring): only make read/write methods generic, not state or runtime
- IoUringState and IoUringRuntime no longer generic over T
- read<T>() allocates Vec<MaybeUninit<T>>, transmutes to Vec<MaybeUninit<u8>> for storage
- finalize returns ReadBuffer; callers use .into_vec::<T>() to get Vec<T>
- Write path unchanged (callers pass bytes via bytemuck::cast_slice)
Made-with: Cursor
* Cleanup
* review n1
* review n2
* fmt
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* use AHash instead of just hash
* add ahash to deps
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
Co-authored-by: qdrant-cloud-bot <111755117+qdrant-cloud-bot@users.noreply.github.com>
Co-authored-by: Cursor Agent <agent@cursor.com>
* Add debug-level load timing logs for storage components
Add log::debug! calls with target "qdrant::load_timing" that report
how long each storage component takes to load during startup:
- Total shard load time (including WAL replay)
- Total segment load time + load_state
- Per-segment: payload_storage, id_tracker, payload_index
- Per-vector: vector_storage (dense/sparse), quantized_vectors,
vector_index (dense/sparse)
These logs are at DEBUG level so they are silent by default. Enable
selectively via log_level config or env, for example:
log_level: "INFO,qdrant::load_timing=debug"
No config fields, no global flags, no function signature changes.
Made-with: Cursor
* Use seconds with 10ms resolution for load timing logs, extract log target constant
Switch from `{:.2}ms` with `as_secs_f64() * 1000.0` to `{:.2}s` with
`as_secs_f64()` for cleaner output. Move the "qdrant::load_timing"
log target string to a LOAD_TIMING_LOG_TARGET constant in each file.
Made-with: Cursor
* Move LOAD_TIMING_LOG_TARGET to common::defaults
Single definition shared by segment and collection crates, avoiding
duplication of the log target string.
Made-with: Cursor
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
* Add ignore_deferred flag for internal scroll API
* Ignore deferred points in ProxyShard::update
* Use enum instead of bool as type in function parameters
* Use consistent parameter name
* fmt
* Apply suggestions from code review
Co-authored-by: Tim Visée <tim+github@visee.me>
* More explicit naming
* Add DeferredBehavior to `count()` and disable filtering in stream_records
* Don't filter `retrieve()` everywhere
* Use consistent filter behavior in read operations
* Add TODO for ignored parameter in remote_shard
* Rebase fixes
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
* restrict recovery from snapshot folder
* fmt
* clippy
* Extend test, path must be inside snapshots directory
* In release builds, hide detailed tar error message to not leak contents
* Change from bad request to forbidden
* fix(auth_tests): place snapshot in peer snapshots dir for recover_collection_snapshot
The recover from snapshot API now requires file:// paths to be inside the
configured snapshots directory. Write the test snapshot into the peer's
snapshots directory instead of a temp file so the request is allowed.
Made-with: Cursor
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Cursor Agent <agent@cursor.com>
* refactor PointToValues to use UniversalRead
It includes a significant change to `MmapValue` trait to be able to
handle Cow reads, instead of just references.
* fix str parsing
* fix incorrect path
* use fallible casting
* clippy
* No eager allocation
this also makes it so that borrowed strs can keep happening :heart-eyes:
* Lifetime cleanup
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* feat(universal_io): add MultiUniversalRead trait and VecMultiUniversalRead impl
- Add SourceId, MultiUniversalRead<T> trait with read_batch_multi, source_len,
populate, clear_ram_cache (latter two with default no-op).
- Add UniversalIoError::InvalidSourceId for invalid source id in batch reads.
- Add VecMultiUniversalRead<T, S>: minimal implementation over Vec<S: UniversalRead<T>>
with attach(source) -> SourceId for adding sources at runtime.
- Add test vec_multi_universal_read_batch_and_attach using MmapUniversal.
- Handle InvalidSourceId in segment OperationError From<UniversalIoError>.
Implements the interface and minimal mmap-based implementation from
docs/design/multi-file-universal-io-plan.md (multi-source universal I/O).
Made-with: Cursor
* refactor(universal_io): move multi-source interface to separate file, drop plan from PR
- Add universal_io/multi_universal_read.rs with SourceId, MultiUniversalRead,
VecMultiUniversalRead and test; re-export from mod.rs.
- Remove docs/design/multi-file-universal-io-plan.md from the branch.
Made-with: Cursor
* refactor(universal_io): require populate/clear_ram_cache; add new, attach, len, is_empty to trait
- MultiUniversalRead: remove default impls for populate() and clear_ram_cache();
they are now required.
- Add to trait: new(), len(), is_empty() (default), attach() (default Err).
- Introduce associated type Source for attach; add AttachUnsupported<T>
placeholder for impls that do not support dynamic attach.
- VecMultiUniversalRead: type Source = S; implement all trait methods.
- Re-export AttachUnsupported from universal_io.
Made-with: Cursor
* refactor(universal_io): remove AttachUnsupported; require attach for all impls
- Drop AttachUnsupported placeholder type and its UniversalRead/Send impls.
- Make attach() a required method on MultiUniversalRead (no default).
- Doc: all implementations must support attaching sources dynamically.
- Remove AttachUnsupported from re-exports.
Made-with: Cursor
* refactor(universal_io): attach by path, new(options), split vec impls, add MultiUniversalWrite
MultiUniversalRead:
- Remove type Source; attach(path, options) opens by path and returns SourceId.
- new(options: OpenOptions) for creating an empty multi-source view.
- Move VecMultiUniversalRead to vec_multi_universal_read.rs.
MultiUniversalWrite (new):
- Trait: new(options), len(), is_empty(), attach(path, options),
write_batch_multi((SourceId, offset, data)...), source_len, flusher(),
populate(), clear_ram_cache().
- VecMultiUniversalWrite in vec_multi_universal_write.rs; flusher()
runs all source flushers.
Re-export MultiUniversalWrite, VecMultiUniversalWrite from universal_io.
Made-with: Cursor
* universal_io: MultiUniversalWrite extends MultiUniversalRead
Make MultiUniversalWrite<T>: MultiUniversalRead<T> like UniversalWrite
extends UniversalRead. Remove duplicated methods (new, len, is_empty,
attach, source_len, populate, clear_ram_cache) from the write trait;
keep only write_batch_multi and flusher. VecMultiUniversalWrite now
impl MultiUniversalRead and MultiUniversalWrite separately.
Made-with: Cursor
* [manual] final fixes
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
* use `Cow<'_, [T]>` as return type in vector storages
* extract complex type
* clippy
* improve maybe_uninit_fill_from
* don't commit to graphlinks yet
* [manunal] Gridstore page use universal IO
* fmt
* Apply review feedback for universal IO gridstore pages (#8230)
* Apply review feedback from PR #8223
- Use `super::Result` import in mmap.rs instead of fully-qualified `crate::universal_io::Result`
- Restructure ValuePointer destructuring in get_value and delete_value to
first match Some(pointer), then destructure separately
* Replace Either<E, GridstoreError> with E: From<GridstoreError> in Gridstore::iter
Use a trait bound instead of Either to combine callback and gridstore
errors, allowing `?` to work directly on GridstoreError. This simplifies
callers by removing Either matching and io::Error conversion workarounds.
---------
Co-authored-by: qdrant-claw <qdrant-claw@users.noreply.github.com>
---------
Co-authored-by: qdrant-claw <qdrant-claw@users.noreply.github.com>
