* refactor PointToValues to use UniversalRead
It includes a significant change to `MmapValue` trait to be able to
handle Cow reads, instead of just references.
* fix str parsing
* fix incorrect path
* use fallible casting
* clippy
* No eager allocation
this also makes it so that borrowed strs can keep happening :heart-eyes:
* Lifetime cleanup
---------
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* feat(universal_io): add MultiUniversalRead trait and VecMultiUniversalRead impl
- Add SourceId, MultiUniversalRead<T> trait with read_batch_multi, source_len,
populate, clear_ram_cache (latter two with default no-op).
- Add UniversalIoError::InvalidSourceId for invalid source id in batch reads.
- Add VecMultiUniversalRead<T, S>: minimal implementation over Vec<S: UniversalRead<T>>
with attach(source) -> SourceId for adding sources at runtime.
- Add test vec_multi_universal_read_batch_and_attach using MmapUniversal.
- Handle InvalidSourceId in segment OperationError From<UniversalIoError>.
Implements the interface and minimal mmap-based implementation from
docs/design/multi-file-universal-io-plan.md (multi-source universal I/O).
Made-with: Cursor
* refactor(universal_io): move multi-source interface to separate file, drop plan from PR
- Add universal_io/multi_universal_read.rs with SourceId, MultiUniversalRead,
VecMultiUniversalRead and test; re-export from mod.rs.
- Remove docs/design/multi-file-universal-io-plan.md from the branch.
Made-with: Cursor
* refactor(universal_io): require populate/clear_ram_cache; add new, attach, len, is_empty to trait
- MultiUniversalRead: remove default impls for populate() and clear_ram_cache();
they are now required.
- Add to trait: new(), len(), is_empty() (default), attach() (default Err).
- Introduce associated type Source for attach; add AttachUnsupported<T>
placeholder for impls that do not support dynamic attach.
- VecMultiUniversalRead: type Source = S; implement all trait methods.
- Re-export AttachUnsupported from universal_io.
Made-with: Cursor
* refactor(universal_io): remove AttachUnsupported; require attach for all impls
- Drop AttachUnsupported placeholder type and its UniversalRead/Send impls.
- Make attach() a required method on MultiUniversalRead (no default).
- Doc: all implementations must support attaching sources dynamically.
- Remove AttachUnsupported from re-exports.
Made-with: Cursor
* refactor(universal_io): attach by path, new(options), split vec impls, add MultiUniversalWrite
MultiUniversalRead:
- Remove type Source; attach(path, options) opens by path and returns SourceId.
- new(options: OpenOptions) for creating an empty multi-source view.
- Move VecMultiUniversalRead to vec_multi_universal_read.rs.
MultiUniversalWrite (new):
- Trait: new(options), len(), is_empty(), attach(path, options),
write_batch_multi((SourceId, offset, data)...), source_len, flusher(),
populate(), clear_ram_cache().
- VecMultiUniversalWrite in vec_multi_universal_write.rs; flusher()
runs all source flushers.
Re-export MultiUniversalWrite, VecMultiUniversalWrite from universal_io.
Made-with: Cursor
* universal_io: MultiUniversalWrite extends MultiUniversalRead
Make MultiUniversalWrite<T>: MultiUniversalRead<T> like UniversalWrite
extends UniversalRead. Remove duplicated methods (new, len, is_empty,
attach, source_len, populate, clear_ram_cache) from the write trait;
keep only write_batch_multi and flusher. VecMultiUniversalWrite now
impl MultiUniversalRead and MultiUniversalWrite separately.
Made-with: Cursor
* [manual] final fixes
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
* use `Cow<'_, [T]>` as return type in vector storages
* extract complex type
* clippy
* improve maybe_uninit_fill_from
* don't commit to graphlinks yet
* [manunal] Gridstore page use universal IO
* fmt
* Apply review feedback for universal IO gridstore pages (#8230)
* Apply review feedback from PR #8223
- Use `super::Result` import in mmap.rs instead of fully-qualified `crate::universal_io::Result`
- Restructure ValuePointer destructuring in get_value and delete_value to
first match Some(pointer), then destructure separately
* Replace Either<E, GridstoreError> with E: From<GridstoreError> in Gridstore::iter
Use a trait bound instead of Either to combine callback and gridstore
errors, allowing `?` to work directly on GridstoreError. This simplifies
callers by removing Either matching and io::Error conversion workarounds.
---------
Co-authored-by: qdrant-claw <qdrant-claw@users.noreply.github.com>
---------
Co-authored-by: qdrant-claw <qdrant-claw@users.noreply.github.com>
* Unify parking_lot/arc_lock feature
* Move lib/common/{io,memory}/* -> lib/common/common/*
- Mmap-related items are grouped into `common::mmap` sub-module:
- `memory/src/chunked_utils.rs` -> `common/src/mmap/chunked.rs`
- `memory/src/madvise.rs` -> `common/src/mmap/advice.rs`
- `memory/src/mmap_ops.rs` -> `common/src/mmap/ops.rs`
- `memory/src/mmap_type_readonly.rs` -> `common/src/mmap/mmap_readonly.rs`
- `memory/src/mmap_type.rs` -> `common/src/mmap/mmap_rw.rs`
- Filesystem-related items are grouped into `common::fs` sub-module:
- `common/src/fs.rs` -> `common/src/fs/sync.rs`
- `io/src/file_operations.rs` -> `common/src/fs/ops.rs`
- `io/src/move_files.rs` -> `common/src/fs/move.rs`
- `io/src/safe_delete.rs` -> `common/src/fs/safe_delete.rs`
- `memory/src/checkfs.rs` -> `common/src/fs/check.rs`
- `memory/src/fadvise.rs` -> `common/src/fs/fadvise.rs`
- Rest is moved straight into `common`:
- `io/src/storage_version.rs` -> `common/src/storage_version.rs`
The old `io` and `memory` are now hollow crates that re-export items
from `common`. These hollow crates will be removed in next commits.
* Replace uses of `io` and `memory` with new paths in `common`
Since `io` and `memory` are just re-exports of `common`, these
replacements are no-op.
* Remove `io` and `memory` crates
* download tar
* compute sha256 for stream download
* wip: propagate unpacking into down to the logic, todo: validation
* unpacked snapshot validation
* Minor tweaks
* Fix typo
* validation during unpack
* cancellation token
* update docstring
* remove redundant dep
* Rearrange unpack functions
- Rename `safe_unpack.rs` into `tar_unpack.rs` so it would be listed
near `tar_ext.rs` in IDEs.
- Replace calls like `ar = open_snapshot_archive(…); safe_unpack(ar, …);`
with a single call to `tar_unpack_file(…)`.
- Put calls to `Archive::new(); Archive::set_overwrite(false);` inside
`tar_unpack_reader` (was `safe_unpack`). So, now it is the only place
that does `set_overwrite`.
* Let clippy complain if tar::Archive::unpack used
* Mock snapshot download URL
Instead of downloading from storage.googleapis.com every time the test
runs, put small snapshot file to the repo.
The snapshot file is created using this command:
curl -s \
https://storage.googleapis.com/qdrant-benchmark-snapshots/test-shard.snapshot \
| tar \
--delete segments/4ea958d8-0b64-4312-9a53-0cd857e93535.tar \
--delete segments/65ac6276-8cca-4f5c-b767-9722190cee8b.tar \
> lib/storage/src/content_manager/snapshots/test-shard.snapshot
File contents:
$ tar tf lib/storage/src/content_manager/snapshots/test-shard.snapshot
wal/
wal/closed-255
newest_clocks.json
replica_state.json
shard_config.json
$ du -sh lib/storage/src/content_manager/snapshots/test-shard.snapshot
12K lib/storage/src/content_manager/snapshots/test-shard.snapshot
$ sha256sum < lib/storage/src/content_manager/snapshots/test-shard.snapshot
5d94eac5c1ede3994a28bc406120046c37370d5d45b489a0d2252531b4e3e1f2 -
---------
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: xzfc <xzfcpw@gmail.com>
* Implement `Optional<T>` type
Define a type with the same presumed layout as `Option<T>`, but with defined behavior.
* Make `transmute_*` functions unsafe
The functions `memory::mmap_ops::transmute_*` are inherently unsafe, but
are not marked as are. Their usage is documented, but it is not always clear
if the code is correct.
* Add `CsrHeader` to resolve another unsoundness
Tuples have no defined layout.
* WIP: introduce new vector store type
* handling of InRamMmap
* fmt
* feature-flag
* fmt
* Use if else
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Update lib/common/common/src/flags.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
* also choose madvise for single-file in-ram-mmap
* simplify generics
* gpu fix
* fix bug
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* Remove call to Archive::set_sync
Also, this method was the last remaining part of our `tar-rs` fork,
so we can switch to the upstream version now.
* Do syncfs
* Replace `Option<Segment>` with `enum LoadSegmentOutcome`
* Replace some Path/PathBuf with str/String
* Rename field Segment::{current_path -> segment_path}
* safe_delete
* introduce IsAliveLock
* add tests
* rename to mark_dead
* update gridstore comment
* Update some comments
* Add must_use attribute to lock_if_alive
* Rename mark_dead to blocking_mark_dead
* make the handle take a `Weak` reference
* make dropping explicit
* use `Mutex::lock_arc` instead
* less nesting
* clippy
---------
Co-authored-by: timvisee <tim@visee.me>
* Separate set and unset lists for each Gridstore pointer update
This makes the list of pointer updates much easier to grasp. The
simplification is desired because this exact structure has been a cause
for bugs multiple times now.
* Add more aggressive debug assertions
* Use consistent terminology
* Correct removal of set
* Simplify drain function
* Add safe guard to ignore empty pointer updates
* Rework tracker getter, explicitly branch variants
* Patch transmute, require Sized type
* rename and adjust comments
- set/unset is now current/to_free
- adjusted descriptions and comments for this nomenclature too
* clippy
* clippyyy
not 4, not 2, but 3 spaces
---------
Co-authored-by: Luis Cossío <luis.cossio@outlook.com>
* Add progress_tracker.rs
* Pass progress tracker around
* Populate progress tracker with actual data
* Expose progress on `/collections/{name}/optimizations` endpoint
* Fix delete_page not dropping sequential mmap
* In Gridstore Page, support using a single mmap
* In Gridstore Page, only open multiple mmaps if supported at runtime
* Support opening single mmap in UniversalMmapChunk
* Rename environment var, log warning when QDRANT_NO_MULTI_MMAP is set
* Support opening single mmap in MmapDenseVectors
* Remove unused result response
* Import LazyLock
