Commit Graph
11 Commits
Author SHA1 Message Date
xzfc 66d0c36009 Merge io and memory into common (#8155)
* Unify parking_lot/arc_lock feature

* Move lib/common/{io,memory}/* -> lib/common/common/*

- Mmap-related items are grouped into `common::mmap` sub-module:
  - `memory/src/chunked_utils.rs`      -> `common/src/mmap/chunked.rs`
  - `memory/src/madvise.rs`            -> `common/src/mmap/advice.rs`
  - `memory/src/mmap_ops.rs`           -> `common/src/mmap/ops.rs`
  - `memory/src/mmap_type_readonly.rs` -> `common/src/mmap/mmap_readonly.rs`
  - `memory/src/mmap_type.rs`          -> `common/src/mmap/mmap_rw.rs`
- Filesystem-related items are grouped into `common::fs` sub-module:
  - `common/src/fs.rs`          -> `common/src/fs/sync.rs`
  - `io/src/file_operations.rs` -> `common/src/fs/ops.rs`
  - `io/src/move_files.rs`      -> `common/src/fs/move.rs`
  - `io/src/safe_delete.rs`     -> `common/src/fs/safe_delete.rs`
  - `memory/src/checkfs.rs`     -> `common/src/fs/check.rs`
  - `memory/src/fadvise.rs`     -> `common/src/fs/fadvise.rs`
- Rest is moved straight into `common`:
  - `io/src/storage_version.rs` -> `common/src/storage_version.rs`

The old `io` and `memory` are now hollow crates that re-export items
from `common`. These hollow crates will be removed in next commits.

* Replace uses of `io` and `memory` with new paths in `common`

Since `io` and `memory` are just re-exports of `common`, these
replacements are no-op.

* Remove `io` and `memory` crates
2026-02-17 11:01:14 +01:00
Luis CossíoandTim Visée 1e23057ab8 More graceful handle of flush cancellation (#7798)
* Reapply "return an error when cancelling a flush (#7781)" (#7799)

This reverts commit 4a1103acb7.

* log::trace cancellation of components

* gracefully handle flush cancellation inside segment flusher

* Update lib/segment/src/segment/entry.rs

Co-authored-by: Tim Visée <tim+github@visee.me>

* don't wrap cancellation errors

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
2026-02-09 23:19:37 +01:00
Tim Visée feff43f477 Use RwLock in flushers, drop is alive lock early (#7811)
* Use RwLock for pending changes in MmapBitSliceBufferedUpdateWrapper

* Use RwLock for pending operations in DatabaseColumnScheduledDeleteWrapper

* Use RwLock for pending updates in DatabaseColumnScheduledUpdateWrapper

* Drop alive guard before reconciliation, we don't touch files after

* Remove redundant clone

* Update comments
2026-02-09 22:26:33 +01:00
Tim Visée 1287d74026 Fix reconciliation in more flushers (#7805)
* Reconcile BufferedDynamicFlags flusher

* Reconcile DatabaseColumnScheduledDeleteWrapper flusher

* Rename function for consistency
2025-12-19 12:16:17 +01:00
Tim Visée 66bc5524dc Revert "return an error when cancelling a flush (#7781)" (#7799)
This reverts commit 2f443ec055.
2025-12-18 17:29:24 +01:00
Luis Cossío 683fc3c964 return an error when cancelling a flush (#7781)
* return an error when cancelling a flush

* fix test

* less verbosity
2025-12-18 17:29:17 +01:00
Luis Cossíoandtimvisee f834a11aad Introduce IsAliveLock (#7751)
* introduce IsAliveLock

* add tests

* rename to mark_dead

* update gridstore comment

* Update some comments

* Add must_use attribute to lock_if_alive

* Rename mark_dead to blocking_mark_dead

* make the handle take a `Weak` reference

* make dropping explicit

* use `Mutex::lock_arc` instead

* less nesting

* clippy

---------

Co-authored-by: timvisee <tim@visee.me>
2025-12-18 17:28:22 +01:00
Arnaud Gourlay 96e60691ec Fix payload indexes drop/flush sequence (#7626)
* Fix release mmap handles before removing files

* weak references to handle after drop

* protect from concurrent drop/flush

* simplify locking

* Revert "simplify locking"

This reverts commit 7ebd324be4.
2025-12-03 10:23:55 +01:00
Arnaud Gourlay 9c4af009c0 Fix BufferedDynamicFlags flushing with concurrent drop/deletion (#7621)
* Fix BufferedDynamicFlags flushing with concurrent drop/deletion

* make sure to drop before fs remove
2025-12-03 10:20:11 +01:00
xzfc 6a2830db1c Use fs-err (#7319) 2025-11-14 12:26:51 +01:00
Luis Cossíoandtimvisee f2d45b558d Use BitvecFlags in vector storages (#6994)
* put all flags impls in a single module

* use in appendable_dense_vector_storage

* use in appendable_mmap_multi_dense_storage

* use in mmap_sparse_vector_storage

* clippy

* Add comment we cannot rely on length of deleted vectors bitvec

---------

Co-authored-by: timvisee <tim@visee.me>
2025-08-11 13:17:39 +02:00