* test: add failing tests for non-idempotent resharding consensus ops
Add unit tests proving that resharding operations (abort, commit_read,
commit_write, finish) return `BadRequest` errors instead of `Ok` when
re-applied or when local state diverges between peers.
Since `apply_entries` silently swallows all non-`ServiceError` results
(including `BadRequest`), these errors cause permanent resharding state
divergence between peers while consensus term/commit remain identical.
All 6 tests fail on dev, demonstrating the bug.
Made-with: Cursor
* fix: make resharding consensus operations idempotent (#8523)
* fix: make resharding consensus operations idempotent
When apply_entries processes a resharding operation that returns
BadRequest (not ServiceError), the error is silently swallowed and the
entry is marked as applied. If local state on a peer diverges (crash
during partial apply, prior swallowed error), the same committed Raft
entry produces different outcomes on different peers — permanent
resharding state divergence despite identical consensus term/commit.
Fix by making all resharding operations return Ok when the desired
post-condition is already met:
ShardHolder level:
- check_abort_resharding: return Ok when no resharding active or
different resharding active (already aborted)
- commit_read_hashring: return Ok when no resharding or stage already
past ReadHashRingCommitted
- commit_write_hashring: return Ok when no resharding or stage already
past WriteHashRingCommitted
- check_finish_resharding: return Ok when no resharding (already
finished)
- check_start_resharding: warn instead of error when shard already
exists for Up direction (leftover from crashed previous attempt)
Collection level (defense in depth):
- start_resharding: return Ok if same resharding already in progress
- commit_read/write_hashring: return Ok if no resharding or past stage
- finish_resharding: return Ok if no resharding active
- abort_resharding: return Ok if no resharding active
Made-with: Cursor
* Fix formatting issue
* Remove resharding idempotency note from logs
* Don't hold shard holder lock
* Committing read/write hash rings are idempotent now and are allowed
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: timvisee <tim@visee.me>
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: timvisee <tim@visee.me>
* rollback: iterator of errors into error of iterator
* [AI] ordering iterator
* [AI + manual] simplify geo-index iterators
* [AI] extra test for consistency
* fmt
* [AI] extend test for deleting points + fix the problem
* Update lib/segment/src/index/field_index/geo_index/mmap_geo_index.rs
Co-authored-by: Tim Visée <tim+github@visee.me>
---------
Co-authored-by: Tim Visée <tim+github@visee.me>
* Test if io_uring handles EINTR properly
* Fix unit test compilation after read_iter API change
Update test_io_uring_eintr_handling to match the new read_iter signature
that takes (Meta, ReadRange) tuples and returns Result<impl Iterator>.
Made-with: Cursor
* Install no-op SIGUSR1 handler in debug mode on Unix
Prevents SIGUSR1 from terminating the process with the default
disposition, so that io_uring EINTR tests can safely bombard
the process with signals.
Made-with: Cursor
* Enter tokio runtime context for SIGUSR1 handler, fix clippy
tokio::signal::unix::signal requires a reactor context, so enter
the runtime handle before installing the handler.
Also fix manual_let_else clippy warning in the EINTR unit test.
Made-with: Cursor
* Cleanup 🙄
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: Roman Titov <ffuugoo@users.noreply.github.com>
* derive Pod for Point<T>
* remove clones on Copy type
* fix annoying clippy lint
* serde skip padding
* extract point into a new file
* assert aligment
* Make subgraph_connectivity deterministic
* Fix missing rng argument in test for subgraph_connectivity
Made-with: Cursor
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Previously `clear_all_tmp_directories()` was called in `main.rs` after
`TableOfContent::new()` had already loaded all collections and applied
WAL. Stale temp files from a previous crash (e.g. interrupted snapshot
transfers) could interfere with the recovery process.
Move the cleanup into `TableOfContent::new()` so it runs before the
collection loading loop. Extract a standalone `clear_tmp_directories()`
function that only needs `StorageConfig`, and delegate the existing
method to it.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
Replace `Vec<(GeoHash, AHashSet<PointOffsetType>)>` with three flat
parallel arrays: `Vec<GeoHash>`, `Vec<u32>` (offsets), and
`Vec<PointOffsetType>` (IDs).
Each AHashSet carried ~64 bytes of stack overhead (32-byte RandomState +
32-byte RawTable metadata) plus hash-table heap allocations with control
bytes — totalling ~124 bytes per entry even with a single point. The
flat layout uses 8 + 4 + 4 = 16 bytes per entry, a ~7.75x reduction.
Deletions use a sentinel value (u32::MAX) in the IDs array, filtered
during iteration. On-disk format is unchanged.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
When `prevent_unoptimized=true` and `wait=true`, the update worker
blocks in `wait_for_deferred_points_ready` until all deferred points
are optimized. If the client specifies a timeout, `LocalShard::update`
drops the oneshot receiver and returns `WaitTimeout` to the caller,
but the update worker remains stuck in the wait loop. Since the worker
processes operations sequentially, this causes head-of-line blocking
for all subsequent updates on that shard.
Fix this by passing the feedback oneshot sender into the wait function
and checking `is_closed()` at the top of each loop iteration. When the
caller's receiver is dropped (timeout), the wait breaks immediately,
allowing the worker to proceed with the next operation.
The data is already written to WAL and applied to segments before we
enter the wait, so deferred points will still be optimized eventually
by the background optimizer.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
* use pageout to clear mmap cache
* Also clear cache of deleted flags in mmap dense vector storage
* Add reference to madvise man pages for probe logic
* use deconstruct
---------
Co-authored-by: timvisee <tim@visee.me>
* Add mincore-based memory stats to MmapFile
Add `resident_bytes()`, `disk_bytes()`, and `probe_memory_stats()` methods
to `MmapFile` for measuring page cache residency via `mincore(2)`. This is
the foundation for per-collection memory usage reporting.
Also extract `page_size()` as a public function in `mmap::advice`, replacing
the internal `PAGE_SIZE_MASK` with a direct page size cache.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AI] introduce trait for reporting memory usage per component
* [AI] memory reporter implementation for vector storage
* [AI] implement MemoryReporter for QuantizedVectors
* [AI] implement MemoryReporter for VectorIndexEnum
* Implement MemoryReporter for IdTrackerEnum with RAM estimation
Add ram_usage_bytes() to all ID tracker types and their data structures:
- PointMappings, CompressedPointMappings, CompressedVersions,
CompressedInternalToExternal, CompressedExternalToInternal
- MutableIdTracker, ImmutableIdTracker, InMemoryIdTracker
All ID trackers load their data into RAM (none use mmap for working data).
Files are reported as OnDisk (persistence only), actual RAM footprint
is reported via extra_ram_bytes. Uses struct destructuring to ensure
new fields trigger compile errors if not accounted for.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AI] implement MemoryReporter for PayloadStorageEnum and adjust FileStorageIntent
* [AI] implement MemoryReporter for PayloadStorageEnum and adjust FileStorageIntent
* [AI] implement MemoryReporter for payload indexes: in-ram structures memory consumtion computation + caching
* [AI] implement MemoryReporter for payload indexes: in-ram structures memory consumtion computation + caching
* [AI] segment-level memory usage report
* [AI] Block 3: Aggregation Layer and Data Model + internal api for remote shard
* [AI] REST API handler
* fmt
* [AI] clippy fixes
* [AI] macos fix + proxy segment fix
* [AI] make text index estimation a bit more correct
* fix is_on_disk reporting for dense_vector_storage
* fix after rebase
* [AI] deep account for quantized vectors RAM usage + unify chunk size + shring volatile storage after load
* remove debug log
* cache in test
* make manual test easier to run
* rollback chunk size diff, but keep it for test only
* review fixes
* Use exhaustive match
* Use div_ceil on bits everywhere
It does not seem to be strictly necessary because the number of bits
should already be a multiple of the used container size bytes. Still
it's good practice to be careful with this calculation.
* Improve heap size bytes for encoded product quantization vectors
* Include vector stats for binary quantized vectors
* In volatile chunked vectors, include heap allocated vector
* Include rest of heap allocated structures for mutable map index
* In mutable geo index, the hash map is also heap allocated
* Update tests/manual/test_memory_reporting.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: timvisee <tim@visee.me>
Co-authored-by: Tim Visée <tim+github@visee.me>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* fix: propagate flush cancellation through payload index flushers
When a payload index is deleted, the gridstore backing it is dropped.
A concurrent background flush detects this (weak Arc refs fail to upgrade)
and returns GridstoreError::FlushCancelled. The From<GridstoreError> impl
correctly maps this to OperationError::Cancelled, and entry.rs gracefully
handles Cancelled by skipping the flush.
However, two intermediate layers unconditionally wrapped all errors as
ServiceError, destroying the Cancelled variant before it reached the
handler in entry.rs:
1. All 4 mutable index flushers (map, text, numeric, geo) used
OperationError::service_error(format!(...)) instead of converting
via Into<OperationError> first.
2. struct_payload_index flusher also unconditionally wrapped sub-flusher
errors as service_error.
This caused the benign FlushCancelled to surface as a fatal
"last background flush failed" ServiceError.
Fix both layers to propagate Cancelled errors as-is, matching the
pattern already used for vector storage, quantization, and id tracker
flushers in entry.rs.
Made-with: Cursor
* simplify error conversion
---------
Co-authored-by: Cursor Agent <agent@cursor.com>
Co-authored-by: generall <andrey@vasnetsov.com>
* fix(segment): reserve full last chunk when capacity aligns to chunks
When the requested vector count is an exact multiple of chunk_capacity,
the remainder modulo chunk_capacity is zero. The previous expression used
that remainder for the last chunk's flattened length, reserving zero
capacity. Reserve chunk_capacity * dim for a full last chunk instead.
* test(segment): regression for try_set_capacity_exact last chunk at chunk boundary
Assert flattened Vec capacity for the last chunk when the requested vector
count is a multiple of chunk_capacity. The buggy implementation reserved
zero for that chunk; this test fails without the fix and passes with it.
* cargo fmt edited file
* Add empty placeholder vector storage types for named vector CRUD
Introduce EmptyDenseVectorStorage and EmptySparseVectorStorage as
placeholder storages for newly created named vectors on immutable
segments. These report all vectors as deleted, consume no disk space,
and are reconstructed from segment config on load via the new
VectorStorageType::Empty and SparseVectorStorageType::Empty variants.
Key design decisions:
- is_on_disk is derived from original user config, not hardcoded
- MultiVectorConfig is preserved for multi-vector support
- Config mismatch optimizer skips Empty storage to avoid false rebuilds
- Quantization delegates normally (handles 0 vectors gracefully)
- get_vector includes debug_assert to catch unexpected access
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [AI] segment-level operations for creating and deleting anmed vectors
* [AI] implement named vector creation and deleting in proxy segment
* [AI] Step 3: Proxy Segment Handling for Named Vector Operations
* [AI] implement for Edge
* [AI] implement consensus operations for named vector operations
* [AI] refactor VectorNameConfig, remove VectorNameConfigInternal
* [AI] handle vector schema inconsistency in raft snapshot recovery
* [AI] rest + grpc API
* [AI] clippy
* [AI] generate openAPI schema
* fmt
* ci fixes
* [AI] fix jwt access test
* [AI] nop operation for awaiting of consensus-commited update ops
* [AI] move vector name operations into points service
* [AI] implement internal api for vector name operations
* [AI] change collection-level config along with segment level operation
* [AI] vector schema reconceliation instead of error
* fmt
* missing compile-time option
* [AI] integration test
* [AI] fix missing JWT tests
* [AI] remove NOP
* [AI] openapi test
* [AI] fix initialization of mutable segment
* [AI] more simple integration tests
* fmt
* [AI] make cluster test a bit harder
* [AI] make test less flacky
* [AI] rabbit comments
* [AI] check params compatibility before writing vector config
* [AI] make sure to register vector storages in structure payload index
* [AI] vector name validation
* lower vector length validation to 200 chars to account for prefix in filename
* [AI] proxy segment: prevent stale data leak through optimization
* fmt
* [AI] filter out removed vectors from proxy response
* [AI] handle vector name in proxy
* fmt
* adjust proxy info based on dropped vectors
* [AI] proxy segment: update filters to correct has_vector condition
* fmt
* clippy
* Fix consensus snapshot applicaiton for vector schema
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix empty sparse vector name validation
* Reorder validation to check empty sparse name before duplicate name
---------
Co-authored-by: leohenon <77656081+lhenon999@users.noreply.github.com>
* Return error instead of panicking for corrupted aliases file on startup
* common::fs::ops: provide file name in error messages
And drop FileStorageError in favor of std::io::Error, since we always
convert all kinds of errors into ServiceError anyway.
* TableOfContent:🆕 return errors instead of panics
Also, drop context strings. We use fs_err anyway, that should be enough.
---------
Co-authored-by: leohenon <77656081+lhenon999@users.noreply.github.com>
Co-authored-by: xzfc <xzfcpw@gmail.com>
When audit logging is enabled, log the API method path (REST path or
gRPC method name) by default. Users who don't want the extra field can
still set `audit.log_api: false` explicitly.
Made-with: Cursor
Co-authored-by: Cursor Agent <agent@cursor.com>
* [ai] Replace manual into mappings with Into::into
* Reformat
* [ai] Use implicit .iter
* Don't iterate over keys too
* [ai] Replace unwrap_or
* Reformat
* [ai] Use as_deref and then_some
* [ai] Use more to_string
* [ai] Use explicitly typed into conversions
* Reformat
* [ai] More explicit into conversions
* Reformat
* Add optional `api` field to audit log events
Add a new `api` field to audit log entries that records the API method
path (REST path or gRPC method name). Controlled by the `log_api` audit
config option. For denied auth requests, `api` is always logged and
`method` is omitted since there is no internal operation name available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* deconstruct
* fix: test edge case
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Boros <dancixx@gmail.com>
