* Replace cgroups-rs with direct cgroup memory file reads
We used cgroups-rs in exactly one place, to read the memory limit and
usage of our own cgroup, so read those files directly instead. Drops 34
crates from the lockfile, including the zbus stack that carries
RUSTSEC-2026-0221.
Also fixes two latent cgroup v1 bugs (the LONG_MAX unlimited sentinel
reported ~9 EB of total memory, an unreadable limit file reported 0
bytes) and the hierarchy mix-up on hybrid hosts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Decline cgroup memory reporting when the usage read fails
Reporting a usage of 0 made available_memory_bytes claim the whole cgroup
limit as free. Fall back to sysinfo when the usage file cannot be read at
init, and keep the last known value on a failed refresh.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Keep the last known memory limit when its read fails
A transient read failure cleared the cached limit and silently fell back
to host memory while the process was still capped, the same direction of
over-reporting as the usage read. Both now keep their last known value,
and a limit lifted at runtime still clears.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Treat a malformed memory limit as an error, not as unlimited
Parse failures returned Ok(None), so garbage in the limit file cleared a
valid cached limit on refresh and read as unlimited at init. Reserve
Ok(None) for "max" and the v1 sentinel, and report anything else as
InvalidData so the last known limit survives.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Report effective (cgroup) CPU, RAM and disk in telemetry
The `system` block reported host-level figures that ignore the limits the
kernel actually enforces on the process:
- `cores` <- sys_info::cpu_num() (host socket count)
- `ram_size` <- sys_info::mem_info() (host total RAM)
- `disk_size` <- sys_info::disk_info() (container root fs)
On any cgroup-limited deployment (containers, Kubernetes pods, systemd
slices) these overstate what Qdrant can use, are misleading for capacity /
oversubscription analysis, and don't match how Qdrant sizes itself.
Report the effective values instead, reusing existing helpers:
- `cores` -> common::cpu::get_num_cpus() (already drives sizing)
- `ram_size` -> segment::utils::mem::total_memory_bytes()
(cgroup limit via cgroups_rs, else sysinfo host total)
- `disk_size` -> common::disk_usage::disk_usage(storage_path)
(data-volume capacity, cached; sys_info host disk fallback)
`Mem::new()` is not free (it builds a sysinfo System and loads the cgroup
memory controller), so `total_memory_bytes()` caches with a 5s TTL — matching
the disk-usage cache — instead of recomputing per call. The TTL (rather than
caching once) means an in-place cgroup memory resize is reflected within a few
seconds, consistent with how `disk_size` and `cores` already behave. The
strict-mode helper in `collection` now delegates to this shared accessor
(dropping its own OnceLock), so it too becomes resize-aware.
ram_size/disk_size stay in KiB to match the previous unit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Simplify comments and code
* Simplify openapi spec
* minor comment improve
* mem: cache total_memory_bytes like disk_usage (5s TTL)
`total_memory_bytes()` mirrors `common::disk_usage::disk_usage`: a small 5s
TTL cache over `Mem::new().total_memory_bytes()`. `Mem::new()` is not free
(builds a sysinfo System + loads the cgroup controller), and the short TTL
keeps the value in step with an in-place cgroup memory resize rather than
freezing at startup. Shared by telemetry and strict-mode, like the disk cache.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Regenerate OpenAPI spec
Add the ram_size / disk_size field descriptions produced by schemars from the
updated telemetry doc comments, keeping the generated spec consistent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* mem: address review — parking_lot mutex, hold lock, saturating_duration_since
- Use parking_lot::Mutex (no poisoning; lock() returns the guard directly).
- Hold the lock across the whole method — single acquisition, simpler.
- saturating_duration_since instead of duration_since (no panic on a cached
timestamp spuriously ahead of now).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Bump Rust edition to 2024
* gen is a reserved keyword now
* Remove ref mut on references
* Mark extern C as unsafe
* Wrap unsafe function bodies in unsafe block
* Geo hash implements Copy, don't reference but pass by value instead
* Replace secluded self import with parent
* Update execute_cluster_read_operation with new match semantics
* Fix lifetime issue
* Replace map_or with is_none_or
* set_var is unsafe now
* Reformat
* WIP: Start working on out-of-RAM errors handling [skip ci]
* Implement basic handling of out-of-RAM errors during Qdrant startup
* Try to fix CI fail by allowing both V1 and V2 cgroups
* Try to fix CI fail by improving cgroups handling
* Fix cgroups path detection/handling (+ some minor stylistic changes)
* fixup! Fix cgroups path detection/handling (+ some minor stylistic changes)
* Add test
* Enable low RAM test
* fixup! Add test
* free memory checks
* rm unused function
* Oom fallback script (#1809)
* add recover mode in qdrant + script for handelling OOM
* fix clippy
* reformat entrypoint.sh
* fix test
* add logging to test
* fix test
* fix test
---------
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
* WIP: Fix `Segment::take_snapshot`
TODO:
- This commit, probably, breaks snapshotting of segments with memmapped vector storage
- `ProxySegment::take_snapshot` seems to potentially similar bug
* WIP: Fix `Segment::take_snapshot`
- Fix snapshotting of `StructPayloadIndex`
- Fix snapshotting of segments with memmapped vector storage
- Temporarily break `ProxySegment::take_snapshot`
* Fix `ProxySegment::take_snapshot`
* Remove `copy_segment_directory` test
* nitpicking
* clippy fixes
* use OperationError::service_error
* Cleanup `TinyMap` trait bounds and derive `Debug`
* Fix `test_snapshot` test
- Derive `Debug` for `NamedVectors`
* Move utility functions from `segment.rs` to `utils` module
* Contextualize `segment::utils::fs::move_all` a bit more carefully
* Fix a typo
* add backward compatibility with old snapshot formats
* fmt
* add snapshot for compatibility test
* git lfs is a piece of shit
* Nitpicking
Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
