Files
VoiceStudio/tests/backend/engines/test_hardware_probe.py
T
Palash DebnathandClaude Opus 4.7 b34dcd9e11 Phase 4 Plan 04-01: SPIKE-01 GGUF — GO + integration (#100)
* Phase 4 Plan 04-01: SPIKE-01 GGUF — GO + Wave 1 integration

Integrates Serveurperso/OmniVoice-GGUF as a hardware-adaptive default
voice-cloning engine, with overridable fallback to the in-process
OmniVoiceBackend. Spike confirmed GO: the model is a clean quantization
of k2-fsa/OmniVoice (Apache-2.0 + MIT runtime, `omnivoice-lm` custom
architecture so it does NOT load in vanilla llama.cpp).

Pinned SHAs:
  * Serveurperso/OmniVoice-GGUF revision: 361609388ae572a820d085185bbbe2a2aac4b30e
  * ServeurpersoCom/omnivoice.cpp master:  886fc079838ca7400cb2b42b36e2a65aa1daabe8

Implements GGUF-01 (hardware probe) through GGUF-05 (default-engine
resolver with graceful fallback). The four `bin/omnivoice-tts-*`
artifacts are committed as zero-byte placeholders; the new CI matrix
job builds the real binaries per platform from the pinned commit SHA
and appends a SHA-256 manifest used by `is_available()` for tampering
detection (T-04-01). The macos-14 (Apple Silicon) slot is marked
`continue-on-error: true` because omnivoice.cpp publishes no
`buildmetal.sh` (Pitfall 1 / Assumption A1) — failure feeds into Task
3's GO/NO-GO call.

Quant override is allow-listed against quant_map.json entries only
(T-04-05). Argv is composed from typed Path objects rooted in
HF_HUB_CACHE; never uses `shell=True`. HF token redaction applies to
captured stderr before logging (AUTH-05 / T-04-04).

Tests: 36 new (8 hardware-probe + 13 GGUF engine + 6 settings_store
quant override + grep gate); 428 passed in full suite vs 402+ baseline.
ADR Status stays "Proposed (research-supported)" — Task 3 (human
checkpoint) flips to Accepted after CI produces real binaries and a
reviewer signs off on the GGUF-06 cross-hardware smoke.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: install libopenblas-dev on linux-x86_64 omnivoice-tts build

The pinned omnivoice.cpp commit (886fc079...) ships a `buildcpu.sh`
that passes `-DGGML_BLAS=ON`. ubuntu-latest has no BLAS implementation
preinstalled, so the cmake configure step fails with
`Could NOT find BLAS (missing: BLAS_LIBRARIES)` and the job exits in
13 s before producing the linux-x86_64 binary.

macOS (Accelerate, built in) and Windows (BLAS off by default in the
ggml CMakeLists for non-APPLE platforms — the build script doesn't
invoke buildcpu.sh on those slots) are unaffected and stay green.

Adds a Linux-gated apt step to install libopenblas-dev + pkg-config
before the build, restoring cross-platform parity per the
CLAUDE.md "default features must work on every platform" rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gguf): constrain ref_audio to project roots — block /etc/shadow on Linux

The GGUF engine's `_build_argv` previously validated ref_audio only via
`ref_path.is_file()` — i.e. "does this path exist?" That check is
platform-dependent: `/etc/shadow` doesn't exist on macOS (rejected
naturally), but it IS a real system file on Linux, so the validation
silently accepted it. CI's ubuntu-22.04 runner exposed the gap via
`test_generate_blocks_freeform_ref_audio`, which exists precisely to
guard the "freeform ref_audio path" attack surface.

Fix: confine ref_audio to one of three allowed roots before existence
checks:
  - VOICES_DIR (user-saved voice profiles)
  - DUB_DIR (per-job auto-clones extracted from source video)
  - tempfile.gettempdir() (browser-upload temp files; existing
    `cleanup_ref` flow in generation.py)

Anything outside those roots → FileNotFoundError, matching the existing
failure-mode contract callers handle. Existence check still runs after,
so the test's mocked subprocess.run is never reached and the test
passes deterministically on all three platforms.

Cross-platform parity (per CLAUDE.md 2026-05-20 rule): identical
behaviour on macOS / Windows / Linux — the allow-list is computed from
core.config which uses platform-specific path resolution but yields the
same logical "project tree" on every OS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(gguf): mark darwin-x86_64 binary build as experimental

GitHub's macos-13 (Intel) runner pool is heavily contended — PR #100
queued for 30+ minutes waiting on darwin-x86_64 while every other
platform finished in ~1m. Intel Macs are also fading hardware (Apple's
platform momentum is entirely on Apple Silicon), and the GGUF engine's
runtime already handles a missing binary gracefully (`is_available()`
returns False on Intel Mac with a "binary not bundled for this
platform" message, same path used for first-launch before any binaries
build).

`experimental: true` mirrors what darwin-arm64 (Metal) already has —
slot still runs and uploads its binary when successful, but a failure
or runner backlog no longer blocks merges. Keeps the GGUF engine
shippable across the dominant arm64 / Linux / Windows surface without
holding the inbox on a slow-runner queue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:51:31 +05:30

132 lines
4.7 KiB
Python

"""Unit tests for backend/engines/omnivoice_gguf/hardware_probe.py (GGUF-01).
Covers the five behaviour bullets from Plan 04-01 Task 1:
1. CUDA + 16 GB VRAM → compute_class="high-vram"
2. CUDA + 4 GB VRAM → compute_class="mid-vram"
3. CUDA + 1.5 GB VRAM → compute_class="low-vram"
4. MPS → uses psutil.virtual_memory().total // 2 as effective VRAM ceiling
5. CPU-only → backend="cpu", vram_mb=0, compute_class="cpu"
Plus a re-export check so callers can keep importing
``detect_capabilities`` from either ``engines.omnivoice_gguf.hardware_probe``
or ``services.gpu_sandbox`` (per the "single entry point" decision in
RESEARCH.md "Architectural Responsibility Map").
"""
from __future__ import annotations
from unittest.mock import patch
def _make_torch_mock(*, cuda_available=False, total_vram_bytes=0, mps_available=False):
"""Build a torch-module mock with controllable cuda / mps shape."""
import types
cuda = types.SimpleNamespace(
is_available=lambda: cuda_available,
mem_get_info=lambda: (total_vram_bytes // 2, total_vram_bytes),
)
mps = types.SimpleNamespace(is_available=lambda: mps_available)
backends = types.SimpleNamespace(mps=mps)
return types.SimpleNamespace(cuda=cuda, backends=backends)
def test_cuda_16gb_returns_high_vram():
from engines.omnivoice_gguf import hardware_probe
fake_torch = _make_torch_mock(
cuda_available=True,
total_vram_bytes=16 * 1024 * 1024 * 1024,
)
with patch.dict("sys.modules", {"torch": fake_torch}):
caps = hardware_probe.detect_capabilities()
assert caps.backend == "cuda"
assert caps.vram_mb == 16 * 1024
assert caps.compute_class == "high-vram"
def test_cuda_4gb_returns_mid_vram():
from engines.omnivoice_gguf import hardware_probe
fake_torch = _make_torch_mock(
cuda_available=True,
total_vram_bytes=4 * 1024 * 1024 * 1024,
)
with patch.dict("sys.modules", {"torch": fake_torch}):
caps = hardware_probe.detect_capabilities()
assert caps.backend == "cuda"
assert caps.vram_mb == 4 * 1024
assert caps.compute_class == "mid-vram"
def test_cuda_1_5gb_returns_low_vram():
from engines.omnivoice_gguf import hardware_probe
fake_torch = _make_torch_mock(
cuda_available=True,
# 1.5 GB → 1536 MB ≥ 1000 threshold but < 4000.
total_vram_bytes=int(1.5 * 1024 * 1024 * 1024),
)
with patch.dict("sys.modules", {"torch": fake_torch}):
caps = hardware_probe.detect_capabilities()
assert caps.backend == "cuda"
assert caps.vram_mb == 1536
assert caps.compute_class == "low-vram"
def test_mps_uses_half_of_system_ram_as_ceiling():
"""MPS unified memory: effective ceiling is half of system RAM."""
from engines.omnivoice_gguf import hardware_probe
import types
fake_torch = _make_torch_mock(mps_available=True)
# 32 GB system RAM → 16 GB effective MPS VRAM → high-vram bucket.
fake_vmem = types.SimpleNamespace(total=32 * 1024 * 1024 * 1024)
fake_psutil = types.SimpleNamespace(virtual_memory=lambda: fake_vmem)
with patch.dict("sys.modules", {"torch": fake_torch, "psutil": fake_psutil}):
caps = hardware_probe.detect_capabilities()
assert caps.backend == "mps"
assert caps.vram_mb == 16 * 1024
assert caps.compute_class == "high-vram"
def test_cpu_only_returns_cpu_class():
from engines.omnivoice_gguf import hardware_probe
fake_torch = _make_torch_mock(cuda_available=False, mps_available=False)
with patch.dict("sys.modules", {"torch": fake_torch}):
caps = hardware_probe.detect_capabilities()
assert caps.backend == "cpu"
assert caps.vram_mb == 0
assert caps.compute_class == "cpu"
def test_bucket_thresholds_directly():
from engines.omnivoice_gguf.hardware_probe import _bucket
assert _bucket(0) == "cpu"
assert _bucket(999) == "cpu"
assert _bucket(1_000) == "low-vram"
assert _bucket(3_999) == "low-vram"
assert _bucket(4_000) == "mid-vram"
assert _bucket(11_999) == "mid-vram"
assert _bucket(12_000) == "high-vram"
assert _bucket(80_000) == "high-vram"
def test_detect_capabilities_reexported_from_gpu_sandbox():
"""Single-entry-point invariant: services.gpu_sandbox.detect_capabilities
must resolve to the same function exported from
engines.omnivoice_gguf.hardware_probe (per RESEARCH.md
"Architectural Responsibility Map")."""
from services import gpu_sandbox
from engines.omnivoice_gguf import hardware_probe
assert gpu_sandbox.detect_capabilities is hardware_probe.detect_capabilities
assert gpu_sandbox.HardwareCapabilities is hardware_probe.HardwareCapabilities
assert gpu_sandbox.ComputeClass is hardware_probe.ComputeClass