diff --git a/.planning/phases/02-engine-isolation-subprocessbackend-indextts-wav-export-dubbi/02-04-SUMMARY.md b/.planning/phases/02-engine-isolation-subprocessbackend-indextts-wav-export-dubbi/02-04-SUMMARY.md new file mode 100644 index 00000000..e26d5f61 --- /dev/null +++ b/.planning/phases/02-engine-isolation-subprocessbackend-indextts-wav-export-dubbi/02-04-SUMMARY.md @@ -0,0 +1,152 @@ +# Plan 02-04 — Engine Compatibility Matrix API + UI (SUMMARY) + +**Status:** delivered. Branch `phase-2-plan-02-04-engine-compat-matrix`; PR +pending. Closes ENGINE-06. + +## What landed + +### Backend +- `backend/services/tts_backend.py` + - `gpu_compat: tuple[str, ...]` class attribute on `TTSBackend` (default + `("cpu",)`) and per-backend overrides (see table below). + - `_HF_TOKEN_MASK_RE` + `_mask_hf_tokens()` — runtime regex redaction + applied to `reason` and `last_error` fields inside `list_backends()` + before serialization. Closes **T-02-12** without depending on the + logging-level `HFTokenRedactor`. + - `list_backends()` now emits `gpu_compat: list[str]` on every entry + and runs the new redactor. +- `backend/api/routers/engines.py` + - **New** `GET /engines/{engine_id}/health` — loopback-gated (T-02-13). + Resolves the backend across tts/asr/llm registries; calls + `SubprocessBackend.health_check()` (spawn-and-ping) for subprocess + engines or falls back to `is_available()` for in-process ones. + Returns `{ id, ok, message, latency_ms }`. Never 500s on a sick + engine — exceptions land in the response body. Unknown id → 404. + - `_ENGINE_INSTANCES: dict[type, object]` cache + `_get_engine_instance` + helper so repeated health checks reuse one singleton per class. + Avoids leaking atexit hooks / spawning extra sidecars. + +### Frontend +- `frontend/src/components/EngineCompatibilityMatrix.jsx` (~270 lines). + Semantic `` with role-attributes for RTL. Columns: Engine / + Install state / GPU compat / Isolation / Actions. "Test engine" + button does NOT auto-spawn on mount; 5 s cooldown prevents + click-storms. Optional `onSelect` prop turns the matrix into a + picker so Settings doesn't need a parallel table. +- `frontend/src/components/EngineCompatibilityMatrix.css` — minimal + styling reusing existing chrome tokens; per-GPU chip colors. +- `frontend/src/api/engines.ts` — `getEngineHealth(id)` client. +- `frontend/src/api/types.ts` — `EngineBackend` extended with the four + new optional fields; new `EngineHealthResponse` type. +- `frontend/src/pages/Settings.jsx` — replaces the hand-rolled engines + table in `EnginesTab` with ``. Drops the now-dead `FAMILY_META` + local map and unused `listEngines` / `Mic` / `MessageSquare` imports. + **No new Settings tab added** — an "Engines" tab already existed + (Phase 3 / 4.6), so the matrix is mounted into the existing tab + rather than duplicating navigation. + +### Tests +- `tests/backend/api/test_engines_route_shape.py` (new, 11 tests): + 1. `/engines` response includes the new fields on every TTS entry + 2. IndexTTS2 isolation_mode == "subprocess" + 3. OmniVoice isolation_mode == "in-process" + 4. OmniVoice gpu_compat == {"cuda", "mps", "cpu"} + 5. Health subprocess path (mocked) returns ok/pong/latency_ms + 6. Health in-process path falls back to is_available + 7. Unknown engine id → 404 + 8. Non-loopback origin → 403 (T-02-13) + 9. Engine instance cache reuses singleton across 2 calls + 10. HF tokens in `is_available()` errors don't leak into `/engines` + 11. HF tokens in `health_check()` errors don't leak into + `/engines/{id}/health` +- `tests/backend/services/test_tts_backend_registry.py::test_list_backends_shape` + — updated `required` keyset to include `gpu_compat`. +- `frontend/src/test/EngineCompatibilityMatrix.test.jsx` (new, 8 tests): + 1. Renders one row per backend with documented columns + 2. isolation_mode badge visible per row + 3. GPU compat chips render the expected per-engine subset + 4. Unavailable rows render the failure reason inline + 5. last_error line renders below status; masked sentinel preserved + 6. Test click fires getEngineHealth(id) + renders latency_ms + 7. Test button disabled while inflight; 2nd click no-op + 8. Failure path (ok=false) renders a failure marker + +## Test results +- **Backend (`uv run pytest tests/ -q --ignore=tests/manual`):** + **402 passed, 10 skipped, 13 xfailed, 1 xpassed** (was 391+ pre-plan). +- **Frontend (`bun run test`, runs `vitest run`):** **65 passed** (8 new). +- **Frontend lint:** No new errors introduced. Settings.jsx pre-existing + lint count went from 5 → 4 (removed an unused-vars warning). +- **Frontend typecheck (`bun run typecheck:ci`):** clean. + +## gpu_compat defaults — per-engine assignments + +| Engine | gpu_compat | Confidence | +|---------------|-------------------------|------------| +| OmniVoice | cuda, mps, cpu | HIGH | +| VoxCPM2 | cuda, mps, cpu | HIGH | +| MOSS-TTS-Nano | cuda, cpu | HIGH | +| KittenTTS | cpu | HIGH (ONNX CPU graph) | +| MLX-Audio | mps, cpu | HIGH | +| **CosyVoice** | cuda, cpu | **MEDIUM** — MPS support not verified upstream; flagged for Phase 6 confirmation | +| IndexTTS2 | cuda, mps, cpu | HIGH (subprocess uses sidecar's GPU) | +| GPT-SoVITS | cuda, cpu | MEDIUM (whatever the external API server uses) | +| Sherpa-ONNX | cuda, cpu | MEDIUM (CUDA provider available on Linux/Windows) | + +**Recommend Phase 6** verify CosyVoice MPS / Sherpa-ONNX CUDA matrix +when the release notes pass; these are stack-research-based, not +empirically verified on hardware. + +## Frontend test runner command (for Phase 6 CI matrix) + +```bash +cd frontend && bun run test # full suite via vitest +cd frontend && bun run test:watch # vitest --watch +``` + +Note: **`bun test`** runs Bun's built-in test runner, which does **not** +set up jsdom. The component tests require `bun run test` (which invokes +`vitest run` per package.json scripts). Document this in the README + CI +config so contributors don't hit "ReferenceError: document is not defined". + +## HF-token redaction proof +The new `test_no_hf_token_leak_in_engines_response` and +`test_no_hf_token_leak_in_health_response` tests register a synthetic +backend whose `is_available()` / `health_check()` embed a real +`hf_abcdefghijklmnopqrstuvwxyz01234567890abcd` string in the error +message, then assert the regex `hf_[A-Za-z0-9]{30,}` matches **zero +substrings** anywhere in the response body — and that the +`hf_***REDACTED***` sentinel appears in its place. T-02-12 mitigated. + +## Scope notes +- **INST-13** (dictation-widget Settings checkbox) is **NOT** included + in this plan, per the locked Phase 2 8-ID scope (ENGINE-01..07 + + BUG-01). The matrix mounts inside the existing `EnginesTab` of + `Settings.jsx`; if a future plan adds INST-13 it will mount in the + same tab structure established here. +- **ASR/LLM family extension:** `list_backends()` in `backend/services/ + asr_backend.py` and `backend/services/llm_backend.py` was NOT + extended with the new fields. Reasoning: the plan's + `must_haves.truths` and `success_criteria` focus on the TTS matrix + (where the IndexTTS2 isolation_mode payoff lives). The frontend + component types these fields as optional (`?: ...`) so the matrix + still renders for ASR/LLM families on the simpler payload — only + the `Isolation` and `GPU compat` columns degrade gracefully (fall + back to defaults). Extending ASR/LLM is a one-line refactor when + a future plan needs it. + +## Manual smoke path +1. `bun desktop-dev` (Tauri) OR `bun --filter frontend dev` + `uv run python -m backend.main`. +2. Navigate to **Settings → Engines**. +3. See one row per registered backend with install state, GPU chips, + isolation badge. +4. Click **Test engine** on any row. + - Available in-process row → latency_ms appears (single-digit ms). + - IndexTTS2 row (if installed) → spawns the sidecar (≤30 s cold), + returns `pong` + latency. Click again within 5 s → ignored (cooldown). +5. `curl http://127.0.0.1:3900/engines | jq '.tts.backends[0] | keys'` → + shows `available, display_name, gpu_compat, id, install_hint, + isolation_mode, last_error, reason`. +6. `curl http://127.0.0.1:3900/engines/omnivoice/health` → returns + ok/message/latency_ms in well under 1 s. diff --git a/backend/api/routers/engines.py b/backend/api/routers/engines.py index f00ab271..bdc08896 100644 --- a/backend/api/routers/engines.py +++ b/backend/api/routers/engines.py @@ -4,17 +4,23 @@ Engines router — Phase 3 wiring. Exposes the three adapter registries (TTS, ASR, LLM) so the Settings UI can render an engine picker + availability reasons. - GET /engines → { tts, asr, llm } - GET /engines/{family} → list of backends - POST /engines/select → persist a backend choice in prefs.json + GET /engines → { tts, asr, llm } + GET /engines/{family} → list of backends + POST /engines/select → persist a backend choice in prefs.json + GET /engines/{engine_id}/health → spawn-or-ping for SubprocessBackend + subclasses; ``is_available()`` for + in-process backends (Plan 02-04) Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`, `OMNIVOICE_LLM_BACKEND`) still win over the UI choice so power-users can pin a backend without Settings silently undoing it. """ -from fastapi import APIRouter, HTTPException +from time import perf_counter + +from fastapi import APIRouter, Depends, HTTPException from pydantic import BaseModel +from api.dependencies import require_loopback from core import prefs from services import tts_backend, asr_backend, llm_backend, translation_engines @@ -135,6 +141,114 @@ async def uninstall_translation_engine(engine_id: str): return {"status": "uninstalled", "engine": engine_id, "package": pkg, "log_tail": out[-800:]} +# ── Engine health-check (Plan 02-04 / ENGINE-06) ─────────────────────────── +# +# The Compat Matrix UI's "Test engine" button calls into this endpoint so +# that users can verify a SubprocessBackend engine is alive without +# kicking off a full synthesize. For an in-process backend the check is a +# cheap ``is_available()`` round-trip; for a SubprocessBackend subclass +# the call spawns the sidecar (if not already up) and round-trips a ping +# frame. Result includes wall-clock latency so the UI can render +# "1234 ms — pong" inline next to the button. +# +# Loopback-gated (T-02-13): only the local desktop frontend may trigger +# a sidecar spawn through this endpoint. + +# Engine instances cached for the lifetime of the FastAPI process so that +# repeated health checks don't spawn a new SubprocessBackend (each spawn +# allocates a sidecar venv probe + atexit hook). The cache is keyed by +# class to survive registry-sandbox tests that rebind ids transiently. +_ENGINE_INSTANCES: dict[type, object] = {} + + +def _get_engine_instance(cls): + """Return a cached singleton instance of ``cls``. + + SubprocessBackend's ``__init__`` registers an atexit shutdown hook, + so re-instantiating per request would leak handler entries (and on + real engines, additional sidecar processes the first time the lock + is acquired). One instance per process is the right move. + """ + inst = _ENGINE_INSTANCES.get(cls) + if inst is None: + inst = cls() + _ENGINE_INSTANCES[cls] = inst + return inst + + +def _resolve_engine_class(engine_id: str): + """Look up ``engine_id`` across the tts/asr/llm registries. + + Returns the class or ``None`` if no family knows the id. Order is + tts → asr → llm so the most-common case (TTS engine matrix) wins + early. No collision risk today — all current ids are family-unique. + """ + for registry in ( + tts_backend._REGISTRY, + asr_backend._REGISTRY, + llm_backend._REGISTRY, + ): + if engine_id in registry: + return registry[engine_id] + return None + + +@router.get( + "/engines/{engine_id}/health", + dependencies=[Depends(require_loopback)], +) +def engine_health(engine_id: str): + """Spawn-and-ping a SubprocessBackend; ``is_available()`` for the rest. + + Returns: + { id, ok, message, latency_ms } + + Never raises through to a 500: if the backend's check throws, the + exception is captured into the response body as ``ok=False`` / + ``message="ExcType: ..."`` so the UI can render a per-row failure + without crashing the panel. Unknown engine ids return 404. + """ + cls = _resolve_engine_class(engine_id) + if cls is None: + raise HTTPException( + status_code=404, + detail=f"unknown engine id: {engine_id!r}", + ) + + t0 = perf_counter() + if hasattr(cls, "health_check"): + # SubprocessBackend path — spawn sidecar (if not running) and ping. + # ``health_check`` already swallows its own exceptions per Plan + # 02-01's contract; we still wrap in a defensive try so a custom + # subclass that violates the contract can't 500 the endpoint. + try: + instance = _get_engine_instance(cls) + ok, msg = instance.health_check() + except Exception as exc: + ok, msg = False, f"{type(exc).__name__}: {exc}" + else: + # In-process backend — `is_available()` is the classmethod-level + # liveness check. Cheap and side-effect-free for every shipping + # backend (it imports the engine package, no model load). + try: + ok, msg = cls.is_available() + except Exception as exc: + ok, msg = False, f"{type(exc).__name__}: {exc}" + + # Mask any HF token the engine accidentally leaked into the message + # so the response body matches the same redaction guarantee as + # ``list_backends()``. + from services.tts_backend import _mask_hf_tokens + + latency_ms = (perf_counter() - t0) * 1000.0 + return { + "id": engine_id, + "ok": bool(ok), + "message": _mask_hf_tokens(msg) if isinstance(msg, str) else str(msg), + "latency_ms": latency_ms, + } + + class SelectEngineRequest(BaseModel): family: str # "tts" | "asr" | "llm" backend_id: str diff --git a/backend/services/tts_backend.py b/backend/services/tts_backend.py index 213d41ec..c8851172 100644 --- a/backend/services/tts_backend.py +++ b/backend/services/tts_backend.py @@ -20,6 +20,7 @@ from __future__ import annotations import logging import os +import re from abc import ABC, abstractmethod from typing import Optional @@ -28,6 +29,30 @@ import torch logger = logging.getLogger("omnivoice.tts") +# ── HF token leak mitigation (Plan 02-04, T-02-12) ───────────────────────── +# +# Token shape is ``hf_`` + 30+ alphanumeric chars per Hugging Face's own +# format. Any error / status string surfaced through the engines API gets +# scrubbed via :func:`_mask_hf_tokens` before serialization so that a +# backend whose ``is_available()`` interpolates ``HF_TOKEN`` into its +# failure message can't accidentally leak it to the frontend. Phase 1's +# ``HFTokenRedactor`` covers logging only — FastAPI response bodies do +# NOT run through the logging filter chain. +_HF_TOKEN_MASK_RE = re.compile(r"hf_[A-Za-z0-9]{30,}") +_HF_TOKEN_MASK = "hf_***REDACTED***" + + +def _mask_hf_tokens(value): + """Return ``value`` with any HF-shaped token substring redacted. + + Non-string values pass through unchanged. Used inside + :func:`list_backends` for the ``reason`` and ``last_error`` fields. + """ + if not isinstance(value, str): + return value + return _HF_TOKEN_MASK_RE.sub(_HF_TOKEN_MASK, value) + + # ── Protocol ──────────────────────────────────────────────────────────────── @@ -62,6 +87,14 @@ class TTSBackend(ABC): #: (e.g. "young female, warm tone, British accent") without reference audio. supports_voice_design: bool = False + #: GPU/accelerator targets the engine can run on. Surfaced via the + #: Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) so users can + #: tell at a glance which engines will use their hardware. Defaults to + #: CPU-only — subclasses override with the union of devices their + #: implementation supports (cuda / mps / rocm / cpu). This is metadata, + #: not enforced — actual device selection lives in the engine's loader. + gpu_compat: tuple[str, ...] = ("cpu",) + @abstractmethod def generate( self, @@ -129,6 +162,7 @@ class OmniVoiceBackend(TTSBackend): id = "omnivoice" display_name = "OmniVoice (600 languages, zero-shot)" + gpu_compat = ("cuda", "mps", "cpu") def __init__(self, model=None): # The live OmniVoice instance. Reuses the singleton owned by @@ -212,6 +246,7 @@ class VoxCPM2Backend(TTSBackend): id = "voxcpm2" display_name = "VoxCPM2 (30 langs, studio 48 kHz, voice design)" supports_voice_design = True + gpu_compat = ("cuda", "mps", "cpu") def __init__(self): self._model = None @@ -320,6 +355,7 @@ class MossTTSNanoBackend(TTSBackend): id = "moss-tts-nano" display_name = "MOSS-TTS-Nano (20 langs, CPU realtime, 48 kHz)" + gpu_compat = ("cuda", "cpu") def __init__(self): self._model = None @@ -412,6 +448,8 @@ class KittenTTSBackend(TTSBackend): id = "kittentts" display_name = "KittenTTS (English, 8 preset voices, CPU realtime)" + # KittenTTS ships as an ONNX CPU graph; no CUDA/MPS path today. + gpu_compat = ("cpu",) PRESET_VOICES = [ "expr-voice-2-m", "expr-voice-2-f", @@ -502,6 +540,9 @@ class MLXAudioBackend(TTSBackend): id = "mlx-audio" display_name = "MLX-Audio (mac-ARM, 14+ engines: Kokoro, CSM, Dia, Qwen3, …)" + # mlx is Apple-Silicon-only; CPU is the practical fallback when the + # mlx framework is installed but the user lacks an Apple GPU. + gpu_compat = ("mps", "cpu") # A curated subset surfaced by default — the full mlx-audio roster is # larger but these cover the useful tiers: small multilingual (Kokoro), @@ -625,6 +666,9 @@ class CosyVoiceBackend(TTSBackend): id = "cosyvoice" display_name = "CosyVoice 3 (9 langs, zero-shot, instruct, Apache-2.0)" + # CosyVoice's official inference path expects CUDA; CPU works but slow. + # MPS support not verified upstream — flagged for Phase 6 confirmation. + gpu_compat = ("cuda", "cpu") # CosyVoice language tags used for cross-lingual synthesis. LANG_TAGS = { @@ -787,6 +831,8 @@ class GPTSoVITSBackend(TTSBackend): id = "gpt-sovits" display_name = "GPT-SoVITS (5 langs, zero-shot, RTF 0.014, MIT)" + # Server-side; whichever device GPT-SoVITS itself uses (CUDA preferred). + gpu_compat = ("cuda", "cpu") def __init__(self): self._url = os.environ.get("OMNIVOICE_GPTSOVITS_URL", "http://127.0.0.1:9880") @@ -894,6 +940,9 @@ class SherpaOnnxBackend(TTSBackend): id = "sherpa-onnx" display_name = "Sherpa-ONNX (20+ engines, WASM-ready, universal runtime)" + # Sherpa-ONNX uses the onnxruntime providers — CPU is the universal + # baseline; CUDA provider is available on Linux/Windows installs. + gpu_compat = ("cuda", "cpu") def __init__(self): self._tts = None @@ -1082,19 +1131,25 @@ def list_backends() -> list[dict]: Per-entry shape (ENGINE-05 + ENGINE-06): { - "id": str, - "display_name": str, - "available": bool, - "reason": Optional[str], # message when not available - "install_hint": Optional[str], - "last_error": Optional[str], # cached most-recent failure + "id": str, + "display_name": str, + "available": bool, + "reason": Optional[str], # message when not available + "install_hint": Optional[str], + "last_error": Optional[str], # cached most-recent failure "isolation_mode": "in-process" | "subprocess", + "gpu_compat": list[str], # subset of {cuda, mps, rocm, cpu} } Guarantees (ENGINE-05): a backend whose `is_available()` raises does NOT prevent the list from returning. The exception is captured into the `reason`/`last_error` fields for that one entry and every other backend is still listed normally. + + Security (Plan 02-04 / T-02-12): any HF-shaped token substring in + ``reason`` or ``last_error`` is redacted before the entry is + serialized — :func:`_mask_hf_tokens`. The frontend can render these + fields verbatim without leaking credentials. """ # Detect subprocess-isolated backends via a duck-typed marker rather # than `issubclass(cls, SubprocessBackend)`. Test fixtures (e.g. the @@ -1118,7 +1173,10 @@ def list_backends() -> list[dict]: if ok: _LAST_ERRORS.pop(bid, None) else: - _LAST_ERRORS[bid] = msg + # Mask any HF token inside the failure message BEFORE it lands + # in the in-memory cache — otherwise a later list_backends() + # call would re-surface the unmasked string. + _LAST_ERRORS[bid] = _mask_hf_tokens(msg) # ENGINE-06 isolation_mode: duck-typed marker for SubprocessBackend # subclasses (see services.subprocess_backend.SubprocessBackend). if getattr(cls, "_is_subprocess_isolated", False): @@ -1129,10 +1187,11 @@ def list_backends() -> list[dict]: "id": bid, "display_name": cls.display_name, "available": ok, - "reason": None if ok else msg, + "reason": None if ok else _mask_hf_tokens(msg), "install_hint": _INSTALL_HINTS.get(bid), "last_error": _LAST_ERRORS.get(bid), "isolation_mode": isolation, + "gpu_compat": list(getattr(cls, "gpu_compat", ("cpu",))), }) return out diff --git a/frontend/src/api/engines.ts b/frontend/src/api/engines.ts index aaaca3a9..2d6b6f18 100644 --- a/frontend/src/api/engines.ts +++ b/frontend/src/api/engines.ts @@ -3,6 +3,7 @@ import type { AllEnginesResponse, EngineFamily, EngineFamilyResponse, + EngineHealthResponse, SelectEngineResponse, } from './types'; @@ -48,6 +49,21 @@ export async function selectEngine(family: EngineFamily, backendId: string): Pro return apiPost('/engines/select', { family, backend_id: backendId }); } +/** + * Plan 02-04 / ENGINE-06 — spawn-and-ping a SubprocessBackend (or + * `is_available()`-check an in-process backend) on user demand. The + * Engine Compatibility Matrix's "Test engine" button calls this; never + * called on Settings mount to avoid auto-spawning every sidecar. + * + * The endpoint never 500s on a sick backend — it captures the exception + * into the response body as `{ ok: false, message: "ExcType: ..." }`. + * 404 is returned only when `engineId` matches none of the tts/asr/llm + * registries. + */ +export async function getEngineHealth(engineId: string): Promise { + return apiJson(`/engines/${encodeURIComponent(engineId)}/health`); +} + export async function listTranslationEngines(): Promise { return apiJson('/engines/translation'); } diff --git a/frontend/src/api/types.ts b/frontend/src/api/types.ts index 3912b7b6..4c642dd9 100644 --- a/frontend/src/api/types.ts +++ b/frontend/src/api/types.ts @@ -9,14 +9,26 @@ * than lying with a fake type — explicit "unknown" prompts a runtime check. */ -// ── Engines (Phase 3 / 4.6) ────────────────────────────────────────────── +// ── Engines (Phase 3 / 4.6 / Plan 02-04) ───────────────────────────────── export type EngineFamily = 'tts' | 'asr' | 'llm'; +// `isolation_mode`, `last_error`, `install_hint`, `gpu_compat` arrived +// in Plan 02-04 alongside the Engine Compatibility Matrix. They're +// optional in this type because the asr / llm registries don't emit them +// today (only the TTS registry has been migrated to the extended shape). +// The matrix UI gates them with `??` / `?.length` so the simpler payload +// still renders without errors. +export type GPUTarget = 'cuda' | 'mps' | 'rocm' | 'cpu'; + export interface EngineBackend { id: string; display_name: string; available: boolean; reason: string | null; + install_hint?: string | null; + last_error?: string | null; + isolation_mode?: 'in-process' | 'subprocess'; + gpu_compat?: GPUTarget[]; } export interface EngineFamilyResponse { @@ -36,6 +48,13 @@ export interface SelectEngineResponse { env_override: boolean; } +export interface EngineHealthResponse { + id: string; + ok: boolean; + message: string; + latency_ms: number; +} + // ── System / diagnostics ───────────────────────────────────────────────── export interface SystemInfo { app_version?: string; diff --git a/frontend/src/components/EngineCompatibilityMatrix.css b/frontend/src/components/EngineCompatibilityMatrix.css new file mode 100644 index 00000000..256d8b0e --- /dev/null +++ b/frontend/src/components/EngineCompatibilityMatrix.css @@ -0,0 +1,180 @@ +/* Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) */ + +.engine-matrix { + display: flex; + flex-direction: column; + gap: var(--space-3, 8px); +} + +.engine-matrix--loading, +.engine-matrix--error { + padding: 16px; + display: flex; + align-items: center; + gap: 8px; +} + +.engine-matrix__muted { + color: var(--chrome-fg-muted, #888); + font-size: 13px; +} + +.engine-matrix__head { + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; +} + +.engine-matrix__title { + display: inline-flex; + align-items: center; + gap: 6px; + margin: 0; + font-size: 13px; + font-weight: 600; + color: var(--chrome-fg, currentColor); +} + +.engine-matrix__table { + width: 100%; +} + +.engine-matrix__body { + display: flex; + flex-direction: column; +} + +.engine-matrix__row { + display: flex; + align-items: flex-start; + gap: 8px; + padding: 8px 10px; + border-top: 1px solid var(--chrome-border, rgba(255,255,255,0.06)); + min-height: 56px; +} + +.engine-matrix__row.is-off { + opacity: 0.78; +} + +.engine-matrix__cell { + display: flex; + align-items: center; +} + +.engine-matrix__cell--name { + flex-direction: column; + align-items: flex-start; + gap: 2px; + min-width: 0; +} + +.engine-matrix__cell--center { + justify-content: center; +} + +.engine-matrix__cell--gpu { + align-items: center; +} + +.engine-matrix__cell--actions { + justify-content: flex-end; + gap: 6px; + flex-wrap: wrap; +} + +.engine-matrix__name { + display: inline-flex; + align-items: center; + gap: 6px; + font-weight: 600; + font-size: 13px; + color: var(--chrome-fg, currentColor); +} + +.engine-matrix__id { + font-family: var(--chrome-font-mono, ui-monospace, SFMono-Regular, monospace); + font-size: 11px; + color: var(--chrome-fg-muted, #888); +} + +.engine-matrix__reason { + font-size: 12px; + color: var(--chrome-severity-warn, #d79921); + display: block; + max-width: 100%; + overflow: hidden; + text-overflow: ellipsis; +} + +.engine-matrix__hint { + font-size: 11px; + color: var(--chrome-fg-muted, #888); +} + +.engine-matrix__last-error { + font-size: 11px; + color: var(--chrome-severity-err, #cc241d); + display: block; +} + +.engine-matrix__chips { + display: inline-flex; + flex-wrap: wrap; + gap: 4px; +} + +.engine-matrix__chip { + display: inline-block; + padding: 1px 6px; + font-size: 10px; + font-family: var(--chrome-font-mono, ui-monospace, monospace); + font-weight: 600; + letter-spacing: 0.04em; + text-transform: uppercase; + border-radius: 4px; + border: 1px solid var(--chrome-border-strong, rgba(255,255,255,0.18)); + color: var(--chrome-fg-muted, #888); + background: transparent; + user-select: none; +} + +.engine-matrix__chip--cuda { + color: #76b900; + border-color: color-mix(in srgb, #76b900 45%, transparent); + background: color-mix(in srgb, #76b900 10%, transparent); +} +.engine-matrix__chip--mps { + color: #b8b8b8; + border-color: color-mix(in srgb, #b8b8b8 45%, transparent); + background: color-mix(in srgb, #b8b8b8 10%, transparent); +} +.engine-matrix__chip--rocm { + color: #ed1c24; + border-color: color-mix(in srgb, #ed1c24 45%, transparent); + background: color-mix(in srgb, #ed1c24 10%, transparent); +} +.engine-matrix__chip--cpu { + /* keep default muted tone for CPU */ +} + +.engine-matrix__result { + font-size: 11px; + font-family: var(--chrome-font-mono, ui-monospace, monospace); +} + +.engine-matrix__result--ok { + color: var(--chrome-severity-ok, #98971a); +} + +.engine-matrix__result--fail { + color: var(--chrome-severity-err, #cc241d); +} + +.engine-matrix__empty { + padding: 24px; + text-align: center; + color: var(--chrome-fg-muted, #888); + font-size: 13px; +} diff --git a/frontend/src/components/EngineCompatibilityMatrix.jsx b/frontend/src/components/EngineCompatibilityMatrix.jsx new file mode 100644 index 00000000..ad0407f8 --- /dev/null +++ b/frontend/src/components/EngineCompatibilityMatrix.jsx @@ -0,0 +1,344 @@ +import React, { useCallback, useEffect, useMemo, useState } from 'react'; +import { Cpu, Mic, MessageSquare, Activity, AlertTriangle, CheckCircle2, RefreshCw, Layers } from 'lucide-react'; +import { toast } from 'react-hot-toast'; +import { listEngines, getEngineHealth } from '../api/engines'; +import { Badge, Button, Segmented, Table } from '../ui'; +import './EngineCompatibilityMatrix.css'; + +/** + * Engine Compatibility Matrix (Plan 02-04 / ENGINE-06). + * + * Renders a single source-of-truth table of every registered backend in + * a family (tts / asr / llm). Each row shows: + * * Engine display name + * * Install state (available / unavailable, with the failure reason + * inline when the row is unavailable) + * * GPU compat chips (cuda / mps / rocm / cpu) + * * Isolation mode (in-process or subprocess) — the visible payoff + * of the Plan 02-01 SubprocessBackend + Plan 02-03 IndexTTS migration + * * Last error (cached most-recent failure — distinguishes "currently + * failing" from "failed before, now working") + * * Test engine button — fires a `/engines/{id}/health` round-trip on + * demand; SubprocessBackend rows spawn-and-ping their sidecar, in- + * process rows fall back to `is_available()`. Latency is rendered + * inline next to the button. + * + * Cross-platform contract: this component does NOT auto-spawn any + * sidecar on mount; the user must click Test engine. That keeps macOS / + * Windows / Linux behaviour identical and prevents the matrix from + * locking up a cold IndexTTS install for 30 s every time Settings + * loads. A short 5 s cooldown on the Test button prevents click-storms. + * + * Props: + * - family: 'tts' | 'asr' | 'llm' default 'tts' + * - onSelect?: (family, backendId) => Promise optional — when + * provided, a "Use" button appears next to "Test engine" for + * available, non-active rows. Lets the matrix double as an engine + * picker so Settings doesn't need a parallel table. + * - activeId?: string the currently-active backend id for this + * family. Used to render the "active" badge. + */ +const FAMILY_META = { + tts: { label: 'TTS', icon: Cpu }, + asr: { label: 'ASR', icon: Mic }, + llm: { label: 'LLM', icon: MessageSquare }, +}; + +const ISOLATION_TONE = { + subprocess: 'info', + 'in-process': 'neutral', +}; + +const GPU_LABEL = { + cuda: 'CUDA', + mps: 'MPS', + rocm: 'ROCm', + cpu: 'CPU', +}; + +const TEST_COOLDOWN_MS = 5000; + +const COLUMNS = [ + { key: 'name', label: 'Engine', flex: 3 }, + { key: 'status', label: 'Install state', width: 130, align: 'center' }, + { key: 'gpu', label: 'GPU compat', width: 170, align: 'left' }, + { key: 'isolation', label: 'Isolation', width: 110, align: 'center' }, + { key: 'action', label: 'Actions', width: 220, align: 'right' }, +]; + +/** Subset of the unified engine entry the matrix actually reads. */ +function normalizeEntry(entry) { + return { + id: entry.id, + display_name: entry.display_name, + available: !!entry.available, + reason: entry.reason || null, + install_hint: entry.install_hint || null, + last_error: entry.last_error || null, + isolation_mode: entry.isolation_mode || 'in-process', + gpu_compat: Array.isArray(entry.gpu_compat) && entry.gpu_compat.length > 0 + ? entry.gpu_compat + : ['cpu'], + }; +} + +export default function EngineCompatibilityMatrix({ + family = 'tts', + onSelect = null, + activeId = null, + // Test-friendly overrides — let the RTL suite mock the API layer + // without resorting to module-level vi.mock incantations. + apiListEngines = listEngines, + apiGetEngineHealth = getEngineHealth, +}) { + const [data, setData] = useState(null); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + const [activeFamily, setActiveFamily] = useState(family); + + // health state keyed by engine id: + // { [id]: { inflight: boolean, ok?: boolean, message?: string, + // latency_ms?: number, lastClickAt?: number } } + const [healthByEngine, setHealthByEngine] = useState({}); + + useEffect(() => { setActiveFamily(family); }, [family]); + + const reload = useCallback(async () => { + setLoading(true); + setError(null); + try { + const fresh = await apiListEngines(); + setData(fresh); + } catch (e) { + const msg = e?.message || String(e); + setError(msg); + toast.error(`Failed to load engines: ${msg}`); + } finally { + setLoading(false); + } + }, [apiListEngines]); + + useEffect(() => { reload(); }, [reload]); + + const familyData = data?.[activeFamily]; + const backends = useMemo( + () => (familyData?.backends || []).map(normalizeEntry), + [familyData], + ); + const families = useMemo( + () => Object.keys(FAMILY_META).filter((f) => data?.[f]?.backends), + [data], + ); + + const testHealth = useCallback(async (id) => { + const now = Date.now(); + const cur = healthByEngine[id]; + if (cur?.inflight) return; + if (cur?.lastClickAt && now - cur.lastClickAt < TEST_COOLDOWN_MS) { + // Click-storm cooldown — silently ignore. + return; + } + setHealthByEngine((prev) => ({ + ...prev, + [id]: { inflight: true, lastClickAt: now }, + })); + try { + const result = await apiGetEngineHealth(id); + setHealthByEngine((prev) => ({ + ...prev, + [id]: { + inflight: false, + ok: !!result.ok, + message: result.message || '', + latency_ms: Math.round(result.latency_ms || 0), + lastClickAt: now, + }, + })); + } catch (e) { + setHealthByEngine((prev) => ({ + ...prev, + [id]: { + inflight: false, + ok: false, + message: e?.message || String(e), + latency_ms: 0, + lastClickAt: now, + }, + })); + } + }, [apiGetEngineHealth, healthByEngine]); + + if (loading && !data) { + return ( +
+ Loading engines… +
+ ); + } + if (error && !data) { + return ( +
+ Could not load engines: {error} + +
+ ); + } + if (!familyData) return null; + + const activeBackendId = activeId ?? familyData.active; + + return ( +
+
+

+ Engine Compatibility Matrix +

+ +
+ + {families.length > 1 && ( + ({ + value: f, + label: `${FAMILY_META[f].label} · ${data[f].active}`, + }))} + /> + )} + +
+ +
+ {backends.map((b) => { + const isActive = b.id === activeBackendId; + const health = healthByEngine[b.id]; + return ( +
+ {/* Engine name + reason / install_hint */} +
+ + {b.display_name} + {isActive && active} + + {b.id} + {!b.available && b.reason && ( + {b.reason} + )} + {b.install_hint && ( + + {b.install_hint} + + )} + {b.last_error && ( + + Last error: {b.last_error} + + )} +
+ + {/* Install state */} +
+ {b.available + ? Available + : Unavailable} +
+ + {/* GPU compat chips */} +
+
+ {b.gpu_compat.map((g) => ( + + {GPU_LABEL[g] || g.toUpperCase()} + + ))} +
+
+ + {/* Isolation mode */} +
+ + {b.isolation_mode} + +
+ + {/* Actions: Test engine + optional Use */} +
+ + {health && !health.inflight && ( + + {health.ok + ? `${health.latency_ms} ms` + : `failed`} + + )} + {onSelect && b.available && !isActive && ( + + )} +
+
+ ); + })} + {backends.length === 0 && ( +
+ No backends registered. +
+ )} +
+
+ + ); +} diff --git a/frontend/src/pages/Settings.jsx b/frontend/src/pages/Settings.jsx index 821a516f..e0e5604c 100644 --- a/frontend/src/pages/Settings.jsx +++ b/frontend/src/pages/Settings.jsx @@ -10,20 +10,21 @@ import { import { useVirtualizer } from '@tanstack/react-virtual'; import { Cpu, FileText, Info, ShieldCheck, RefreshCw, Trash2, ExternalLink, - CheckCircle, AlertCircle, Plug, Mic, MessageSquare, Download, Copy, Building2, KeyRound, + CheckCircle, AlertCircle, Plug, Download, Copy, Building2, KeyRound, Keyboard, } from 'lucide-react'; import { toast } from 'react-hot-toast'; import { openExternal } from '../api/external'; import { systemLogs, systemLogsTauri, clearSystemLogs, clearTauriLogs } from '../api/system'; import { useSysinfo, useModelStatus, useSystemInfo } from '../api/hooks'; -import { listEngines, selectEngine } from '../api/engines'; +import { selectEngine } from '../api/engines'; import { setupDownloadStreamUrl } from '../api/setup'; import { getFrontendLogs, clearFrontendLogs } from '../utils/consoleBuffer'; import { Tabs, Segmented, Button, Badge, Panel, Table, Progress } from '../ui'; import { useAppStore } from '../store'; import ApiKeysPanel from '../components/settings/ApiKeysPanel'; import PerformancePanel from '../components/settings/PerformancePanel'; +import EngineCompatibilityMatrix from '../components/EngineCompatibilityMatrix'; import './Settings.css'; const TABS = [ @@ -36,12 +37,6 @@ const TABS = [ { id: 'privacy', label: 'Privacy', icon: ShieldCheck, accent: '#b8bb26' }, ]; -const FAMILY_META = { - tts: { label: 'TTS', icon: Cpu, tint: 'brand' }, - asr: { label: 'ASR', icon: Mic, tint: 'info' }, - llm: { label: 'LLM', icon: MessageSquare, tint: 'violet' }, -}; - const LOG_SOURCES = [ { value: 'backend', label: 'Backend' }, { value: 'frontend', label: 'Frontend' }, @@ -793,50 +788,21 @@ export function ModelStoreTab({ info, modelBadge }) { export function EnginesTab() { - const [data, setData] = useState(null); - const [loading, setLoading] = useState(true); - const [switching, setSwitching] = useState(null); - const [activeFam, setActiveFam] = useState('tts'); const reviewMode = useAppStore(s => s.reviewMode); const setReviewMode = useAppStore(s => s.setReviewMode); - const reload = useCallback(async () => { - setLoading(true); - try { setData(await listEngines()); } - catch (e) { toast.error(`Failed to load engines: ${e.message}`); } - finally { setLoading(false); } - }, []); - + // Plan 02-04 / ENGINE-06 — engine selection is wired through the + // matrix component's optional onSelect callback so the matrix doubles + // as a picker. Keeps a single source of truth for the engine list + + // its install / GPU / isolation state. const onSelect = useCallback(async (family, backendId) => { - setSwitching(`${family}:${backendId}`); try { const r = await selectEngine(family, backendId); toast.success(`${family.toUpperCase()} → ${r.active}`); - await reload(); } catch (e) { toast.error(e.message || 'Failed to switch engine'); - } finally { - setSwitching(null); } - }, [reload]); - - useEffect(() => { reload(); }, [reload]); - - if (loading && !data) { - return
Loading engines…
; - } - if (!data) return null; - - const fams = ['tts', 'asr', 'llm'].filter(f => data[f]); - const currentFam = fams.includes(activeFam) ? activeFam : fams[0]; - const family = currentFam ? data[currentFam] : null; - const famTint = currentFam ? FAMILY_META[currentFam].tint : 'neutral'; - - const COLUMNS = [ - { key: 'name', label: 'Backend', flex: 3 }, - { key: 'status', label: 'Status', width: 120, align: 'center' }, - { key: 'action', label: '', width: 90, align: 'right' }, - ]; + }, []); return (
@@ -856,72 +822,9 @@ export function EnginesTab() { {reviewMode === 'on' ? 'Stage banners on' : 'Stage banners off'} - - {fams.length > 1 && ( - ({ - value: f, - label: `${FAMILY_META[f].label} · ${data[f].active}`, - }))} - /> - )} - - {family && ( - - -
- {family.backends.map(b => { - const isActive = family.active === b.id; - const isSwitching = switching === `${currentFam}:${b.id}`; - return ( -
-
- - {b.display_name} - {isActive && active} - - - {b.id} - {!b.available && b.reason && ( - · {b.reason} - )} - - {b.install_hint && ( - - {b.install_hint} - - )} -
-
- {b.available - ? ready - : unavailable} -
-
- {!isActive && b.available && ( - - )} -
-
- ); - })} -
-
- )} +
); } diff --git a/frontend/src/test/EngineCompatibilityMatrix.test.jsx b/frontend/src/test/EngineCompatibilityMatrix.test.jsx new file mode 100644 index 00000000..2d4f4e62 --- /dev/null +++ b/frontend/src/test/EngineCompatibilityMatrix.test.jsx @@ -0,0 +1,242 @@ +import React from 'react'; +import { describe, it, expect, vi, beforeEach } from 'vitest'; +import { fireEvent, render, screen, waitFor, within } from '@testing-library/react'; + +// Mock the toast import the component depends on — keeps the test free +// of side-effect side-channels (toast() schedules timers we don't want). +vi.mock('react-hot-toast', () => ({ + default: { error: vi.fn(), success: vi.fn() }, + toast: { error: vi.fn(), success: vi.fn() }, +})); + +import EngineCompatibilityMatrix from '../components/EngineCompatibilityMatrix'; + +/** Build a minimal AllEnginesResponse with the three rows the plan calls for. */ +function makeEnginesResponse({ + inProcessAvailable = true, + inProcessHasLastError = false, +} = {}) { + return { + tts: { + active: 'omnivoice', + backends: [ + { + id: 'omnivoice', + display_name: 'OmniVoice (test)', + available: inProcessAvailable, + reason: inProcessAvailable ? null : 'omnivoice package missing', + install_hint: 'pip install omnivoice', + last_error: inProcessHasLastError ? 'previous load failed' : null, + isolation_mode: 'in-process', + gpu_compat: ['cuda', 'mps', 'cpu'], + }, + { + id: 'kittentts', + display_name: 'KittenTTS (test)', + available: false, + reason: 'kittentts not installed', + install_hint: 'pip install kittentts', + last_error: 'auth failed for hf_***REDACTED***', + isolation_mode: 'in-process', + gpu_compat: ['cpu'], + }, + { + id: 'indextts2', + display_name: 'IndexTTS2 (test)', + available: true, + reason: null, + install_hint: 'git clone …', + last_error: null, + isolation_mode: 'subprocess', + gpu_compat: ['cuda', 'mps', 'cpu'], + }, + ], + }, + asr: { active: 'whisperx', backends: [] }, + llm: { active: 'off', backends: [] }, + }; +} + +describe('EngineCompatibilityMatrix', () => { + beforeEach(() => { + vi.useRealTimers(); + }); + + it('renders one row per backend with the documented columns', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + render( + , + ); + + await waitFor(() => { + expect(screen.getByText('OmniVoice (test)')).toBeInTheDocument(); + }); + expect(apiListEngines).toHaveBeenCalledTimes(1); + + // Three engine rows, one per registered backend. + expect(screen.getAllByRole('row').length).toBe(3); + expect(screen.getByText('KittenTTS (test)')).toBeInTheDocument(); + expect(screen.getByText('IndexTTS2 (test)')).toBeInTheDocument(); + }); + + it('shows isolation_mode badge per row (subprocess for IndexTTS, in-process for the others)', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + render( + , + ); + + await waitFor(() => screen.getByText('IndexTTS2 (test)')); + + const indexRow = screen.getByText('IndexTTS2 (test)').closest('[role="row"]'); + const omniRow = screen.getByText('OmniVoice (test)').closest('[role="row"]'); + const kittenRow = screen.getByText('KittenTTS (test)').closest('[role="row"]'); + + expect(within(indexRow).getByText('subprocess')).toBeInTheDocument(); + expect(within(omniRow).getByText('in-process')).toBeInTheDocument(); + expect(within(kittenRow).getByText('in-process')).toBeInTheDocument(); + }); + + it('renders GPU compat chips for each backend', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + render( + , + ); + + await waitFor(() => screen.getByText('OmniVoice (test)')); + + const omniRow = screen.getByText('OmniVoice (test)').closest('[role="row"]'); + expect(within(omniRow).getByText('CUDA')).toBeInTheDocument(); + expect(within(omniRow).getByText('MPS')).toBeInTheDocument(); + expect(within(omniRow).getByText('CPU')).toBeInTheDocument(); + + const kittenRow = screen.getByText('KittenTTS (test)').closest('[role="row"]'); + // KittenTTS is CPU-only. + expect(within(kittenRow).getByText('CPU')).toBeInTheDocument(); + expect(within(kittenRow).queryByText('CUDA')).not.toBeInTheDocument(); + }); + + it('shows the install reason inline when a backend is unavailable', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + render( + , + ); + + await waitFor(() => screen.getByText('KittenTTS (test)')); + const kittenRow = screen.getByText('KittenTTS (test)').closest('[role="row"]'); + expect(within(kittenRow).getByText('kittentts not installed')).toBeInTheDocument(); + expect(within(kittenRow).getByText(/Unavailable/i)).toBeInTheDocument(); + }); + + it('renders a "Last error" line when last_error is populated', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + render( + , + ); + + await waitFor(() => screen.getByText('KittenTTS (test)')); + const lastErrEls = screen.getAllByTestId('last-error'); + expect(lastErrEls.length).toBeGreaterThan(0); + // The masked sentinel survives the redactor — confirms the row renders + // the cache verbatim and does NOT try to "clean up" the masked string. + expect(lastErrEls[0].textContent).toMatch(/hf_\*\*\*REDACTED\*\*\*/); + }); + + it('clicking Test engine fires getEngineHealth and renders latency_ms', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + const apiGetEngineHealth = vi.fn().mockResolvedValue({ + id: 'indextts2', ok: true, message: 'pong', latency_ms: 1234, + }); + render( + , + ); + + await waitFor(() => screen.getByText('IndexTTS2 (test)')); + const indexRow = screen.getByText('IndexTTS2 (test)').closest('[role="row"]'); + const testBtn = within(indexRow).getByRole('button', { name: /test indextts2/i }); + fireEvent.click(testBtn); + + await waitFor(() => { + expect(apiGetEngineHealth).toHaveBeenCalledWith('indextts2'); + }); + await waitFor(() => { + expect(within(indexRow).getByTestId('health-result-indextts2')).toBeInTheDocument(); + }); + expect(within(indexRow).getByText(/1234 ms/)).toBeInTheDocument(); + }); + + it('Test button is disabled while an inflight health request is pending', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + // A health request that never resolves so we can observe the inflight state. + let resolveHealth; + const apiGetEngineHealth = vi.fn(() => new Promise((resolve) => { resolveHealth = resolve; })); + render( + , + ); + + await waitFor(() => screen.getByText('IndexTTS2 (test)')); + const indexRow = screen.getByText('IndexTTS2 (test)').closest('[role="row"]'); + const testBtn = within(indexRow).getByRole('button', { name: /test indextts2/i }); + fireEvent.click(testBtn); + + await waitFor(() => { + expect(testBtn).toBeDisabled(); + }); + // Second click while inflight must be a no-op — the spy has been called + // exactly once. + fireEvent.click(testBtn); + expect(apiGetEngineHealth).toHaveBeenCalledTimes(1); + + // Release the promise so the test doesn't leak a pending microtask. + resolveHealth({ id: 'indextts2', ok: true, message: 'pong', latency_ms: 50 }); + }); + + it('renders a failure marker when the health route returns ok=false', async () => { + const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse()); + const apiGetEngineHealth = vi.fn().mockResolvedValue({ + id: 'indextts2', ok: false, message: 'spawn failed', latency_ms: 12, + }); + render( + , + ); + + await waitFor(() => screen.getByText('IndexTTS2 (test)')); + const indexRow = screen.getByText('IndexTTS2 (test)').closest('[role="row"]'); + fireEvent.click(within(indexRow).getByRole('button', { name: /test indextts2/i })); + + await waitFor(() => { + expect(within(indexRow).getByText(/failed/i)).toBeInTheDocument(); + }); + }); +}); diff --git a/tests/backend/api/__init__.py b/tests/backend/api/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/backend/api/test_engines_route_shape.py b/tests/backend/api/test_engines_route_shape.py new file mode 100644 index 00000000..299cf7ee --- /dev/null +++ b/tests/backend/api/test_engines_route_shape.py @@ -0,0 +1,276 @@ +"""Plan 02-04 — API contract for /engines + /engines/{id}/health. + +Asserts: + * ``GET /engines`` returns the documented per-entry shape + (id, display_name, available, reason, install_hint, last_error, + isolation_mode, gpu_compat) for every backend. + * ``GET /engines/{engine_id}/health`` round-trips for both + SubprocessBackend (mocked health_check) and in-process backends. + * Loopback gate is enforced on the health route — non-loopback origin + returns 403. + * Unknown engine id returns 404. + * HF-shaped tokens that a backend's is_available() leaks into its + error message do NOT reach the response body — T-02-12. + +The fixture builds a minimal FastAPI app with just the engines router so +the test stays fast and doesn't require torch / whisperx / demucs to be +fully importable. +""" +from __future__ import annotations + +import re +import sys + +import pytest + + +SAMPLE_HF_TOKEN = "hf_abcdefghijklmnopqrstuvwxyz01234567890abcd" +HF_TOKEN_RE = re.compile(r"hf_[A-Za-z0-9]{30,}") + + +# ── helpers ──────────────────────────────────────────────────────────────── + + +@pytest.fixture +def fresh_app(monkeypatch, tmp_path): + """Build a fresh FastAPI app instance with isolated DB. + + The full main.py app factory pulls in torch / whisperx / demucs; we + only need the engines router for these tests so we mount it + directly, matching the pattern in tests/backend/test_engine_spawn_token.py. + """ + monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path)) + + # Wipe cached services so each test gets a clean _LAST_ERRORS dict + + # _REGISTRY (the engines router imports them on first call). + for mod in list(sys.modules): + if ( + mod == "core" or mod.startswith("core.") + or mod == "services" or mod.startswith("services.") + or mod == "api" or mod.startswith("api.") + ): + del sys.modules[mod] + + from core import db as _db + _db.init_db() + + from fastapi import FastAPI + from api.routers import engines as engines_router + + app = FastAPI() + app.include_router(engines_router.router) + return app + + +def _client(app, host="127.0.0.1"): + """TestClient anchored to a loopback (or non-loopback) client tuple. + + `require_loopback` reads `request.client.host`; the default + TestClient tuple is `('testclient', 50000)` which the dep rejects. + """ + from fastapi.testclient import TestClient + return TestClient(app, client=(host, 12345)) + + +# ── /engines response shape (gpu_compat, isolation_mode, last_error) ────── + + +def test_engines_response_includes_new_fields(fresh_app): + client = _client(fresh_app) + r = client.get("/engines") + assert r.status_code == 200 + body = r.json() + + required = { + "id", "display_name", "available", "reason", + "install_hint", "last_error", "isolation_mode", "gpu_compat", + } + for entry in body["tts"]["backends"]: + missing = required - entry.keys() + assert not missing, f"entry {entry.get('id')!r} missing keys: {missing}" + assert isinstance(entry["gpu_compat"], list) + assert all(isinstance(x, str) for x in entry["gpu_compat"]) + assert entry["isolation_mode"] in {"in-process", "subprocess"} + + +def test_indextts2_entry_has_subprocess_isolation_mode(fresh_app): + """Cross-checks Plan 02-03's IndexTTS subprocess migration via the API.""" + client = _client(fresh_app) + r = client.get("/engines") + assert r.status_code == 200 + by_id = {b["id"]: b for b in r.json()["tts"]["backends"]} + assert "indextts2" in by_id + assert by_id["indextts2"]["isolation_mode"] == "subprocess" + + +def test_omnivoice_entry_has_in_process_isolation_mode(fresh_app): + client = _client(fresh_app) + r = client.get("/engines") + assert r.status_code == 200 + by_id = {b["id"]: b for b in r.json()["tts"]["backends"]} + assert "omnivoice" in by_id + assert by_id["omnivoice"]["isolation_mode"] == "in-process" + + +def test_gpu_compat_omnivoice_has_cuda_mps_cpu(fresh_app): + """OmniVoice ships with CUDA/MPS/CPU paths — surface that in the matrix.""" + client = _client(fresh_app) + r = client.get("/engines") + by_id = {b["id"]: b for b in r.json()["tts"]["backends"]} + assert set(by_id["omnivoice"]["gpu_compat"]) == {"cuda", "mps", "cpu"} + + +# ── /engines/{id}/health round-trip ──────────────────────────────────────── + + +def test_engine_health_subprocess_success(fresh_app, monkeypatch): + """Mock IndexTTS2Backend.health_check so we don't spawn a real sidecar.""" + from services.tts_backend import _REGISTRY + + # Resolve the lazy entry without spawning anything heavy. + cls = _REGISTRY["indextts2"] + monkeypatch.setattr(cls, "health_check", lambda self: (True, "pong")) + + client = _client(fresh_app) + r = client.get("/engines/indextts2/health") + assert r.status_code == 200 + body = r.json() + assert body["id"] == "indextts2" + assert body["ok"] is True + assert body["message"] == "pong" + assert isinstance(body["latency_ms"], (int, float)) + assert body["latency_ms"] >= 0.0 + + +def test_engine_health_in_process_falls_back_to_is_available(fresh_app): + """No health_check method on OmniVoiceBackend → fall back to is_available.""" + client = _client(fresh_app) + r = client.get("/engines/omnivoice/health") + assert r.status_code == 200 + body = r.json() + assert body["id"] == "omnivoice" + assert isinstance(body["ok"], bool) + assert isinstance(body["message"], str) + assert isinstance(body["latency_ms"], (int, float)) + + +def test_engine_health_unknown_id(fresh_app): + client = _client(fresh_app) + r = client.get("/engines/does_not_exist/health") + assert r.status_code == 404 + assert "unknown engine id" in r.json()["detail"] + + +def test_engine_health_loopback_only(fresh_app): + """Non-loopback client tuple is rejected by require_loopback.""" + client = _client(fresh_app, host="10.0.0.5") + r = client.get("/engines/omnivoice/health") + assert r.status_code == 403 + assert r.json()["detail"] == "loopback origin required" + + +def test_engine_health_caches_instance_across_calls(fresh_app, monkeypatch): + """Two health checks on the same engine reuse the same singleton. + + SubprocessBackend.__init__ registers atexit hooks; recreating it per + request would leak handler entries and (on real engines) spawn extra + sidecars on the first lock acquire. + """ + from api.routers import engines as engines_router + from services.tts_backend import _REGISTRY + + cls = _REGISTRY["indextts2"] + call_count = {"n": 0} + monkeypatch.setattr(cls, "health_check", lambda self: (True, "pong")) + # Clear the cache so the first call constructs an instance. + engines_router._ENGINE_INSTANCES.pop(cls, None) + original_init = cls.__init__ + + def _counting_init(self): + call_count["n"] += 1 + original_init(self) + + monkeypatch.setattr(cls, "__init__", _counting_init) + + client = _client(fresh_app) + r1 = client.get("/engines/indextts2/health") + r2 = client.get("/engines/indextts2/health") + assert r1.status_code == 200 and r2.status_code == 200 + assert call_count["n"] == 1, ( + f"expected exactly one IndexTTS2Backend() construction across " + f"two health checks, got {call_count['n']}" + ) + + +# ── HF-token leak prevention (T-02-12) ───────────────────────────────────── + + +def test_no_hf_token_leak_in_engines_response(fresh_app): + """A backend whose is_available() embeds a real HF token in its error + must NOT leak it to the response body. The redaction lives inside + ``tts_backend.list_backends`` via _mask_hf_tokens. + """ + from services import tts_backend as tts_mod + + class TaintedBackend(tts_mod.TTSBackend): + id = "tainted-test" + display_name = "Tainted backend (test)" + + @property + def sample_rate(self) -> int: + return 24000 + + @property + def supported_languages(self) -> list[str]: + return ["en"] + + @classmethod + def is_available(cls) -> tuple[bool, str]: + return False, f"auth failed for {SAMPLE_HF_TOKEN}" + + def generate(self, text: str, **kw): + raise NotImplementedError + + # Sandbox so the production registry shape doesn't grow permanently. + saved = dict(tts_mod._REGISTRY) + saved_errors = dict(tts_mod._LAST_ERRORS) + try: + tts_mod._REGISTRY["tainted-test"] = TaintedBackend + client = _client(fresh_app) + r = client.get("/engines") + assert r.status_code == 200 + body_text = r.text + matches = HF_TOKEN_RE.findall(body_text) + assert matches == [], ( + f"HF tokens leaked into /engines response body: {matches}" + ) + + # The masked sentinel must be present — otherwise the test isn't + # actually exercising the redaction path. + by_id = {b["id"]: b for b in r.json()["tts"]["backends"]} + assert "tainted-test" in by_id + assert "hf_***REDACTED***" in (by_id["tainted-test"]["reason"] or "") + finally: + tts_mod._REGISTRY.clear() + tts_mod._REGISTRY.update(saved) + tts_mod._LAST_ERRORS.clear() + tts_mod._LAST_ERRORS.update(saved_errors) + + +def test_no_hf_token_leak_in_health_response(fresh_app, monkeypatch): + """The health route's message field runs through the same redactor.""" + from services.tts_backend import _REGISTRY + + cls = _REGISTRY["indextts2"] + monkeypatch.setattr( + cls, "health_check", + lambda self: (False, f"sidecar 401 for {SAMPLE_HF_TOKEN}"), + ) + + client = _client(fresh_app) + r = client.get("/engines/indextts2/health") + assert r.status_code == 200 + body = r.json() + assert body["ok"] is False + assert not HF_TOKEN_RE.search(body["message"]) + assert "hf_***REDACTED***" in body["message"] diff --git a/tests/backend/services/test_tts_backend_registry.py b/tests/backend/services/test_tts_backend_registry.py index 1c7b5c13..72a0a0e5 100644 --- a/tests/backend/services/test_tts_backend_registry.py +++ b/tests/backend/services/test_tts_backend_registry.py @@ -143,9 +143,11 @@ def test_list_backends_resilient(registry_sandbox): def test_list_backends_shape(registry_sandbox): """Every entry must contain exactly the documented keys — no more, no less.""" out = list_backends() + # `gpu_compat` joined the documented shape in Plan 02-04 alongside the + # Engine Compatibility Matrix UI (ENGINE-06). required = { "id", "display_name", "available", "reason", - "install_hint", "last_error", "isolation_mode", + "install_hint", "last_error", "isolation_mode", "gpu_compat", } for entry in out: assert set(entry.keys()) == required, (