* feat(settings): compute-device override — auto | CUDA | ROCm | XPU | MPS | CPU
Auto-detect stays the default; the override kills the 'auto-detect picked
wrong' issue class. Applied at the single choke point (_probe()'s family
selection) so routing, get_best_device(), and every badge inherit it.
Resolution: OMNIVOICE_DEVICE env > Settings pick (prefs.json) > auto (#981
pattern). An override can steer, never invent hardware: a family the host
lacks is noted and ignored; cpu is always honorable. Applies at next
backend start (host caps are immutable per process — same restart contract
as the rest of the Performance tab, RestartBadge shown).
GET/PUT /api/settings/compute-device (admin-gated) reports resolved vs
applied so the panel shows restart-required truthfully and disables itself
under an env pin instead of pretending.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): entry for the compute-device override (#1557)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): harvest — the override reaches CT2 ASR, full i18n, honest edge states
- _ctranslate2_cuda_ok() and the ASR sidecar now gate on the probe's family,
so a cpu pin (or ROCm host) can never hand CTranslate2 a CUDA device —
the override reaches every CT2 loader through one shared gate
- override_ignored exposed by the API and shown by the panel (env pin naming
a device this machine lacks: auto is in effect, restart won't change it)
- all 8 panel strings + 5 device-family labels translated into all 21
locales; failed saves keep their error visible through the re-sync
- test isolation: cleanup drops OMNIVOICE_DEVICE before re-probing so no
overridden caps leak into later tests; panel tests wait for loaded state
- xpu/intel search keywords; oxfmt formatting
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): round 2 — fail-safe probe fallbacks, complete i18n, combined pin state
- a broken capability probe now means CPU everywhere (CT2 gate + ASR
sidecar) — never a torch-derived guess that would bypass a cpu pin or
re-open #1529 on ROCm; regression test added
- env-pinned AND not-detected shows both facts in one subtitle
- device_load_failed/perf_save_failed translated into all 21 locales;
CJK/th/vi/ar strings no longer say literal 'Auto'
- test_ctranslate2_never_gets_cuda_on_a_rocm_build pins the probe family
(it was order-dependent on the lru_cache before)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): pin the probe family in the faster-whisper OOM-fallback test
Same class as the rocm-build test: it mocked torch but not the probe the
new override gate consults first, so on a cpu-family CI host the CUDA
fallback chain under test was unreachable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The About page reported 'Compute device: cpu / GPU active: no / VRAM 0.00 GB' on a machine with a working GPU. Every line was true and none was usable — it is also exactly what a machine with no GPU at all reports, so the report could not distinguish a driver that isn't loaded from a container that cannot open the device from a ROCm older than the card.
The probe already knew all of it; torch.cuda.is_available() returning False simply produced no note. Each cause now reads differently: a missing device node names the --device flags, a permissions failure names --group-add and how to find the host's real render/video GIDs (copied numbers are the most common way this ends up on CPU in Docker), a card newer than the shipped ROCm points at rocminfo, an HSA_OVERRIDE_GFX_VERSION that is doing more harm than good is named first because it is both likelier and cheaper to test, and an unreachable NVIDIA driver gets its own advice.
The probe never diagnoses from a measurement it did not complete: when torch.cuda.is_available() itself raises, the exception is reported and no device findings are asserted beside it. Metadata access that raises is contained too — this runs on the path whose whole job is to explain a failure, so it cannot become one.
* fix(engines): warn about under-provisioned hardware before the synth, not after
Six reports are the same story: #1240, #1246, #1248, #1277, #1283, #1284 —
4 GB and 6 GB cards running an engine that wants 6 GB, each one waiting out
the full 300s compute budget to be told the job "was too heavy". The routing
layer knew the whole time. The error text even names the card and the figure.
The caveat only ever surfaced on the engine-PICK toast, so it reached people
who changed engines and nobody whose engine was already selected — the
default, or one persisted from a previous session. That is most users.
/generate does return X-OmniVoice-Routing, but a response header arrives when
the job ends, five minutes too late to be a warning.
So the check moves to the chokepoint every synth path shares (api/generate.ts,
same argument as the in-flight count). Fire-and-forget: never awaited, so it
cannot add latency to the request it warns about; never throws, so an
unreachable backend costs a warning rather than a generate; once per
engine+reason per session, so it informs instead of nagging. Advisory, not
blocking — the driver can page to system RAM and short inputs fit where long
ones don't.
Extracts routingNotice() as the single frontend mirror of the backend's
routing_notice(). Two callers now need "is this verdict worth interrupting
for", and two inline copies would drift — invisibly, until someone on DirectML
or an unavailable engine gets a hardware warning for a normal pick.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(cuda): stop sending every RTX 40-series card to the CPU
The SM-arch gate required the device's exact tag in get_arch_list(). NVIDIA's
rules are not exact, and PyTorch depends on that: SASS is binary-compatible
UPWARD within a major version, so the official wheels ship sm_80/sm_86 and
deliberately no sm_89 — the 8.6 kernels already cover Ada. Exact matching
therefore declared sm_89 unsupported, check_device_compatibility() returned
False, and get_best_device() silently returned "cpu".
That is every RTX 4060/4070/4080/4090, not just the reporter's card (#1285) —
each one running TTS on the CPU on hardware that works fine, with a message
telling them their GPU was unsupported.
cuda_build_covers() now applies the real rules: sm_XY covers same-major
devices with minor >= Y; compute_XY PTX JITs forward to anything newer; an
a/f suffix (sm_90a) is architecture-specific and matches exactly. Unparseable
entries are skipped, and an empty arch list still degrades to "compatible" —
the pre-existing fail-open contract.
The remediation text also pointed at a NIGHTLY index for what is a stable
supported card; it now names the stable cu128 index.
12 tests: the Ada regression, Jetson Orin (8.7), downward-within-major and
cross-major rejection, PTX forward-JIT, arch-specific suffixes, and a genuine
sm_120-on-old-wheel mismatch so the gate is proven to still work.
Closes#1285
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(cuda): resolve app modules at call time, not import time
The tests/** review contract forbids module-level imports of app modules —
they go stale under sys.modules pollution from other suites, which is the live
cause of #1269's cross-suite failures. Binds core.device_caps per call.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: restore generate.ts and generatePreflight.test.js from main
Conflict markers were committed in the previous merge — `git add` on the
directory staged both files as resolved while the markers were still in them.
Both belong to #1288 and are unchanged by this PR, so they take main's version
verbatim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Any HSA_OVERRIDE_GFX_VERSION was treated as proof of compatibility. The #1228
reporter had 11.0.0 set from older advice — if the installed build ships no
gfx1100, honouring that blindly routes them into kernels that don't exist
rather than falling back to CPU. The override is now parsed to its gfx target
and checked against the build, same rule the auto-remap already follows.
Unparseable values are still left alone: the user asked for something we don't
understand, and guessing is worse than trusting them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two P1s, both correct:
- Being IN the override map was treated as proof of compatibility. If the
wheel ships neither the native arch nor the remap target, setting
HSA_OVERRIDE_GFX_VERSION only changes WHICH kernel is missing — gfx1151 with
a gfx1030-only build was routed to the GPU and would fail at launch. Both
arch_unsupported() and _configure_rocm_if_needed() now require the target to
be present, and fall back to CPU otherwise.
- An EMPTY arch list means the build's metadata is unavailable, not that the
GPU is unsupported. The remap branch read that unknown state as a confirmed
mismatch and would push a natively-supported gfx1151 onto foreign gfx1100
kernels. Now fails open and changes nothing, matching the fail-open contract
the rest of the probe follows.
ROCM_GFX_OVERRIDES values are now the target gfx NAME rather than the HSA
version string, so the "is the target present?" check is a direct membership
test; hsa_override_for() derives the env-var form, covered by a test that
every entry in the map converts cleanly.
Also: CHANGELOG entries shortened with refs last, and the MD028 blank line
between the two docker.md blockquotes (CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GPU compatibility gate built a CUDA-namespace tag from
`get_device_capability()` (`sm_115` on a gfx1151 Strix Halo) and looked for it
in `torch.cuda.get_arch_list()` — which on a ROCm wheel returns gfx *names*
(`gfx1100`, `gfx1151`, …). The two namespaces can never intersect, so
`check_device_compatibility()` returned False on every ROCm build and
`get_best_device()` silently returned "cpu": torch saw the GPU,
`torch.cuda.is_available()` was True, and the app ran on the CPU anyway.
The comparison was copy-pasted in three places, all with the same bug, so it
now lives once in `core.device_caps.arch_unsupported()` and branches on the
build (gfx names on ROCm, sm_/compute_ tags on CUDA):
- `model_manager.check_device_compatibility` — the CPU force-route, plus a
ROCm-specific remedy instead of telling AMD users to install a cu128 wheel
- `device_caps._probe` — the kernel-risk note that downgraded the routing badge
- `engine_env._cuda_arch_supported_for_compile` — torch.compile off on all AMD
`_configure_rocm_if_needed` also applied `HSA_OVERRIDE_GFX_VERSION` from a
static map without checking whether the GPU needed it, remapping cards the
installed build supports natively onto foreign kernels. It now applies only
when the native GFX ID is genuinely absent from the arch list, and knows
gfx1150/gfx1151 (Strix Point/Halo).
Regression test: tests/test_rocm_arch_gate.py pins the reporter's host
resolving to "cuda", a genuine ROCm mismatch still being caught, the CUDA
path (#756 Blackwell fallback) unchanged, and the narrowed override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(routing): canonical host device probe + routing resolver (#21 PR 1/5)
Foundational, backend-only slice of the GPU compatibility matrix (#21).
No API or UI change — wiring lands in PRs 3–5.
- `core/device_caps.py`: single source of truth for host accelerator
capability. `detect_host_caps()` distinguishes ROCm from CUDA (unlike
the gguf hardware_probe), never raises, makes no network call, stays
kernel-free on cold start, and caches per process. Enumerates the full
degradation contract (torch-unimportable→probe_ok=False, CUDA-init
raises, device_count==0, multi-GPU, mem_get_info failure, arch
mismatch, MPS, XPU, DirectML). Plus shared `mlx_supported()` gate
(#390 groundwork) — exact-string platform check, no regex.
- `services/engine_routing.py`: pure `resolve_routing(gpu_compat, caps)`
→ `{effective_device, routing_status, routing_reason}`; deterministic
and byte-identical across OSes. Rules for accelerated / cpu_fallback
(the no-silent-fallback signal) / cpu_only / unavailable, incl. the
ROCm-not-in-set, DirectML-neutral, and XPU edges.
- `get_best_device()` delegates its family decision to the probe so the
loader and probe can never disagree; keeps the ROCm HSA env override
and DirectML device-string return (probe reads, loader writes). String
contract unchanged.
- 39 unit tests (probe / resolver / mlx gate / reason-scrub contract);
no new regex (CodeQL-clean), English-only (CJK guard green).
The gguf hardware_probe rebase is a deliberate follow-up: it has its own
torch-mocked suite and a VRAM-driven quant table unaffected by the family
rename, so it stays out of this zero-risk slice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(routing): address review — full available_families + empty-except comments
CodeRabbit / CodeQL review on PR 1:
- `available_families` no longer drops secondary accelerators on hybrid hosts
(e.g. NVIDIA + Intel-iGPU-via-IPEX). The probe now detects every accelerator
independently and picks `family` by priority at the end, instead of
short-circuiting after the first hit. Routing is unaffected (it keys off
`family`), but the field is now honest. + hybrid-host test.
- Annotated every `except: pass` in device_caps with an explanatory comment
(CodeQL py/empty-except).
- Removed the unused `_MIN_NVIDIA_DRIVER` constant — the driver-version check
stays in wizard preflight (no subprocess on the probe path); documented why.
- `get_best_device()` now checks MPS before DirectML, mirroring the probe's
family-priority order so loader and probe never disagree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Palash Debnath