fix(device): fall back to CPU when the GPU arch is unsupported, not 500 every generate (#756) (#757)

* fix(settings): contain + tighten the whole Settings surface (measure cap, container-query stacking, wrap the shared rows)

Two systemic issues drove 'too spread out' + 'elements go out of view' across
many Settings pages:

1. Spread — .settings-content capped at 1280px, so on wide windows every
   label-left/control-right row left a huge void. Introduce a --settings-measure
   token (720px, macOS-like) + --settings-rail, and cap the content to it,
   left-aligned under the nav. One token now controls the reading width.

2. Overflow + bad responsiveness — the row stack break was a *viewport* media
   query (560px), but the 168px nav rail means a 760px-viewport window only has
   ~530px of content, so rows went side-by-side in a cramped box. Make
   .settings-content a container (container-type: inline-size) and stack on the
   CONTENT width via @container, keeping the viewport @media as a fallback for
   the .st-row instances used outside Settings (Splash/FirstRun/Dub/SetupWizard).

3. The shared .perfpanel__row (button/badge row reused by 6+ panels:
   RemoteBackend, HFMirror, LLMEndpoint, Pronunciation, MCPBindings, …) was an
   inline-flex with no wrap and no max-width, so it ran off the right edge —
   add flex-wrap + max-width:100% + min-width:0. Plus two rigid-width fixes that
   escaped the row cap: ApiKeys input min-width:220→0, Appearance scale floor.

Frontend builds clean; tokens, @container query, and the wrap all verified in the
emitted CSS bundle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(settings): center the settings block + tighten measure (kill the lopsided right void)

The capped content was left-aligned, so on a wide window everything jammed to the
left with a dead empty third on the right (screenshot). Center the whole settings
block (nav rail + content) as a unit via max-width + margin-inline:auto, and drop
the measure 720→660 so label→control rows read denser. The cap is computed from
the tokens (rail + gap + measure + page padding) so the content track lands
exactly at --settings-measure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(device): fall back to CPU when the GPU arch is unsupported, instead of 500-ing every generate (#756)

get_best_device() called check_device_compatibility() and, on an unsupported
compute capability, only LOGGED a warning then still returned 'cuda' — so the
model loaded on a GPU whose kernels can't launch and every generate 500'd with
'CUDA error: no kernel image is available for execution'. Both a too-old card
(Pascal sm_61, GTX 10-series) and a too-new one (Blackwell sm_120 on pre-cu128
wheels) hit this.

Now an unsupported arch falls back to CPU (works, just slower) with a clear
warning; OMNIVOICE_FORCE_CUDA=1 overrides. Belt-and-suspenders: _oom_friendly_reraise
classifies a raw 'no kernel image is available' as an unsupported-GPU error
(switch to CPU / install matching torch) rather than the OOM/Flush message.

Tests: get_best_device → cpu on incompatible, stays cuda on compatible, honors
the force override; reraise gives the actionable GPU message, not OOM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(device): patch detect_host_caps via string path so the #756 fallback test is full-suite robust

The first version aliased the import + inserted backend on sys.path, which patched
a module copy get_best_device's local 'from core.device_caps import detect_host_caps'
didn't resolve in the full suite (passed alone, failed in CI). Use the string-form
monkeypatch target; verified passing alongside the other device/model tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): fold #757 device-fallback entry into [0.3.8]; drop the merge's stale [Unreleased] dupe

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CHANGELOG.md

@@ -177,6 +177,16 @@ across dub, generate, and design (a corrupt-binary failure no longer poses as
   Closes the whole class of GPU-job-hang reports (#851 — #850, #802, #755, #723,
   #721, and the 0.3.7 cohort, all tracked in #730).
 
+- **An unsupported GPU now falls back to CPU instead of 500-ing every generate.**
+  When the installed PyTorch build has no kernels for your GPU's compute
+  capability — a too-old card (Pascal / GTX 10-series) or a too-new one
+  (Blackwell RTX 50-series on pre-cu128 wheels) — CUDA failed at launch with the
+  cryptic `CUDA error: no kernel image is available for execution`. The backend
+  now detects that up front and runs on CPU (slower, but it works), and any raw
+  occurrence is reported as "your GPU isn't supported — switch to CPU or install a
+  matching PyTorch," not a Flush-the-memory dead end. Force the GPU anyway with
+  `OMNIVOICE_FORCE_CUDA=1`. (#756)
+
 - **The "TRANSLATION FAILED" banner now dismisses and clears itself.** The Dub
   translation-error banner used to be sticky — it survived a successful re-try and
   never went away. It now has a close (×), auto-clears on the next corrective

backend/api/routers/generation.py

@@ -214,6 +214,18 @@ def _oom_friendly_reraise(e):
     # sys.stdout/stderr to swallow EPIPE, but a C-level write inside the native
     # engine/torch can still raise one past that guard. Flush won't help —
     # relaunching the app re-parents the backend to a live shell.
+    # #756: the GPU's compute capability isn't in this PyTorch build's arch list,
+    # so CUDA can't launch kernels ("no kernel image is available for execution").
+    # NOT OOM. get_best_device() now falls back to CPU up front, but classify the
+    # raw error too in case CUDA was forced (OMNIVOICE_FORCE_CUDA) or a sub-path
+    # still ran on the GPU — point at the real fix, not the Flush button.
+    if "no kernel image is available" in _low:
+        raise RuntimeError(
+            f"Your GPU isn't supported by the installed PyTorch build (CUDA can't "
+            f"launch kernels for its compute capability). Switch the compute device "
+            f"to CPU in Settings, or install a matching PyTorch (e.g. a cu128 build "
+            f"for newer GPUs). The Flush button won't help. Underlying error: {e}"
+        ) from e
     if isinstance(e, BrokenPipeError) or "broken pipe" in _low or "errno 32" in _low:
         raise RuntimeError(
             f"The backend lost its output pipe mid-generation — the desktop app "

backend/services/model_manager.py

@@ -370,6 +370,19 @@ def get_best_device():
         compatible, warning = check_device_compatibility()
         if not compatible:
             logger.warning(warning)
+            # #756: the GPU's compute capability isn't in this torch build's arch
+            # list, so CUDA kernels can't launch ("no kernel image is available
+            # for execution") — every generate would 500. Too-old (Pascal sm_61)
+            # and too-new (Blackwell sm_120 on pre-cu128 wheels) both land here.
+            # Fall back to CPU so the app WORKS (slowly) instead of dead-ending;
+            # OMNIVOICE_FORCE_CUDA=1 overrides for users who installed a matching
+            # torch and know the arch_list probe is wrong for their setup.
+            if not _env_flag("OMNIVOICE_FORCE_CUDA"):
+                logger.warning(
+                    "Falling back to CPU: this GPU is unsupported by the installed "
+                    "PyTorch build (set OMNIVOICE_FORCE_CUDA=1 to force CUDA anyway)."
+                )
+                return "cpu"
         return "cuda"
 
     # ── Intel Arc / discrete GPU via IPEX ────────────────────────────

tests/test_device_unsupported_fallback.py

@@ -0,0 +1,46 @@
+"""#756: a GPU whose compute capability isn't in the installed PyTorch build's
+arch list can't launch CUDA kernels ("no kernel image is available for
+execution"), so every generate 500s. get_best_device() must fall back to CPU so
+the app still works (slowly) instead of dead-ending — unless the user explicitly
+forces CUDA. These tests pin that fallback (and the override) without a GPU.
+"""
+from types import SimpleNamespace
+
+import pytest
+
+import services.model_manager as mm
+
+
+@pytest.fixture
+def cuda_host(monkeypatch):
+    # Pretend a CUDA GPU is present. Patch detect_host_caps via its string path so
+    # the lookup resolves the same module object get_best_device imports locally
+    # (`from core.device_caps import detect_host_caps`) — patching an aliased
+    # import can miss that in a full-suite run.
+    monkeypatch.setattr(
+        "core.device_caps.detect_host_caps", lambda: SimpleNamespace(family="cuda")
+    )
+    monkeypatch.setattr(mm, "_lazy_torch", lambda: SimpleNamespace())
+    monkeypatch.setattr(mm, "_configure_rocm_if_needed", lambda _torch: None)
+    monkeypatch.delenv("OMNIVOICE_FORCE_CUDA", raising=False)
+
+
+def test_unsupported_gpu_falls_back_to_cpu(cuda_host, monkeypatch):
+    monkeypatch.setattr(
+        mm, "check_device_compatibility",
+        lambda: (False, "GTX 1080 Ti (sm_61) is not supported by this PyTorch build"),
+    )
+    assert mm.get_best_device() == "cpu"
+
+
+def test_supported_gpu_stays_on_cuda(cuda_host, monkeypatch):
+    monkeypatch.setattr(mm, "check_device_compatibility", lambda: (True, None))
+    assert mm.get_best_device() == "cuda"
+
+
+def test_force_cuda_overrides_the_fallback(cuda_host, monkeypatch):
+    monkeypatch.setattr(
+        mm, "check_device_compatibility", lambda: (False, "unsupported arch"),
+    )
+    monkeypatch.setenv("OMNIVOICE_FORCE_CUDA", "1")
+    assert mm.get_best_device() == "cuda"

tests/test_generation_audio_guard.py

@@ -102,6 +102,22 @@ def test_broken_pipe_is_a_lost_pipe_not_oom():
         assert "ran out of memory" not in msg
 
 
+def test_no_kernel_image_is_an_unsupported_gpu_not_oom():
+    # #756: a GPU whose compute capability isn't in the torch build's arch list
+    # (Pascal sm_61 on new wheels, Blackwell sm_120 on old wheels) raises "CUDA
+    # error: no kernel image is available for execution". That's NOT OOM and Flush
+    # won't help — point at CPU / a matching torch.
+    err = RuntimeError(
+        "CUDA error: no kernel image is available for execution on the device"
+    )
+    with pytest.raises(RuntimeError) as ei:
+        _oom_friendly_reraise(err)
+    msg = str(ei.value)
+    assert "GPU isn't supported" in msg or "isn't supported by the installed" in msg
+    assert "CPU" in msg
+    assert "ran out of memory" not in msg
+
+
 def test_winerror_193_is_a_corrupt_binary_not_oom():
     # #705: a corrupt / wrong-architecture native component (torch, ffmpeg, an
     # engine binary) fails on Windows with "[WinError 193] %1 is not a valid