* fix(tts): torch.compile failures fall back to eager — generation never fails on unsupported GPUs (#278)
On GPU architectures the bundled Triton doesn't support (e.g. Blackwell
sm_120 / RTX 5060), the compiled model dies mid-generation inside the
Dynamo/Inductor/Triton/cudagraph stack — previously surfaced as a fake
'ran out of memory' error and a dead Archetype preview. Now:
- up-front arch gate: skip compile when the GPU's compute capability is
not in this torch build's arch list (OMNIVOICE_FORCE_TORCH_COMPILE=1
overrides for PTX forward-compat setups)
- runtime fallback: model.generate is wrapped once; a compile-stack
failure (classified by exception chain: module, message, traceback
paths — the cudagraph case is a bare AssertionError) logs a warning,
restores the eager module, disables compile for the session, resets
dynamo state, and retries eagerly. Non-compile errors propagate
unchanged.
- the /generate OOM handler no longer mislabels compile crashes as OOM
and points users at the actual remedy.
Fixes#278
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Potential fix for pull request finding 'CodeQL / Empty except'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for pull request finding 'CodeQL / Empty except'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Update backend/api/routers/generation.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: mergetest <test@local>
* fix(windows): gate torch.compile on Triton availability (#129, closes#65)
plan-02. torch.compile(mode="reduce-overhead") needs Triton at runtime;
Triton has no Windows wheel, so the old `device=="cuda"`-only guard in
model_manager.py failed on Windows+CUDA and surfaced as a confusing "OOM"
(#65). Inference-time, hard to diagnose.
- engine_env.should_torch_compile(device): requires CUDA + find_spec("triton")
+ the existing perf.torch_compile_disabled setting being off; logs the skip
reason at INFO and falls back to eager.
- model_manager.py call site uses it instead of the bare cuda check.
- smoke-test.sh INST-02: import torch + ctranslate2 + whisperx (full ASR path)
so a missing transitive dep fails the build instead of crashing mid-
transcription (#116). Runs in the CI smoke-matrix on Win/macOS/Linux.
setuptools>=75.0 (fix-sequence step 1) already pinned (#58). Linux/CUDA+Triton
behaviour unchanged.
Tests (TDD): tests/test_torch_compile_gate.py (4). Closes#65; addresses
#129/#116.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(windows): also gate subprocess torch.compile on Triton (Greptile #138)
Greptile flagged that the in-process gate left a parallel gap: engine
subprocesses honour TORCH_COMPILE_DISABLE, but build_engine_env() only set
it on the user's Performance toggle — so a Triton-absent host (Windows, or
macOS) still exposed subprocess engines to the same crash this PR fixes
in-process.
- build_engine_env(): set TORCH_COMPILE_DISABLE=1 when the user disabled
compile OR Triton is unavailable (find_spec), cross-platform — mirrors
should_torch_compile(). Drops the Windows-only scoping (and the now-unused
`import sys`).
- Refreshed the stale module docstring.
- 3 new tests cover the subprocess gate (triton-missing, triton-present,
user-opt-out). 7/7 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* revert(engine_env): keep subprocess TORCH_COMPILE_DISABLE user-driven
Reverts the build_engine_env() broadening from the previous commit. Auto-
disabling subprocess torch.compile on Triton-absence conflicts with a
deliberate, tested contract (test_perf_settings: Windows + flag-off ⇒ no
injection; non-Windows ⇒ never inject) — the subprocess var is intentionally
under the user's explicit control.
The #65 fix is the in-process should_torch_compile() gate (unchanged here),
which IS automatic and fully tested. Pushing back on the subprocess auto-gate
as a separate, deliberate contract change rather than forcing it through by
rewriting established tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
