diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c6df8b7..817240bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,10 @@ the frozen-backend fallback mirror it for their toolchains. - Handle missing Electron signing credentials and retry packaging fixes without moving release tags (#2157) +### Fixed + +- Distinguish architecture preflight rejection from independent compile-stack failures (#2085) — thanks @Moep90! + ## [0.5.3] — 2026-09-17 **Highlights** diff --git a/backend/services/engine_env.py b/backend/services/engine_env.py index 925db150..6f802bb5 100644 --- a/backend/services/engine_env.py +++ b/backend/services/engine_env.py @@ -203,8 +203,8 @@ def _cuda_arch_supported_for_compile() -> "tuple[bool, str]": upstream support lands (issue #278): the eager model runs via PTX forward-compat, but Inductor/Triton kernel compilation targets the new arch directly and fails mid-generation. Blackwell sm_120 was that case; it no - longer is on the pinned torch 2.8.0+cu128, where this probe returns - supported and compile runs correctly. If the device's arch tag is + longer is on the pinned torch 2.8.0+cu128, where this probe can return + supported; independent compiler/runtime failures still need eager fallback. If the device's arch tag is absent from this build's arch list we treat compile as unsupported and use eager. The comparison is delegated to ``core.device_caps.arch_unsupported`` so it stays CUDA/ROCm-aware — a ROCm build lists ``gfx…`` names, and the diff --git a/backend/services/model_manager.py b/backend/services/model_manager.py index d7dea8e7..1b0d8e28 100644 --- a/backend/services/model_manager.py +++ b/backend/services/model_manager.py @@ -1565,10 +1565,10 @@ def _is_compile_runtime_failure(exc: BaseException) -> bool: """True when an exception originates in the torch.compile stack (Dynamo / Inductor / Triton / FX / CUDA-graph trees) rather than in the model itself. - #278: on a GPU architecture missing from the running torch build's arch - list, the compiled model dies mid-generation with an AssertionError out of - torch/_inductor/cudagraph_trees.py. Blackwell sm_120 was the reported case - and no longer applies to the pinned torch 2.8.0+cu128, which supports it. + #278: an independent compile-stack failure can surface during generation + as an AssertionError out of torch/_inductor/cudagraph_trees.py. An + architecture missing from the running torch build's arch list is rejected + earlier by should_torch_compile(), before this runtime fallback applies. #278 also quotes "Detected that you are using FX to symbolically trace a dynamo-optimized function"; Dynamo raises that on any device, CPU included, so it is a compile-stack error to catch here but never an arch signal. diff --git a/docs/install/troubleshooting.md b/docs/install/troubleshooting.md index ee9701d9..a303dc57 100644 --- a/docs/install/troubleshooting.md +++ b/docs/install/troubleshooting.md @@ -1148,3 +1148,6 @@ remove the app binary itself are in [docs/install/uninstall.md](uninstall.md). **Linked issue:** [#1089](https://github.com/debpalash/VoiceStudio/issues/1089) + + +An architecture accepted by the torch.compile preflight may still encounter independent Dynamo, Inductor, Triton, or CUDA-graph runtime errors. VoiceStudio distinguishes those from GPU memory exhaustion and retries with eager execution; architecture support alone does not guarantee compilation succeeds. diff --git a/tests/test_compile_fallback.py b/tests/test_compile_fallback.py index ea06805e..0f6215de 100644 --- a/tests/test_compile_fallback.py +++ b/tests/test_compile_fallback.py @@ -1,9 +1,9 @@ """#278 — torch.compile failures must fall back to eager, never fail generation. -On a GPU architecture missing from the running torch build's arch list, -`torch.compile` succeeds at load time but the *first generation* dies inside -the Dynamo/FX/Inductor stack (an AssertionError out of -torch/_inductor/cudagraph_trees.py) and was mislabeled as an OOM. +An independent compile-stack failure can occur during generation inside +Dynamo/FX/Inductor (an AssertionError out of torch/_inductor/cudagraph_trees.py) +and must not be mislabeled as an OOM. Unsupported architectures are rejected +by should_torch_compile() before compilation; they do not reach this path. Blackwell sm_120 was the reported case and is no longer an example: the pinned torch 2.8.0+cu128 lists sm_120, and compile, Triton 3.4.0 and the