The compute-time error told users to raise a generation timeout that had no control anywhere in the app — the only knob was an environment variable, and on Windows the docs explicitly warn against the usual way of setting one. Both budgets are now editable in Settings under Performance & Device, persisted and applied on the next start.
Two defects found in review and fixed here rather than shipped: an explicit universal budget silently overrode a separately saved CPU budget, so the CPU row would have looked like it worked and done nothing; and a value already set in the environment shadowed the saved preference while the panel still reported success. A shadowed row now says so instead. Long-input warnings also fire on Apple Silicon, which gets the accelerated budget and was the device in one of the duplicate reports.
Fixes#1787. Closes the reports tracked in #1774 and #1778.
The guard itself (#851/#896) is working as designed — this is a message
gap, not a logic bug. The GPU-branch timeout guidance explained VRAM
contention but never mentioned the Flush/Unload action that actually
resolves it, even though: (a) that action already exists (POST
/system/flush-memory, wired to the header's Flush button), and (b) the
sibling ASR-timeout guard's message already recommends it verbatim
(asr_backend.py's _CUDA_VRAM_BUDGET_GB guidance). The maintainer ended up
manually explaining 'Settings → Models → Flush caches / Unload' in an
issue thread reply — information the error message should have carried
itself.
String-only change, no control-flow touched, mirrors the exact precedent
of #896 (a guidance-only change to this same function).
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(tts): bound + reset the GPU pool on a hung generate so it can't brick the backend (#730 class)
A GPU job that wedges on some Windows+CUDA setups occupies its worker
forever — run_in_executor can't cancel the thread — so on the 1–2 worker
pools we ship, one stuck job starves every other request and the next
action surfaces as the misleading "Can't reach the local backend" even
though the process is alive.
ASR/dub/model-load already bound+reset the pool on hang (#730). The TTS
**generate** paths (generation.py, tts_stream.py) were the last unguarded
GPU dispatch — and the residual on-main reports (#850#802#755#723#721,
plus the 0.3.7 generate cohort) all fail on generate:start (audio).
- model_manager: add run_on_gpu_pool_guarded() + GpuJobTimeoutError, a
generalized version of the ASR guard so every GPU dispatch shares one
bound+reset recovery path. Env-tunable via OMNIVOICE_GENERATE_TIMEOUT_S
(default 300s).
- generation.py: route both inference branches + the reference-clip
transcribe through the guard; map a timeout to an actionable 503.
- tts_stream.py: same guard on the streaming path (timeout → error frame).
- test_generate_timeout_730: fail-before/pass-after regression (timeout
resets pool + restores capacity, happy path, env override, no-reset exec).
- docs + CHANGELOG: extend troubleshooting §14 to cover generate; document
the new env var.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(tts): extend the GPU-pool hang guard to batch/dub/archetype/openai-compat generate (#730 class)
The generate-hang class wasn't only in Studio + streaming: batch generate,
the dub per-segment + preview generate, archetype preview render, and the
OpenAI-compat /v1/audio/speech path all dispatched the TTS model to the GPU
pool with no wall-clock bound either. Any one of them wedging on a
Windows+CUDA hang starves the pool and bricks the backend the same way.
Route all of them through run_on_gpu_pool_guarded so the whole class is
closed — a hung generate anywhere resets the pool and returns an actionable
timeout instead of a dead backend. Batch/dub recover per-segment on a fresh
worker; drop the now-dead loop/_gpu_pool/asyncio locals ruff flagged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Palash Debnath