The wave of "Can't reach the local backend" reports — all on ~8 GB NVIDIA cards,
all during generate bursts — is the backend *process* dying, not a transport
blip. Root cause: the GPU pool was sized at 2.5 GB/job, so an 8 GB card (~7 GB
free) got 2 workers. The interactive clone path co-loads WhisperX large-v3 ASR
(~3 GB) alongside TTS (~1.6 GB), so two concurrent clone jobs is ~10 GB on an
8 GB card → a sticky CUDA "illegal memory access" that aborts the whole
interpreter (uncatchable by the per-request OOM guard, which only re-raises a
clean torch.cuda.OutOfMemoryError as HTTP 500).
Budget 5 GB/job (the real TTS+ASR concurrent footprint) instead of 2.5 GB:
≤10 GB cards now serialize to a single GPU worker — no concurrent-kernel
contention, so the crash can't happen — while 16/24 GB cards still parallelize.
Overridable via OMNIVOICE_GPU_WORKERS. This *prevents* the crash; the
auto-restart supervisor (#572) *recovers* from any other cause — defense in
depth.
Extracted `_workers_for_free_vram()` (pure) with tests pinning 8 GB → 1 worker,
the larger-card ladder, the floor/cap, and a guard on the budget constant so a
regression toward 2.5 GB can't silently re-enable the crash.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
