torch.compile(mode="reduce-overhead") captures CUDA graphs whose state is
thread-local (torch/_inductor/cudagraph_trees keys its tree manager off the
capturing thread). The _gpu_pool ThreadPoolExecutor runs up to 4 workers, so
the first render captured the graph on worker A and a later render dispatched
to worker B replayed mismatched cudagraph state — silently corrupting the
audio (static noise + slowed playback from the second render onward, no
exception, so the #327 eager fallback never fired).
Fix: when the model is compiled with a cudagraph mode, wrap model.generate
(the same single choke point #327 uses) so every call hops to a dedicated
1-thread "compiled-infer" executor — capture and replay always happen on the
same thread, deterministically. A thread-ident re-entrancy guard runs inline
when already on that thread (a 1-worker executor submitting to itself would
deadlock). Installed after the #327 fallback wrapper, so the eager retry path
also runs on the dedicated thread.
No behavior change for CPU / MPS / Windows-no-Triton / compile-disabled
paths: should_torch_compile() gates exactly as before and uncompiled models
keep the full pool.
Closes#315
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
