* feat(omnivoice): port upstream VoiceClonePrompt persistence + FlashInfer opt-in
Upstream k2-fsa teardown ports, verified with generated voice samples:
- VoiceClonePrompt.save()/.load() (upstream format v1, weights_only-safe)
on the vendored model, and a disk layer under the in-memory prompt LRU
(DATA_DIR/prompt_cache, keyed by ref path+mtime+ref_text+preprocess,
32 newest kept, OMNIVOICE_PROMPT_DISK_CACHE=0 opts out). First generation
of a session with a known voice skips the reference re-encode and any
auto-transcription pass — verified across two real processes (encodes=1
then encodes=0, same voice).
- omnivoice_flashinfer.py ported (packed CFG attention, fused kernels,
optional CUDA graphs), schedule adapted to our num_step+1 divergence.
Opt-in via OMNIVOICE_FLASHINFER=1|graph, CUDA-only, replaces
torch.compile for the session; missing package / apply failure / runtime
failure all degrade with a named reason (same #278 contract as compile:
classify → unapply → retry once, session latch). Measured 2.20x at
batch=1 on an RTX 4090 with byte-identical text and clean ASR round-trip.
- Docs: OmniVoice guide gains instruct+reference combination semantics
(consistent instruct stabilizes cloning, reference wins conflicts),
inline pronunciation control (pinyin / CMU), prompt persistence, and
corrects the 'no voice design' claim; performance.md documents both new
env knobs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: point changelog entries at the real PR number (#1565)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pr): harden FlashInfer lifecycle + prompt-cache writes per review
Bot harvest round 1 (#1565): unapply on apply-failure (half-patched model
could crash the next render); pin eager-mode FlashInfer inference to one
thread too — the attention plan and packed position ids are per-generation
module state, so interleaved _gpu_pool workers would corrupt each other;
restore the CAPTURED pre-apply attention impl (could be flash_attention_2)
instead of assuming sdpa; unique tmp name per prompt-cache write; correct
the _forward_logits layout docstring; resolve VoiceClonePrompt at test
runtime; docs — Known limits keeps only the limitation, performance.md
states the VRAM cost and scopes the fallback claim to classified kernel
failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pr): round-2 review — publish only a fully restored model, redact latch reason, tighten CPU-persistence test
Greptile: the runtime fallback now unapplies BEFORE swapping generate, so
a concurrent render keeps queuing behind the thread-affinity wrapper while
teardown mutates modules. CodeRabbit: FlashInfer failure reasons pass
through core.failure.sanitize before latching/logging (wheel paths embed
the user's home); the save-portability test now creates the tokens on CUDA
when available and asserts the persisted payload itself is CPU-resident.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(pr): fail-closed latch reason when the sanitizer itself breaks
CodeQL empty-except + CodeRabbit round 3: if core.failure.sanitize raises,
the raw reason (home paths, wheel paths) was latched anyway. Now only the
exception class survives with a fixed redaction note; two regression tests
(normal redaction + sanitizer failure).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
