Palash DebnathandClaude Fable 5
030d5ea01f
* docs(engines): a guide for every engine + index; fix two engine-metadata bugs 21 new pages under docs/engines/ (10 TTS, 10 ASR, index README) — every registered engine now has one: what it's for, platform support, model env vars, quirks with issue refs. Linked from both READMEs' engine sections. Code fixes found while verifying facts against the registries: - KittenTTS docstring claimed default voice 'Jasper'; the code default is expr-voice-2-f - the isolated-ASR sidecar read only ASR_MODEL_FW while the download preflight read ASR_MODEL_FASTER — set one and the other quietly used a different model; both now resolve ASR_MODEL_FW-override → ASR_MODEL_FASTER - moonshine's install hint named 'useful-moonshine', a package the backend never imports; now moonshine-onnx / moonshine-voice Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): entries for the engine guides + sidecar model fix (#1556) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(engines): second-harvest fixes — TLS guidance, matrix/code alignment, CN counts - README matrix aligned to gpu_compat (the code is the source of truth): CosyVoice macOS is CPU not MPS, IndexTTS and GGUF gain their real CUDA/CPU/MPS cells - gpt-sovits guide: prefer https/tunnel for non-loopback servers, plaintext warning; first-use download guidance on both OmniVoice pages - preflight empty-env fallback matches the sidecar (ASR_MODEL_FASTER='' no longer resolves a different repo) - nano installs via uv pip; kitten log level wording; index links install guides incl. the Gatekeeper step; README_CN engine counts 16/11 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(readme-cn): the all-engines-local claim now excludes the remote client Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2.7 KiB
2.7 KiB
VoiceStudio — Faster-Whisper (Crash-Isolated) Engine
The same CTranslate2 Whisper engine as faster-whisper, run in a separate child process ("sidecar"). CTranslate2's GPU teardown can segfault — the endemic faster-whisper crash — and a hung or crashed transcribe in-process takes the whole backend down with it. Isolated, the child can crash or be force-killed to reclaim a hung transcribe and its VRAM while the backend stays up (#730).
There is nothing extra to install: the sidecar reuses the app's own venv — only the process boundary is new.
Selecting it
- Model Catalogue → Engines, ASR tab → Use on the crash-isolated row, or
- pin it with
OMNIVOICE_ASR_BACKEND=faster-whisper-isolated.
It is never picked by auto-detect — it's an explicit opt-in escape hatch.
Best at
- Long batch runs where one bad file must not kill the backend.
- Machines where in-process faster-whisper has crashed or hung before: a sidecar crash fails only that job, and the next transcribe respawns a fresh sidecar automatically.
Platform support
Same as faster-whisper: CUDA float16 or CPU int8 on macOS, Windows, and Linux. The sidecar picks cuda/cpu itself and walks the same float16 → int8_float16 → int8 degrade chain on GPUs without efficient fp16 (#551).
Model selection
ASR_MODEL_FASTER— the shared model selection, same as the in-process engine: set it once and both variants load the same weights.ASR_MODEL_FW— optional sidecar-only override; when set it wins overASR_MODEL_FASTERfor this engine. Defaultlarge-v3.ASR_COMPUTE_TYPE— optional: pin the sidecar to one CTranslate2 compute type instead of the automatic degrade chain.
Weights download on first load — see downloading-models.
Trade-offs and quirks
- Slightly slower per call than in-process faster-whisper (IPC overhead); the model stays warm inside the sidecar between calls, so the cost is per request, not per chunk of audio.
- Word timestamps are Whisper-native (±100–300 ms) — no forced alignment. For dubbing lip-sync, use whisperx or mlx-whisper.
- If the sidecar dies mid-transcription the job fails with a clear "sidecar crashed" error and the backend stays up — retry to respawn.
- cuDNN 8 is still required on CUDA — same CTranslate2 requirement as the in-process engine. It's checked up front so a missing cuDNN 8 shows as "unavailable" in Model Catalogue → Engines instead of a sidecar that silently fails every transcribe (#1371).