Palash DebnathandClaude Fable 5
030d5ea01f
* docs(engines): a guide for every engine + index; fix two engine-metadata bugs 21 new pages under docs/engines/ (10 TTS, 10 ASR, index README) — every registered engine now has one: what it's for, platform support, model env vars, quirks with issue refs. Linked from both READMEs' engine sections. Code fixes found while verifying facts against the registries: - KittenTTS docstring claimed default voice 'Jasper'; the code default is expr-voice-2-f - the isolated-ASR sidecar read only ASR_MODEL_FW while the download preflight read ASR_MODEL_FASTER — set one and the other quietly used a different model; both now resolve ASR_MODEL_FW-override → ASR_MODEL_FASTER - moonshine's install hint named 'useful-moonshine', a package the backend never imports; now moonshine-onnx / moonshine-voice Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): entries for the engine guides + sidecar model fix (#1556) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(engines): second-harvest fixes — TLS guidance, matrix/code alignment, CN counts - README matrix aligned to gpu_compat (the code is the source of truth): CosyVoice macOS is CPU not MPS, IndexTTS and GGUF gain their real CUDA/CPU/MPS cells - gpt-sovits guide: prefer https/tunnel for non-loopback servers, plaintext warning; first-use download guidance on both OmniVoice pages - preflight empty-env fallback matches the sidecar (ASR_MODEL_FASTER='' no longer resolves a different repo) - nano installs via uv pip; kitten log level wording; index links install guides incl. the Gatekeeper step; README_CN engine counts 16/11 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(readme-cn): the all-engines-local claim now excludes the remote client Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
3.2 KiB
3.2 KiB
VoiceStudio — Faster-Whisper Engine
Faster-Whisper runs Whisper on CTranslate2 — the same transcription core WhisperX uses, without the wav2vec2 forced-alignment pass. It's the safe cross-platform fallback when whisperx isn't installed, and the capture/dictation fallback on non-Apple machines.
Selecting it
- Model Catalogue → Engines, ASR tab → Use on the Faster-Whisper row, or
- pin it with
OMNIVOICE_ASR_BACKEND=faster-whisper.
Auto-detect only picks it when whisperx is unavailable.
Best at
- Subtitles, dictation buffers, and batch transcription where Whisper's native word timing (±100–300 ms) is good enough.
- For dubbing lip-sync, prefer whisperx (or mlx-whisper on Apple Silicon) — their forced alignment is an order of magnitude tighter on word boundaries.
Platform support
- CUDA — float16, with automatic degradation (below).
- CPU — int8 on macOS, Windows, and Linux.
- Apple Silicon GPU / ROCm — not supported: CTranslate2 has no Metal or HIP build, so those hosts run on CPU (#1529); auto-detect routes them to mlx-whisper / pytorch-whisper instead.
Model selection
ASR_MODEL_FASTER — default Systran/faster-whisper-large-v3. Accepts the
size aliases (tiny … large-v3, distil-large-v3) or any CTranslate2
Whisper repo on HF. Weights download on first load — see
downloading-models.
Segments are cleaned up by faster-whisper's built-in Silero VAD before transcription.
Degradation chains
- GPUs without efficient fp16 (older Maxwell/Pascal, GTX 16xx, or a CTranslate2/cuDNN mismatch) fail at model construction with a compute-type error; the engine walks float16 → int8_float16 → int8 instead of failing every chunk (#551).
- A CUDA out-of-memory falls back to CPU (slower, same model and accuracy) — flushing the resident TTS model frees VRAM for GPU-speed ASR (#255).
Quirks
- cuDNN 8 required on CUDA — a missing cuDNN 8 would fast-fail the whole process, so the engine checks up front and reports itself unavailable instead (#1371). pytorch-whisper covers that case on torch's bundled cuDNN 9.
- On some hardened Linux kernels the CTranslate2 native library is rejected with "cannot enable executable stack" (an OSError, not an ImportError) — reported as unavailable rather than crashing engine selection (#692).
- CTranslate2's GPU teardown can rarely segfault the process at unload. If you hit that, switch to the crash-isolated variant — faster-whisper-isolated (#730).
- Transcribes are time-bounded:
OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S(default 120 s per dub chunk) andOMNIVOICE_ASR_TRANSCRIBE_TIMEOUT_S(default 300 s whole-file).
Speed comparisons across engines live in performance.