Palash DebnathandClaude Fable 5
030d5ea01f
* docs(engines): a guide for every engine + index; fix two engine-metadata bugs 21 new pages under docs/engines/ (10 TTS, 10 ASR, index README) — every registered engine now has one: what it's for, platform support, model env vars, quirks with issue refs. Linked from both READMEs' engine sections. Code fixes found while verifying facts against the registries: - KittenTTS docstring claimed default voice 'Jasper'; the code default is expr-voice-2-f - the isolated-ASR sidecar read only ASR_MODEL_FW while the download preflight read ASR_MODEL_FASTER — set one and the other quietly used a different model; both now resolve ASR_MODEL_FW-override → ASR_MODEL_FASTER - moonshine's install hint named 'useful-moonshine', a package the backend never imports; now moonshine-onnx / moonshine-voice Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): entries for the engine guides + sidecar model fix (#1556) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(engines): second-harvest fixes — TLS guidance, matrix/code alignment, CN counts - README matrix aligned to gpu_compat (the code is the source of truth): CosyVoice macOS is CPU not MPS, IndexTTS and GGUF gain their real CUDA/CPU/MPS cells - gpt-sovits guide: prefer https/tunnel for non-loopback servers, plaintext warning; first-use download guidance on both OmniVoice pages - preflight empty-env fallback matches the sidecar (ASR_MODEL_FASTER='' no longer resolves a different repo) - nano installs via uv pip; kitten log level wording; index links install guides incl. the Gatekeeper step; README_CN engine counts 16/11 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(readme-cn): the all-engines-local claim now excludes the remote client Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2.5 KiB
2.5 KiB
VoiceStudio — FunASR (SenseVoice) Engine
FunASR drives Alibaba's SenseVoiceSmall with FSMN-VAD: an all-in-one multilingual pipeline — transcription with punctuation and inverse text normalization across 50+ languages, plus optional inline speaker diarization via the cam++ speaker model. It's the opt-in alternative to WhisperX (#182); WhisperX remains the cross-platform default.
Selecting it
- Install it into the app venv:
uv pip install funasr. - Then Model Catalogue → Engines, ASR tab → Use on the FunASR row, or
OMNIVOICE_ASR_BACKEND=funasr.
Auto-detect never picks it; it's an explicit opt-in.
Best at
- Multi-speaker transcription without any HuggingFace token. This is the
only ASR engine with diarization built in: cam++ labels each sentence
(
Speaker 1,Speaker 2, ...) in the same pass — no gated pyannote model, no license click-through. Compare diarization for the pyannote/WhisperX route and what each buys you. - Broad language coverage beyond Whisper's strongest languages, with punctuation included.
Not suited for
- Lip-sync dubbing — FunASR returns sentence-level timestamps, not word-level ones. Use whisperx / mlx-whisper when word timing matters.
Platform support
CUDA or CPU, on macOS, Windows, and Linux.
Model selection
| Variable | Default | Role |
|---|---|---|
ASR_MODEL_FUNASR |
iic/SenseVoiceSmall |
main ASR model |
ASR_FUNASR_VAD |
fsmn-vad |
VAD segmentation model |
ASR_FUNASR_SPK |
cam++ |
speaker model; set to empty (ASR_FUNASR_SPK=) to disable diarization and use the dub pipeline's pyannote/heuristic path instead |
Weights download on first load (through FunASR's own model hub) — see downloading-models.
Quirks
- With the speaker model enabled, long recordings are transcribed in one call and split by FunASR's internal VAD — cam++ assigns speaker cluster IDs per call, so this is what keeps "Speaker 1" meaning the same person across the whole file.
- The engine runs with
spk_mode="vad_segment": FunASR 1.3.1's default (punc_segment) requires a separate punctuation model and crashes when SenseVoice is loaded without one. - SenseVoice's rich-token markup (language/emotion/event tags around the text) is stripped from the output automatically.
- Language detection is automatic (
language: auto); the detected language is reported per file.