Files

T

Shivendra-CoherentandClaude Opus 5 fbede643c1 fix(generate): name the voice profile behind a refused language

A user on mlx-audio with the language picker on "Auto" got:

    400: mlx-audio's Kokoro model doesn't support language='Persian'.
    … Pick one of those, leave language as 'Auto', or switch to a
    multilingual engine …

They had left it on Auto. The UI omits `language` entirely while its
picker reads "Auto" (useProfiles.js only appends a non-Auto value), and
#533 fills that gap from the selected voice profile. So Auto is exactly
how 'Persian' got there: the remedy the message leads with is the state
the user was already in, and nothing points at the profile that actually
supplied the language. Three generate attempts in their action log, a
detour through Settings, then the report.

Provenance was the missing fact, and only the request scope has it — the
engine adapters are handed a language with no idea who chose it. So
_resolve_profile_conditioning now reports whether it filled the language,
and /generate uses that to answer a refused profile language by naming
the profile and the remedies that exist: change the profile's language,
pick a supported one explicitly, or switch engine.

An explicitly requested language is untouched — the user really did pick
it, so blaming the profile would be a lie — and #533 still drives
generation whenever the engine can speak the profile's language.

Recognising the refusal needed one more thing: Kokoro's wording
("doesn't support language=…") matched none of #1257's signatures, so
the engine that issue was written for was the one engine its rewrite
never fired for. That wording is now recognised, but kept out of #1257's
rewrite path — it already names its engine and its languages, and
re-wrapping it only nests "Engine's own message:" twice.

No per-model language map: #1257 weighed that and chose engine-naming
over "a brittle map that goes stale on each engine update". This follows
the same principle — say where the language came from, don't enumerate.

docs/engines/mlx-audio.md repeated the same "leave language on Auto"
advice and is corrected here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-17 18:04:51 +05:30

3.8 KiB

Raw Blame History

VoiceStudio — MLX-Audio Engine (Apple Silicon)

MLX-Audio (Blaizzy/mlx-audio) wraps 14+ TTS engines — Kokoro, CSM, Dia, Qwen3-TTS, Chatterbox, MeloTTS, OuteTTS, and more — behind a single adapter that runs on Apple's MLX framework. It is Apple Silicon only: the engine is not shipped on Linux, Windows, or Intel Macs, and a stray wheel on those platforms never reports as available (#390).

When to pick it

You're on an M-series Mac and want small, fast models tuned for it.
You want one of the specific hosted models (Kokoro for small multilingual, CSM for cloning, Qwen3-TTS for voice design, Dia for dialogue, …).

Setup

pip install mlx-audio

Then select the engine via Model Catalogue (TTS tab → Use) or OMNIVOICE_TTS_BACKEND=mlx-audio.

Model selection

One backend hosts many models. The curated set:

Key	Model	Niche
`kokoro` (default)	`mlx-community/Kokoro-82M-bf16`	small multilingual
`csm`	`mlx-community/csm-1b-8bit`	voice cloning
`qwen3-tts`	`mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-4bit`	voice design
`dia`	`mlx-community/Dia-1.6B`	dialogue
`chatterbox`	`mlx-community/Chatterbox-TTS-4bit`	expressive
`melotts`	`mlx-community/MeloTTS-English-v3-MLX`	lightweight VITS
`outetts`	`mlx-community/Llama-OuteTTS-1.0-1B-4bit`	LM-based

Pick a model in the Model Catalogue curated picker (#981) or set OMNIVOICE_MLX_AUDIO_MODEL to either a curated key (kokoro) or any full HF repo id. The env var overrides the persisted UI choice.

Behaviour notes

Output is 24 kHz mono. Results from models with a different native rate (such as Dia at 44.1 kHz) are resampled before stitching and export.
Cloning works only with the csm model — it is the only curated model confirmed to accept a reference clip. Other models silently ignore reference audio, so the engine reports cloning support only when CSM is selected (dub/batch jobs gate on this).
Voice design (text description → voice) is available through the Qwen3-TTS VoiceDesign model.
Language support is per-model (Kokoro ~8 languages, others vary). An unsupported language for Kokoro produces a clear error naming what it does support (#977) — pick a language it supports or switch to a multilingual engine.
Auto is not an escape hatch from that. With the picker on Auto the request carries no language, and a selected voice profile's saved language fills the gap (#533) — so a profile saved as, say, Persian still reaches Kokoro and is still refused. The error names the profile as the source in that case (#2156); change the profile's language, or pick a supported one explicitly for the render.

Platform notes

This engine is exempt from cross-platform parity as a platform-only capability behind explicit opt-in: it exists only where Apple's MLX runtime exists. On any other platform the engine picker shows it unavailable with the reason.

Troubleshooting

Unavailable on an M-series Mac: pip install mlx-audio into VoiceStudio's Python environment; in a packaged app build, MLX's native libraries may fail to load — the engine reports unavailable rather than crashing.
Other issues: install/troubleshooting.md.

Consecutive chunks with the same native sample rate are resampled together to preserve filter context at chunk boundaries; rate changes start a new group.

3.8 KiB Raw Blame History