fix(generate): name the voice profile behind a refused language

A user on mlx-audio with the language picker on "Auto" got:

    400: mlx-audio's Kokoro model doesn't support language='Persian'.
    … Pick one of those, leave language as 'Auto', or switch to a
    multilingual engine …

They had left it on Auto. The UI omits `language` entirely while its
picker reads "Auto" (useProfiles.js only appends a non-Auto value), and
#533 fills that gap from the selected voice profile. So Auto is exactly
how 'Persian' got there: the remedy the message leads with is the state
the user was already in, and nothing points at the profile that actually
supplied the language. Three generate attempts in their action log, a
detour through Settings, then the report.

Provenance was the missing fact, and only the request scope has it — the
engine adapters are handed a language with no idea who chose it. So
_resolve_profile_conditioning now reports whether it filled the language,
and /generate uses that to answer a refused profile language by naming
the profile and the remedies that exist: change the profile's language,
pick a supported one explicitly, or switch engine.

An explicitly requested language is untouched — the user really did pick
it, so blaming the profile would be a lie — and #533 still drives
generation whenever the engine can speak the profile's language.

Recognising the refusal needed one more thing: Kokoro's wording
("doesn't support language=…") matched none of #1257's signatures, so
the engine that issue was written for was the one engine its rewrite
never fired for. That wording is now recognised, but kept out of #1257's
rewrite path — it already names its engine and its languages, and
re-wrapping it only nests "Engine's own message:" twice.

No per-model language map: #1257 weighed that and chose engine-naming
over "a brittle map that goes stale on each engine update". This follows
the same principle — say where the language came from, don't enumerate.

docs/engines/mlx-audio.md repeated the same "leave language on Auto"
advice and is corrected here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Shivendra-Coherent
2026-09-17 18:04:51 +05:30
co-authored by Claude Opus 5
parent a8581f899e
commit fbede643c1
4 changed files with 395 additions and 4 deletions
+1
View File
@@ -16,6 +16,7 @@ the frozen-backend fallback mirror it for their toolchains.
### Fixed
- Name the voice profile when its saved language is one the active engine can't speak, instead of advising a language picker already set to Auto (#2174, #2156) — thanks @shivsin25!
- Repair CTranslate2 loading safely across ASR and translation, and retain the loaded Whisper model during CPU fallback (#2165) — thanks @guruthechosen!
- Avoid pedalboard wheels that crash on unsupported CPU instructions (#2080) — thanks @D3nii!
- Include cuDNN 8 compatibility libraries for CTranslate2 in CUDA containers (#2072) — thanks @basil-k-aji-dev!
+74 -3
View File
@@ -143,7 +143,7 @@ def _resolve_profile_conditioning(row, *, ref_text=None, instruct=None,
out = {
"ref_audio_path": None, "ref_text": ref_text, "instruct": instruct,
"seed": seed, "language": language, "kind": None,
"persist_ref_text": False,
"persist_ref_text": False, "language_from_profile": False,
}
# `kind` is authoritative (0005): 'design' profiles condition on their
# deterministic rendered sample + instruct; 'clone' on the user's
@@ -212,6 +212,12 @@ def _resolve_profile_conditioning(row, *, ref_text=None, instruct=None,
prof_lang = None
if prof_lang and prof_lang != "Auto":
out["language"] = prof_lang
# #2156: record that the caller never asked for this language. The
# UI omits `language` entirely while its picker reads "Auto", so a
# profile-filled language must not be reported back as if the user
# had picked it — an engine that can't speak it would otherwise
# tell them to "leave language as Auto", which is what they did.
out["language_from_profile"] = True
return out
@@ -1043,6 +1049,18 @@ _LANGUAGE_REJECTION_SIGNATURES = (
"unsupported language code",
)
# Engine-specific rejections that ALREADY name the engine and what it supports,
# so #1257's generic rewrite deliberately leaves them alone — re-wrapping them
# only nests "Engine's own message:" twice. They still have to be recognised as
# language rejections for #2156's provenance check, which cares about the
# *cause* of the language, not the quality of the wording.
_SELF_DESCRIBING_LANGUAGE_REJECTIONS = (
# services/tts_backend.py: "…doesn't support language='Persian'. Kokoro
# supports: …" — mlx-audio's Kokoro, the engine reported in #2156.
"doesn't support language",
"does not support language",
)
#: `unsupported language: xx` / `unsupported language 'xx'` — but not
#: `unsupported language model ...`.
_LANGUAGE_REJECTION_RE = re.compile(
@@ -1052,15 +1070,54 @@ _LANGUAGE_REJECTION_RE = re.compile(
)
def _is_language_rejection(text: str) -> bool:
"""True when an engine failure is about the LANGUAGE it was handed.
Matched on the message, not the type: the engines multiplex third-party
libraries that each raise their own class. Covers the self-describing
wordings too — #1257's rewrite skips those, but #2156 still needs to know a
language was refused so it can say where that language came from.
"""
low = text.lower()
return (
any(sig in low for sig in _LANGUAGE_REJECTION_SIGNATURES)
or any(sig in low for sig in _SELF_DESCRIBING_LANGUAGE_REJECTIONS)
or bool(_LANGUAGE_REJECTION_RE.search(text))
)
def _profile_language_rejection_detail(exc: BaseException, language) -> str:
"""The 400 body for a language the *voice profile* supplied, not the user.
#2156: the UI omits `language` while its picker reads "Auto", and #533
fills that gap from the selected profile. When the active engine can't
speak the profile's language the engine's own message tells the user to
"leave language as 'Auto'" — which is exactly what they did, so the advice
cannot be acted on. Name the real source and the remedies that exist.
"""
return (
f"This voice profile is saved with the language '{language}', and the "
f"active engine can't speak it. The language picker being on \"Auto\" "
f"does not override that — Auto fills the language in from the "
f"profile. Set this voice profile's language to one the engine "
f"supports, pick a supported language explicitly for this render, or "
f"switch engine in Model Catalogue (the VoiceStudio engine has the "
f"widest coverage). Engine's own message: {exc}"
)
def _language_rejection_or(e: BaseException, backend, language):
"""``e`` rewritten with engine context when it's a language rejection.
Returns ``e`` unchanged otherwise, so this is safe to wrap any failure in.
Matched on the message, not the type: the engines multiplex third-party
libraries that each raise their own class.
Deliberately narrower than :func:`_is_language_rejection`: a message that
already names its engine and the languages it supports is left alone rather
than nested inside a second "Engine's own message:".
"""
text = str(e)
low = text.lower()
if any(sig in low for sig in _SELF_DESCRIBING_LANGUAGE_REJECTIONS):
return e
if not any(sig in low for sig in _LANGUAGE_REJECTION_SIGNATURES) and not (
_LANGUAGE_REJECTION_RE.search(text)
):
@@ -1570,6 +1627,9 @@ async def generate_speech(
ref_lease = None
used_seed = seed
resolved_profile_id = None
# #2156: True once a profile's stored language fills a language the caller
# never sent, so a rejection can name the profile instead of the picker.
language_from_profile = False
history_mode = None # profile.kind when a profile drives; else inferred at insert
# #1032: profile id to persist an auto-transcribed reference transcript to.
# Set only for a plain (unlocked) clone profile whose stored ref_text is
@@ -1598,6 +1658,7 @@ async def generate_speech(
instruct = _cond["instruct"]
used_seed = _cond["seed"]
language = _cond["language"]
language_from_profile = _cond["language_from_profile"]
if _cond["persist_ref_text"]:
persist_ref_text_profile_id = profile_id
elif ref_audio is not None:
@@ -2426,6 +2487,16 @@ async def generate_speech(
raise HTTPException(status_code=503, detail=str(e)) from e
except ValueError as e:
logger.error("Validation failed: %s", e)
# #2156: the language the engine refused was never chosen by the user —
# it came from the selected voice profile because the picker was on
# "Auto". The engine's own remedy ("leave language as 'Auto'") is then
# unfollowable, so say where the language actually came from. Only this
# scope knows that; the engine adapters never see the provenance.
if language_from_profile and _is_language_rejection(str(e)):
raise HTTPException(
status_code=400,
detail=_profile_language_rejection_detail(e, language),
) from e
# Most ValueErrors here are VoiceStudio's own validation messages and
# are exactly what the user should read. A few are raw library text
# naming parameters and files the user cannot act on — those get the
+8 -1
View File
@@ -54,7 +54,14 @@ HF repo id. The env var overrides the persisted UI choice.
- Language support is per-model (Kokoro ~8 languages, others vary). An
unsupported language for Kokoro produces a clear error naming what it
does support ([#977](https://github.com/debpalash/VoiceStudio/issues/977))
leave language on Auto or switch to a multilingual engine.
pick a language it supports or switch to a multilingual engine.
- Auto is not an escape hatch from that. With the picker on Auto the request
carries no language, and a selected voice profile's saved language fills the
gap ([#533](https://github.com/debpalash/VoiceStudio/issues/533)) — so a
profile saved as, say, Persian still reaches Kokoro and is still refused. The
error names the profile as the source in that case
([#2156](https://github.com/debpalash/VoiceStudio/issues/2156)); change the
profile's language, or pick a supported one explicitly for the render.
## Platform notes
@@ -0,0 +1,312 @@
"""#2156: a language the user never picked must not be blamed on the picker.
The reporter was on mlx-audio (Kokoro) with the language picker on "Auto" and
got:
400 Bad Request: mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16)
doesn't support language='Persian'. … Pick one of those, leave language as
'Auto', or switch to a multilingual engine …
They had left it on Auto. The UI omits `language` entirely while its picker
reads "Auto" (`frontend/src/hooks/useProfiles.js`: `if (reqLang && reqLang !==
'Auto') formData.append(...)`), and #533 fills that gap from the selected voice
profile. So "Auto" is precisely how 'Persian' got there — the one remedy the
message leads with is the state the user was already in, and nothing in it
points at the voice profile that actually supplied the language.
This completes #1257's line of work rather than reopening it: that issue chose
to name the engine and the way out instead of maintaining per-model language
maps ("a brittle map that goes stale on each engine update"). Same principle
here — say where the language came from, don't enumerate languages.
"""
import importlib
import os
import uuid
import pytest
import torch
os.environ.setdefault("OMNIVOICE_MODEL", "test")
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
def _gen_mod():
"""Imported lazily so the end-to-end tests below still run (and fail on
their assertions, not on a missing symbol) against a tree without the fix."""
return importlib.import_module("api.routers.generation")
# The real wording from services/tts_backend.py::resolve_kokoro_lang_code.
KOKORO_REFUSAL = (
"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) doesn't support "
"language='Persian'. Kokoro supports: Chinese, English, French, Hindi, "
"Italian, Japanese, Portuguese, Spanish. Pick one of those, leave language "
"as 'Auto', or switch to a multilingual engine (e.g. OmniVoice) for other "
"languages."
)
KOKORO_SUPPORTED = ("Chinese", "English", "French", "Hindi",
"Italian", "Japanese", "Portuguese", "Spanish")
def _tts_mod():
return importlib.import_module("services.tts_backend")
def _make_refusing_engine(engine_id="fake-kokoro-2156"):
"""An engine that refuses unknown languages the way Kokoro really does."""
class _FakeEngine(_tts_mod().TTSBackend):
id = engine_id
display_name = "Fake Kokoro (test)"
applies_own_mastering = False
gpu_compat = ("cpu",)
calls: list = []
@property
def sample_rate(self) -> int:
return 24000
@property
def supported_languages(self) -> list[str]:
return ["multi"]
@classmethod
def is_available(cls):
return True, "ready"
def generate(self, text, **kw) -> torch.Tensor:
type(self).calls.append((text, kw))
language = kw.get("language")
if language and language not in KOKORO_SUPPORTED:
raise ValueError(
f"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) "
f"doesn't support language={language!r}. Kokoro supports: "
f"{', '.join(KOKORO_SUPPORTED)}. Pick one of those, leave "
f"language as 'Auto', or switch to a multilingual engine "
f"(e.g. OmniVoice) for other languages."
)
return torch.zeros(1, 24000)
return _FakeEngine
@pytest.fixture()
def client():
from fastapi.testclient import TestClient
from main import app
return TestClient(app, client=("127.0.0.1", 50000))
@pytest.fixture()
def _init_db():
from core.db import init_db
init_db()
def _profile(language):
from core.db import db_conn
pid = f"vp-{uuid.uuid4().hex[:8]}"
with db_conn() as conn:
conn.execute(
"INSERT INTO voice_profiles (id, name, language, kind, created_at) "
"VALUES (?,?,?,?,?)",
(pid, f"{language} Narrator", language, "clone", 0.0),
)
return pid
def _drop(pid):
from core.db import db_conn
with db_conn() as conn:
conn.execute("DELETE FROM generation_history WHERE profile_id=?", (pid,))
conn.execute("DELETE FROM voice_profiles WHERE id=?", (pid,))
@pytest.fixture()
def persian_profile(_init_db):
pid = _profile("Persian")
yield pid
_drop(pid)
@pytest.fixture()
def english_profile(_init_db):
pid = _profile("English")
yield pid
_drop(pid)
# ── the reported failure ────────────────────────────────────────────────────
def test_a_profile_supplied_language_names_the_profile_not_the_picker(
client, monkeypatch, persian_profile
):
fake = _make_refusing_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
# `language` omitted — exactly what the UI sends with the picker on "Auto".
res = client.post("/generate", data={
"text": "Salam", "profile_id": persian_profile, "engine": fake.id,
})
assert res.status_code == 400, res.text
detail = res.json()["detail"]
# Says where the language actually came from …
assert "voice profile" in detail.lower()
assert "Persian" in detail
# … and that Auto is not an escape from it, since Auto is what filled it in.
assert "does not override" in detail
# … and keeps the engine's own capability list, quoted once, not nested.
assert "Kokoro supports:" in detail
assert detail.count("Engine's own message:") == 1
def test_the_profile_language_still_reached_the_engine(
client, monkeypatch, persian_profile
):
"""Guards the premise: this is a profile fill, not the user's choice."""
fake = _make_refusing_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
client.post("/generate", data={
"text": "Salam", "profile_id": persian_profile, "engine": fake.id,
})
assert [kw.get("language") for _t, kw in fake.calls] == ["Persian"]
def test_an_explicitly_requested_language_is_not_blamed_on_the_profile(
client, monkeypatch, english_profile
):
"""The user really did pick it, so the profile wording would be a lie —
they get the engine's own message, unchanged."""
fake = _make_refusing_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
res = client.post("/generate", data={
"text": "Salam", "profile_id": english_profile, "engine": fake.id,
"language": "Persian",
})
assert res.status_code == 400, res.text
detail = res.json()["detail"]
assert "voice profile" not in detail.lower()
assert "does not override" not in detail
assert "doesn't support language='Persian'" in detail
def test_a_supported_profile_language_still_drives_generation(
client, monkeypatch, english_profile
):
"""#533 is untouched: a profile language the engine *can* speak still
reaches it and still renders."""
fake = _make_refusing_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
res = client.post("/generate", data={
"text": "Hello", "profile_id": english_profile, "engine": fake.id,
})
assert res.status_code == 200, res.text
assert [kw.get("language") for _t, kw in fake.calls] == ["English"]
def test_a_non_language_failure_under_a_profile_is_untouched(
client, monkeypatch, persian_profile
):
"""Over-matching guard: having a profile language must not rewrite every
ValueError as a language problem."""
class _Boom(_make_refusing_engine("fake-boom-2156")):
def generate(self, text, **kw):
raise ValueError("Reference clip is shorter than 3 seconds.")
monkeypatch.setitem(_tts_mod()._REGISTRY, _Boom.id, _Boom)
res = client.post("/generate", data={
"text": "Salam", "profile_id": persian_profile, "engine": _Boom.id,
})
assert res.status_code == 400, res.text
detail = res.json()["detail"]
assert "shorter than 3 seconds" in detail
assert "voice profile" not in detail.lower()
# ── units ───────────────────────────────────────────────────────────────────
def test_the_real_kokoro_wording_is_recognised_as_a_language_rejection():
# #1257's signature list never matched this — "doesn't support language="
# contains none of "invalid language code" / "unsupported language …" — so
# the provenance check would have skipped the engine actually reported.
assert _gen_mod()._is_language_rejection(KOKORO_REFUSAL)
def test_a_self_describing_rejection_is_not_wrapped_twice():
"""Kokoro already names its engine and its languages. #1257's rewrite must
leave it alone, or the user reads "Engine's own message:" twice."""
class _Engine:
id = "mlx-audio"
display_name = "MLX Audio"
original = ValueError(KOKORO_REFUSAL)
assert _gen_mod()._language_rejection_or(original, _Engine(), "Persian") is original
@pytest.mark.parametrize("reason", [
"Invalid language code. Supported languages: ar (Arabic), da (Danish)",
"Unsupported language: bn",
])
def test_generic_rejections_are_still_rewritten_with_engine_context(reason):
"""#1257 keeps working for the messages it was written for."""
class _Engine:
id = "mlx-audio"
display_name = "MLX Audio"
rewritten = _gen_mod()._language_rejection_or(ValueError(reason), _Engine(), "bn")
assert rewritten is not ValueError
assert "MLX Audio" in str(rewritten)
def _row(**over):
row = {
"kind": "clone", "instruct": None, "is_locked": 0,
"ref_audio_path": None, "locked_audio_path": None, "ref_text": None,
"seed": None, "vd_states": None, "language": None,
}
row.update(over)
return row
def test_the_resolver_flags_a_profile_filled_language():
out = _gen_mod()._resolve_profile_conditioning(_row(language="Persian"))
assert out["language"] == "Persian"
assert out["language_from_profile"] is True
def test_the_resolver_does_not_flag_an_explicit_request_language():
out = _gen_mod()._resolve_profile_conditioning(_row(language="Persian"), language="French")
assert out["language"] == "French"
assert out["language_from_profile"] is False
def test_the_resolver_does_not_flag_when_the_profile_has_no_language():
out = _gen_mod()._resolve_profile_conditioning(_row(language=None))
assert out["language"] is None
assert out["language_from_profile"] is False
def test_an_explicit_auto_is_still_filled_from_the_profile():
# "Auto" and an absent value mean the same thing to #533; the flag must be
# set either way, since neither is the user naming a language.
out = _gen_mod()._resolve_profile_conditioning(_row(language="Persian"), language="Auto")
assert out["language"] == "Persian"
assert out["language_from_profile"] is True