* fix(tts): Vietnamese consistency — Voice vs Audiobook divergences (#1139) Three root causes behind "Vietnamese Voice generation is inconsistent compared to Audiobook": 1. Numbers: num2words' vi cardinals are wrong for 2001-2099 (misused "lẻ": 2024 → "hai nghìn lẻ hai mươi bốn") and vi has no year form, so normalization mangled the years the engine used to read natively. Vietnamese now keeps its digits, and _num2words_lang's display-name path now gates on _NUM2WORDS_LANGS like the ISO path (the loophole that let "Vietnamese" bypass the vetting "vi" would have failed). 2. Seed: the longform resolver fetched a profile's pinned seed but only the cache signature ever used it — book renders ran unseeded. Both longform synth wrappers now seed torch per segment via the new pure segment_seed(base_seed, text) helper (crc32-decorrelated, order- and cache-independent, mirroring /generate's used_seed + i). 3. Quality preset: the audiobook synth inherited num_step=32 / guidance_scale=2.0 from model-config defaults by accident of omission while /generate defaults to 16 — the main audible gap. Now explicit (LONGFORM_NUM_STEP / LONGFORM_GUIDANCE_SCALE), pinned by a test so upstream default drift can't silently change books. The Voice-page fast default (16) is deliberately unchanged. Also (issue part 3): the finished audiobook's player + Download link lived in component useState and evaporated on tab switch — the last render's filename is now store-backed and persisted. Regression tests fail-before/pass-after (verified by stashing the fix): vi digit passthrough + vetted-set gate invariant; segment_seed + seeding in both synth branches + explicit preset kwargs; lastOutput store round-trip. Full backend suite 3004 passed; frontend 1237 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ui): loadProject clears lastOutput; document longform seeding contracts (review) Review-bot findings on #1142, evaluated: - FIXED (Greptile P1 "Output Escapes Its Project" + CodeRabbit): loadProject now resets lastOutput like newProject already did, so loading project B never presents A's finished render as B's output. Regression test added (set lastOutput → loadProject → cleared). - REFUTED (P1 "Global RNG Races Between Workers"): the exposure is identical to /generate's existing #526 seeding — generation.py calls torch.manual_seed on the same global RNG inside the same GPU pool, and has since that PR. The pool is 1 worker on MPS/CPU and small-VRAM CUDA (model_manager._pick_gpu_workers), where determinism is strict; a >1-worker CUDA pool is best-effort for BOTH paths. A race-free fix means threading a per-call torch.Generator through the model's samplers app-wide (covering /generate too) — out of scope for this PR and pointless to do one-sided. Contract now documented on _seed_segment_rng. - REFUTED (P2 "Repeated Text Reuses One Seed"): identical takes for identical repeated lines is the pipeline's shipped semantic — the content-addressed SegmentCache (segment_cache_key hashes text + voice sig, not position) already replays one WAV for every identical span — and seeding only activates when the user pinned a seed, i.e. asked for reproducibility. Position-based keys would shift every later span's seed on a one-paragraph insert, breaking the cache-independent partial re-render guarantee. Documented on segment_seed. - DECLINED (P2 "Persisted Filename Can Outlive File"): longform outputs in OUTPUTS_DIR are not auto-pruned (prune_cache_dir bounds only longform_cache), so a dangling name requires manual deletion; auto-clearing on an <audio> error would instead wipe a valid link whenever the backend is briefly down at mount. Projects → Audiobooks stays the authoritative library. Also rebased onto main past #1141 (CHANGELOG resolved keeping both Unreleased→Fixed entries, this PR's on top). Affected suites: 264 passed; frontend format clean, 1245 tests passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: mergetest <nizam4103@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
154 lines
5.3 KiB
Python
154 lines
5.3 KiB
Python
"""Longform synth generation params (#1139).
|
|
|
|
Two divergences between the Voice page (/generate) and the audiobook/longform
|
|
path made "the same profile + text + settings" behave differently:
|
|
|
|
* A profile's pinned ``seed`` was fetched by ``_resolve_voice`` but only ever
|
|
used in the cache signature — generation itself ran unseeded, so a locked
|
|
take's pinned seed silently did nothing in a book render.
|
|
* The omnivoice synth wrapper passed no ``num_step``/``guidance_scale``,
|
|
silently inheriting the model-config defaults. That is the intended quality
|
|
preset for longform, but it must be explicit (LONGFORM_NUM_STEP) so it can't
|
|
drift with upstream config changes.
|
|
|
|
Engine layer stubbed throughout — no model loads, no GPU.
|
|
"""
|
|
import asyncio
|
|
import os
|
|
|
|
os.environ.setdefault("OMNIVOICE_MODEL", "test")
|
|
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
|
|
|
|
import pytest
|
|
import torch
|
|
|
|
from services.audiobook import segment_seed
|
|
|
|
|
|
# ── segment_seed: pure helper ────────────────────────────────────────────────
|
|
|
|
def test_segment_seed_deterministic_and_in_torch_range():
|
|
a = segment_seed(1234, "xin chào")
|
|
assert a == segment_seed(1234, "xin chào") # stable across calls/runs
|
|
assert 0 <= a < 2**31 # valid torch.manual_seed input
|
|
|
|
|
|
def test_segment_seed_decorrelates_chunks_but_tracks_base_seed():
|
|
# Different chunk text → different seed (mirrors /generate's used_seed + i);
|
|
# different pinned seed → different seed for the same text.
|
|
assert segment_seed(1234, "chunk one") != segment_seed(1234, "chunk two")
|
|
assert segment_seed(1234, "chunk one") != segment_seed(99, "chunk one")
|
|
|
|
|
|
# ── generic-engine branch: pinned profile seed reaches torch ─────────────────
|
|
|
|
def _fake_backend_cls(calls):
|
|
from services.tts_backend import TTSBackend
|
|
|
|
class _Fake(TTSBackend):
|
|
id = "fake-longform-engine"
|
|
display_name = "Fake Longform Engine (test)"
|
|
gpu_compat = ("cpu",)
|
|
|
|
@property
|
|
def sample_rate(self):
|
|
return 24000
|
|
|
|
@property
|
|
def supported_languages(self):
|
|
return ["multi"]
|
|
|
|
@classmethod
|
|
def is_available(cls):
|
|
return True, "ready"
|
|
|
|
def generate(self, text, **kw):
|
|
calls.append((text, kw))
|
|
return torch.zeros(1, 2400)
|
|
|
|
return _Fake
|
|
|
|
|
|
def _patch_generic_engine(monkeypatch, calls):
|
|
import services.tts_backend as tb
|
|
fake = _fake_backend_cls(calls)
|
|
monkeypatch.setattr(tb, "active_backend_id", lambda: "fake-longform-engine")
|
|
monkeypatch.setattr(tb, "get_backend_class", lambda _id: fake)
|
|
|
|
|
|
def _record_manual_seed(monkeypatch):
|
|
seeds = []
|
|
real = torch.manual_seed
|
|
monkeypatch.setattr(torch, "manual_seed", lambda s: (seeds.append(s), real(s))[1])
|
|
return seeds
|
|
|
|
|
|
def test_generic_synth_applies_pinned_profile_seed(monkeypatch):
|
|
import api.routers.audiobook as ab
|
|
|
|
calls, seeds = [], _record_manual_seed(monkeypatch)
|
|
_patch_generic_engine(monkeypatch, calls)
|
|
monkeypatch.setattr(ab, "_resolve_voice", lambda _vid: {
|
|
"ref_audio": None, "ref_text": None, "instruct": None, "seed": 1234,
|
|
})
|
|
|
|
info = ab._build_synth("prof-1")
|
|
info["synth"]("hello world", None)
|
|
|
|
assert calls, "stub engine was not reached"
|
|
assert seeds == [segment_seed(1234, "hello world")] # fails before the fix
|
|
|
|
|
|
def test_generic_synth_without_pinned_seed_stays_unseeded(monkeypatch):
|
|
import api.routers.audiobook as ab
|
|
|
|
calls, seeds = [], _record_manual_seed(monkeypatch)
|
|
_patch_generic_engine(monkeypatch, calls)
|
|
monkeypatch.setattr(ab, "_resolve_voice", lambda _vid: {
|
|
"ref_audio": None, "ref_text": None, "instruct": None, "seed": None,
|
|
})
|
|
|
|
info = ab._build_synth(None)
|
|
info["synth"]("hello world", None)
|
|
|
|
assert calls
|
|
assert seeds == [] # fresh-render variety unchanged when nothing is pinned
|
|
|
|
|
|
# ── omnivoice branch: explicit quality preset + pinned seed ──────────────────
|
|
|
|
def test_omnivoice_synth_pins_quality_preset_and_seed(monkeypatch):
|
|
import api.routers.audiobook as ab
|
|
import services.model_manager as mm
|
|
import services.tts_backend as tb
|
|
|
|
gen_calls = []
|
|
|
|
class _FakeModel:
|
|
sampling_rate = 24000
|
|
|
|
def generate(self, **kw):
|
|
gen_calls.append(kw)
|
|
return [torch.zeros(1, 2400)]
|
|
|
|
async def fake_get_model():
|
|
return _FakeModel()
|
|
|
|
seeds = _record_manual_seed(monkeypatch)
|
|
monkeypatch.setattr(tb, "active_backend_id", lambda: "omnivoice")
|
|
monkeypatch.setattr(mm, "get_model", fake_get_model)
|
|
monkeypatch.setattr(ab, "_resolve_voice", lambda _vid: {
|
|
"ref_audio": None, "ref_text": None, "instruct": None, "seed": 42,
|
|
})
|
|
|
|
synth, sr, _resolve, engine_id = asyncio.run(ab._prepare_synth("prof-1"))
|
|
synth("một đoạn văn", None)
|
|
|
|
assert sr == 24000 and engine_id == "omnivoice"
|
|
assert len(gen_calls) == 1
|
|
# The quality preset is explicit, not an accident of model defaults.
|
|
assert gen_calls[0]["num_step"] == ab.LONGFORM_NUM_STEP == 32
|
|
assert gen_calls[0]["guidance_scale"] == ab.LONGFORM_GUIDANCE_SCALE == 2.0
|
|
# The pinned profile seed reached torch, decorrelated per chunk text.
|
|
assert seeds == [segment_seed(42, "một đoạn văn")]
|