"20~30초" is read aloud as a single number — OmniVoice says "이십삼" (23).
The separator never reaches the listener, so any written range is heard as
the wrong figure.
`normalize_text` only ran its number pass behind `_num2words_lang`, which
returns None for ko/ja/zh/th/vi (those scripts read digits natively and are
deliberately outside num2words). Nothing else looked at the range mark, so
the tilde went to the engine untouched and the two numbers ran together.
Rewrite `N~M` into the spoken form before the engine sees it, outside the
num2words gate so the CJK languages are covered too. Verified by rendering
each candidate and transcribing it back (ko, OmniVoice, cloned voice):
"대략 20~30초짜리" heard "23초" WRONG
"대략 20-30초짜리" heard "23초" WRONG (reproduces it)
"대략 20에서 30초짜리" heard "20에서 30초짜리" correct
"20〜30分ぐらい" → "20から30分" heard "20〜30分くらい" correct
Deliberately narrow:
* Only the tilde family (U+007E, U+301C, U+FF5E). Japanese and Korean IMEs
emit the latter two. An ASCII hyphen is left alone — between digits it
also spells dates, phone numbers and product codes, where "to" is wrong
(`tests` already pin "pages 3-5" as unchanged).
* Only languages with a verified spoken form (ko/ja/zh/en). Anything else
keeps its tilde, matching how `_PERCENT_WORD` is scoped.
* Spacing belongs to the form, not the caller: a Korean postposition binds
to its numeral ("20에서 30"), Japanese and Chinese set no spaces, English
needs them on both sides.
* Neighbour guards block digits and ASCII letters but allow CJK, because
CJK writes the unit hard against the digits ("20~30초"); a `\w` guard
rejects exactly the cases the rule exists for.
`ko`/`ja`/`zh` join `_FULL_NAME_TO_CODE` so the new resolver can see them.
They stay out of `_NUM2WORDS_LANGS`, so this does not open a num2words path
for them — the same inert-entry pattern the file already documents for
"vietnamese".
`backend/services/text_normalization.py` joins the functional-CJK allowlist
in tests/test_no_hardcoded_cjk.py, under the text-processing group and by
the procedure that file documents: the range words are engine input, not
user-facing UI strings.
Tests: 7 new change-cases and 8 new leave-unchanged cases (hyphen, date,
phone number, product code, decimals, a non-numeric tilde, an unverified
language, and no language at all). All 7 change-cases fail against the
previous implementation.
Full suites before and after: the same 17 failures, none of them touched by
this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(tts): Vietnamese consistency — Voice vs Audiobook divergences (#1139)
Three root causes behind "Vietnamese Voice generation is inconsistent
compared to Audiobook":
1. Numbers: num2words' vi cardinals are wrong for 2001-2099 (misused
"lẻ": 2024 → "hai nghìn lẻ hai mươi bốn") and vi has no year form, so
normalization mangled the years the engine used to read natively.
Vietnamese now keeps its digits, and _num2words_lang's display-name
path now gates on _NUM2WORDS_LANGS like the ISO path (the loophole
that let "Vietnamese" bypass the vetting "vi" would have failed).
2. Seed: the longform resolver fetched a profile's pinned seed but only
the cache signature ever used it — book renders ran unseeded. Both
longform synth wrappers now seed torch per segment via the new pure
segment_seed(base_seed, text) helper (crc32-decorrelated, order- and
cache-independent, mirroring /generate's used_seed + i).
3. Quality preset: the audiobook synth inherited num_step=32 /
guidance_scale=2.0 from model-config defaults by accident of
omission while /generate defaults to 16 — the main audible gap.
Now explicit (LONGFORM_NUM_STEP / LONGFORM_GUIDANCE_SCALE), pinned
by a test so upstream default drift can't silently change books.
The Voice-page fast default (16) is deliberately unchanged.
Also (issue part 3): the finished audiobook's player + Download link
lived in component useState and evaporated on tab switch — the last
render's filename is now store-backed and persisted.
Regression tests fail-before/pass-after (verified by stashing the fix):
vi digit passthrough + vetted-set gate invariant; segment_seed +
seeding in both synth branches + explicit preset kwargs; lastOutput
store round-trip. Full backend suite 3004 passed; frontend 1237 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ui): loadProject clears lastOutput; document longform seeding contracts (review)
Review-bot findings on #1142, evaluated:
- FIXED (Greptile P1 "Output Escapes Its Project" + CodeRabbit):
loadProject now resets lastOutput like newProject already did, so
loading project B never presents A's finished render as B's output.
Regression test added (set lastOutput → loadProject → cleared).
- REFUTED (P1 "Global RNG Races Between Workers"): the exposure is
identical to /generate's existing #526 seeding — generation.py calls
torch.manual_seed on the same global RNG inside the same GPU pool,
and has since that PR. The pool is 1 worker on MPS/CPU and small-VRAM
CUDA (model_manager._pick_gpu_workers), where determinism is strict;
a >1-worker CUDA pool is best-effort for BOTH paths. A race-free fix
means threading a per-call torch.Generator through the model's
samplers app-wide (covering /generate too) — out of scope for this
PR and pointless to do one-sided. Contract now documented on
_seed_segment_rng.
- REFUTED (P2 "Repeated Text Reuses One Seed"): identical takes for
identical repeated lines is the pipeline's shipped semantic — the
content-addressed SegmentCache (segment_cache_key hashes text +
voice sig, not position) already replays one WAV for every identical
span — and seeding only activates when the user pinned a seed, i.e.
asked for reproducibility. Position-based keys would shift every
later span's seed on a one-paragraph insert, breaking the
cache-independent partial re-render guarantee. Documented on
segment_seed.
- DECLINED (P2 "Persisted Filename Can Outlive File"): longform
outputs in OUTPUTS_DIR are not auto-pruned (prune_cache_dir bounds
only longform_cache), so a dangling name requires manual deletion;
auto-clearing on an <audio> error would instead wipe a valid link
whenever the backend is briefly down at mount. Projects → Audiobooks
stays the authoritative library.
Also rebased onto main past #1141 (CHANGELOG resolved keeping both
Unreleased→Fixed entries, this PR's on top).
Affected suites: 264 passed; frontend format clean, 1245 tests passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
New conservative, idempotent pre-TTS normalization pass
(services/text_normalization.py): strips zero-width/control junk, caps
pathological repeat runs, expands digits/times/ordinals/currency via
num2words (29 locales) and per-language abbreviation maps (EN/DE/ES/FR).
Wired once at each text-to-engine choke point — /generate, dub segments
(+ preview), and longform chapters — BEFORE the pronunciation dictionary
so user respellings stay the final say. Pref-gated
(text_normalization_enabled, default ON) with OMNIVOICE_TEXT_NORMALIZATION
env override; num2words promoted to a direct dependency.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Palash Debnath