Two defensive fixes for non-Latin / long-form synthesis quality:
#502 (Vietnamese clone distorted/unintelligible): the /generate text path never
NFC-normalized its input, so pasted decomposed (NFD) Vietnamese — base letter +
combining diacritic instead of the single composed codepoint — reached the
tokenizer/model as two characters and rendered as garbled speech. Normalize the
input text to NFC at the endpoint (no-op for already-composed text), mirroring
what the duration estimator already does so the estimate and synthesis agree.
#505 (long-form 5+ min degrades — repeated/skipped/mispronounced): the chunker
split purely by character count (800), but CJK/kana/Hangul pack ~1 char =
1 syllable, so an 800-char chunk is ~4-5 minutes of audio in a single shot —
past the model's reliable range, where it starts repeating/skipping. When a
chunk is predominantly dense-script, cap it to max_chars/2.5 so each chunk's
spoken length stays bounded; Latin/spaced text is unchanged. Dense-script
detection is by code point (no literal CJK in source — no-literal-CJK gate stays
clean).
Tests: _dense_char_count, _effective_max_chars (shrink-when-dense, unchanged-for-
Latin, disabled-passthrough, floor), and that a 400-CJK-char string now splits
(was one chunk) while a Latin paragraph still doesn't.
Note: #502's exact distortion still wants a user sample to fully confirm; this is
the defensive NFC fix that's correct regardless.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
