feat(dub): underrun fill — short dubbed lines are slowed toward their slot instead of leaving dead air (#1137)

* feat(dub): underrun fill — short dubbed lines are slowed toward their slot instead of leaving dead air

The dub pipeline has always handled audio that is too LONG for its slot
(atempo compression, Smart Fit's audio/video split, trims). Audio that is too
SHORT was start-aligned and abandoned — and that is the common case, not the
corner: translations routinely speak faster than the source delivery.
Measured on a real 4-segment dub, 8.8 of 18.7 seconds of original speech time
had no dubbed voice. What fills those holes is the separated bed's
under-speech residue (37% of the original energy, measured), so the user
hears them as BOTH "little silences" AND "the music is numbed" — and sees
them as lip-sync failure, since the mouth keeps moving after the dub stopped.

The fill: when a line's natural duration covers less than UNDERRUN_TOLERANCE
(95%) of its slot, slow it toward the slot with the same pitch-preserving
atempo pipe the compression path uses, bounded at min_audio_rate (default
0.85x — comfortably natural; atempo handles <1 natively). Wired into both
fitting strategies:

- fit_planner._fit_one: need < 1 now resolves to audio_rate=max(need, floor),
  status "audio_slowed" — planner stays a pure function; golden fixtures
  regenerated per their own instructions (10 substantive lines: five
  underrun segments across four scenarios flip to audio_slowed@0.85).
- dub_generate smart_fit branch: applies the rate in both directions (the
  target formula was already direction-agnostic).
- dub_generate strict_slot branch: mirror of its compression arm.
- stretch_video and concise strategies deliberately untouched (natural-rate
  by design / never-intervene by design).

OMNIVOICE_UNDERRUN_MIN_RATE overrides the floor (1.0 disables; clamped to
atempo's sane range). The per-segment fit badge shows "slowed N.NNx" with a
tooltip, translated in all 21 locales.

Tests: planner contracts (fill bounded by floor, tolerance zone untouched,
disable switch, empty-audio guard), the flipped unit/golden/integration
expectations updated with the rationale, and the existing smart_fit
integration test now exercises the fill through the real mix loop (its seg0
comes out audio_slowed@0.85 end to end). Full suite: 2987 backend + 1236
frontend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(dub): strict-slot slow-downs report themselves honestly (review); ru pitch wording

Review round on #1137:

- Greptile P1 "slowdown reports fits" — REAL: the strict_slot underrun fill
  fell through to the unconditional {"status": "fits"} entry, so a slowed
  segment's badge hid the applied rate (and compression_applied mislabeled
  it). The branch now emits {"status": "audio_slowed", "audio_rate": …} like
  the smart_fit path — same honesty contract everywhere.
- Greptile P1 "padded audio hides underruns" — REFUTED with evidence: nothing
  pads strict-slot audio before the check (_load_entry_wav returns the
  natural-length WAV; only error/silence slots are slot-sized, and those are
  synthetic silence by design). On-disk segment WAVs measure both shorter and
  longer than their slots, which pre-padding would make impossible.
- CodeRabbit: Russian tooltip now says "высота тона сохранена" (pitch), not
  "высота сохранена" (height).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Palash Debnath
2026-07-13 19:09:02 +05:30
committed by GitHub
co-authored by Claude Opus 4.8 mergetest
parent a7efaa24f7
commit a4d9d9f128
31 changed files with 186 additions and 26 deletions
+2
View File
@@ -14,6 +14,8 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
### Fixed
- **Dubbed lines that finish early no longer leave dead air — they now speak at the pace of the scene.** Translations routinely come out shorter than the original delivery, and the dub used to just stop early: measured on a real dub, **8.8 of 18.7 seconds of speech time had no voice at all** — the mouth kept moving on screen over the thin residue the vocal separation leaves behind, which reads as silence and as "the music got quiet". Short lines are now gently slowed toward their time slot (pitch preserved, never below 0.85× — comfortably natural), so speech covers the speaking time the way the original did. This also does most of the work people expect from "lip sync": the voice now starts *and ends* with the mouth. Near-full lines are left untouched, the per-segment badge shows the applied rate, and `OMNIVOICE_UNDERRUN_MIN_RATE=1.0` turns the fill off. (#1137)
- **The dub's background music no longer comes out quiet and muffled.** Every dub export mixes your synthesized voice over the video's separated music/ambience bed — and that mix had two fidelity bugs stacked on top of each other. The mixer *normalizes* its inputs, so the weights meant to gently favor dialogue actually played the music at **~57% of its original level** (measured); and because the voice track is synthesized at 24 kHz, the mixer silently pulled the 44.1 kHz music down to 24 kHz — deleting everything above 12 kHz: cymbals, brightness, air. The batch pipeline was harsher still, pinning the bed near 8%. All six mix sites now share one filter that resamples both sides up to 48 kHz, cancels the normalization so the music plays at **90% of its true level** (a hair of headroom keeps dialogue legible), and adds a transparent peak limiter. Measured on a real dub: bed level 57% → 90%, bandwidth 12 kHz → 24 kHz. (#1136)
- **A rate-limited translation polish pass no longer sabotages the dub — or lies about it.** The Cinematic quality mode runs an optional critique-and-rewrite pass after translating. When that pass hit a rate limit (free-tier LLM endpoints throttle hard), three bad things happened at once: the app reported **"N/N segment(s) failed"** in red over a translate that had actually succeeded; the affected segments were **silently skipped by the speech-rate fit pass and duration planner** — so overlong lines went to synthesis unfitted and came out audibly time-compressed; and the two-second "retry shortly" hint the provider sent was ignored. All three are fixed: a rate-limited call now waits out the provider's own `Retry-After` (bounded, once) and usually just succeeds; a segment that still misses the polish keeps its plain translation, **stays in every downstream fitting pass**, and is reported honestly — "translated, polish skipped" as a warning with the reason, not a failure. Rows that really failed still say so. (#1135)
+57 -8
View File
@@ -26,7 +26,7 @@ from services.ffmpeg_utils import (
)
from services.rvc import apply_rvc, is_enabled as rvc_is_enabled
from services.incremental import segment_fingerprint, fit_fingerprint
from services.fit_planner import FitParams, plan_fit
from services.fit_planner import UNDERRUN_TOLERANCE, FitParams, plan_fit
from services.watermark import embed_watermark
from api.routers.dub_core import _get_job, _save_job
from omnivoice.utils.voice_design import heal_design_instruct
@@ -41,6 +41,17 @@ logger = logging.getLogger("omnivoice.dub")
# in services/speech_rate.py, gap absorption below) keeps us under this
# in practice — this is only a guard rail.
MAX_STRETCH_RATIO = 1.8
def _underrun_min_rate() -> float:
"""Floor for the underrun fill (audio slowed toward its slot, never below
this rate). Default 0.85 stays natural-sounding; OMNIVOICE_UNDERRUN_MIN_RATE=1.0
disables the fill. Clamped to atempo's per-stage sane range."""
try:
v = float(os.environ.get("OMNIVOICE_UNDERRUN_MIN_RATE", "0.85"))
except ValueError:
v = 0.85
return min(1.0, max(0.5, v))
# How far a too-long segment is allowed to bleed into the silent gap
# before the next segment. Buys headroom on languages with higher
# information density (Bengali, Hindi, Arabic…) without the audio
@@ -884,6 +895,7 @@ async def dub_generate(job_id: str, req: DubRequest):
video_slow_cap=float(getattr(_fo, "video_slow_cap", None) or _fit_defaults.video_slow_cap),
gap_guard_s=float(_fo.gap_guard_s) if _fo is not None and _fo.gap_guard_s is not None else _fit_defaults.gap_guard_s,
allow_video_retime=bool(_fo.allow_video_retime) if _fo is not None and _fo.allow_video_retime is not None else _fit_defaults.allow_video_retime,
min_audio_rate=_underrun_min_rate(),
)
_seg_order = job.get("seg_order") or []
fit_plan = plan_fit(
@@ -958,7 +970,10 @@ async def dub_generate(job_id: str, req: DubRequest):
# chunk) is persisted below for the export pipeline.
sf = fit_plan.segments[i]
place_at = sf.new_start
if sf.audio_rate > 1.0 + 1e-6 and wl > 0:
# Both directions: >1 compresses an overrun, <1 slows an
# underrun toward the slot (the "hole" fix — a dub that
# finishes early leaves the mouth moving over near-silence).
if abs(sf.audio_rate - 1.0) > 1e-6 and wl > 0:
target = max(1, int(round(wl / sf.audio_rate)))
try:
adjusted = await _pitch_preserving_stretch(
@@ -985,7 +1000,7 @@ async def dub_generate(job_id: str, req: DubRequest):
wl = adjusted.shape[-1]
# Truthful per-segment verdict for the UI badge.
entry = {"status": sf.status}
if sf.audio_rate > 1.0 + 1e-6:
if abs(sf.audio_rate - 1.0) > 1e-6:
entry["audio_rate"] = round(sf.audio_rate, 3)
if sf.video_ratio > 1.0 + 1e-6:
entry["video_ratio"] = round(sf.video_ratio, 3)
@@ -1032,6 +1047,7 @@ async def dub_generate(job_id: str, req: DubRequest):
# keep passing.
place_at = start
effective_end = end
slowed_rate = None
if i + 1 < len(all_segment_wavs):
next_start = all_segment_wavs[i + 1][0]
gap = next_start - end
@@ -1072,11 +1088,44 @@ async def dub_generate(job_id: str, req: DubRequest):
else: # "trim"
adjusted = adjusted[..., :slot_samples]
wl = adjusted.shape[-1]
fit_status.append({
"status": "fits",
"compression_applied": (slot_fit == "time_stretch"
and wl != int(natural_dur * sr)),
})
elif (
slot_fit == "time_stretch"
and slot_samples > 0
and wl > 0
and wl < slot_samples * UNDERRUN_TOLERANCE
and _underrun_min_rate() < 1.0 - 1e-6
):
# Underrun fill (mirror of the compression above): the
# dub finished early, leaving the on-screen mouth moving
# over the thin under-speech bed residue — perceived as
# dead air. Slow toward the slot, never below the floor.
rate = max(wl / slot_samples, _underrun_min_rate())
target = min(slot_samples, int(round(wl / rate)))
try:
adjusted = await _pitch_preserving_stretch(
adjusted, target, sr,
)
slowed_rate = rate
except Exception as e:
logger.warning(
"underrun fill failed for seg %d (%.2f×), "
"keeping natural rate: %s", i, rate, e,
)
wl = adjusted.shape[-1]
# Truthful verdict: a slowed segment says so (and by how
# much) instead of hiding behind "fits" — the same honesty
# contract the smart_fit branch keeps.
if slowed_rate is not None:
fit_status.append({
"status": "audio_slowed",
"audio_rate": round(slowed_rate, 3),
})
else:
fit_status.append({
"status": "fits",
"compression_applied": (slot_fit == "time_stretch"
and wl != int(natural_dur * sr)),
})
# Common: short fades to avoid pops, then mix into disk-backed audio.
fade_ms = 15
+21 -1
View File
@@ -45,6 +45,16 @@ from dataclasses import dataclass, field
# garbled stream no DSP can rescue.
MAX_AUDIO_RATE_HARD = 1.8
# Underrun fill: a dubbed line that finishes well before its slot leaves a
# hole — on screen the mouth keeps moving while the dub has gone quiet, and
# what the listener hears in the hole is the thin under-speech residue of the
# separated background (measured at ~37% of the original's energy), which
# reads as dead air. Translations routinely run shorter than the source
# delivery (measured live: 8.8s of holes across 18.7s of speech), so this is
# the common case, not a corner. Slots filled to within this fraction are
# left alone — a <5% hole is imperceptible and not worth an ffmpeg pass.
UNDERRUN_TOLERANCE = 0.95
_EPS = 1e-9
@@ -61,6 +71,10 @@ class FitParams:
video_slow_cap: float = 2.0
gap_guard_s: float = 0.05
allow_video_retime: bool = True
# Underrun fill floor: a segment shorter than its slot is slowed toward it
# (pitch-preserving), never below this rate — 0.85× stays comfortably
# natural-sounding. 1.0 disables the fill entirely.
min_audio_rate: float = 0.85
@dataclass
@@ -68,7 +82,7 @@ class SegmentFit:
"""Planner verdict for one segment."""
index: int
seg_id: str
audio_rate: float # ≥ 1.0 — pitch-preserving speed-up applied to TTS audio
audio_rate: float # pitch-preserving rate: >1 speeds up (fit), <1 slows down (fill)
video_ratio: float # ≥ 1.0 — setpts slow-down applied to the video chunk
new_start: float # placement on the fitted (possibly longer) timeline
new_end: float # end of the video chunk on the fitted timeline
@@ -97,6 +111,12 @@ class FitPlan:
def _fit_one(need: float, params: FitParams) -> tuple[float, float, str]:
"""Resolve one segment's need ratio into (audio_rate, video_ratio, status)."""
if need <= 1.0 + _EPS:
# Underrun fill: slow the audio toward the slot so the dub keeps
# speaking while the on-screen mouth does. Bounded by min_audio_rate;
# near-full slots (within UNDERRUN_TOLERANCE) and degenerate needs
# (empty audio) stay untouched.
if need > _EPS and need < UNDERRUN_TOLERANCE and params.min_audio_rate < 1.0 - _EPS:
return max(need, params.min_audio_rate), 1.0, "audio_slowed"
return 1.0, 1.0, "fits"
if need <= params.max_audio_only_rate + _EPS:
return need, 1.0, "audio_stretched"
+10
View File
@@ -127,6 +127,16 @@ function DubSegmentRow({
label: t('segment.fit_stretched', { ratio: r.toFixed(2) }),
title: t('segment.fit_stretched_title', { ratio: r.toFixed(2) }),
};
} else if (fitStatus.status === 'audio_slowed') {
// Underrun fill: the line ran shorter than its slot and was slowed
// (pitch-preserved) toward it so speech covers the on-screen mouth time.
const r = fitStatus.audio_rate || 1.0;
fitBadge = {
color: '#83a598',
Icon: Circle,
label: t('segment.fit_slowed', { ratio: r.toFixed(2) }),
title: t('segment.fit_slowed_title', { ratio: r.toFixed(2) }),
};
}
} else if (seg.sync_ratio !== undefined) {
const r = seg.sync_ratio;
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "كان النص المترجم أطول من الفتحة الأصلية بمقدار {{seconds}}s. كان الصوت مقصوصًا بشدة. قم بتقصير النص أو قم بتبديل التوقيت إلى \"تمدد الفيديو\".",
"fit_stretched": "فيديو {{ratio}}×",
"fit_stretched_title": "وضع الفيديو الممتد: تم إبطاء فيديو هذا المقطع إلى {{ratio}}× ليناسب الصوت الطبيعي المدبلج.",
"fit_slowed": "أُبطئ {{ratio}}×",
"fit_slowed_title": "السطر المدبلج أقصر من فترته — أُبطئ {{ratio}}× (مع الحفاظ على طبقة الصوت) ليغطي الكلام مدة الحديث الأصلية بدل ترك فراغ",
"fit_compressed_title": "يمثل صوت تحويل النص إلى كلام (TTS) {{pct}}% من الفتحة - وهو مضغوط بشدة.",
"fit_audio_title": "الصوت مناسب داخل الفتحة.",
"fit_ratio_title": "يمثل صوت تحويل النص إلى كلام (TTS) {{pct}}% من الفتحة.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Der übersetzte Text war um {{seconds}}s länger als der ursprüngliche Text. Der Ton war stark gekürzt; kürzen Sie den Text oder stellen Sie das Timing auf „Video dehnen“ um.",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Modus „Video dehnen“: Das Video dieses Segments wurde auf {{ratio}}× verlangsamt, um dem natürlichen Dub-Audio zu entsprechen.",
"fit_slowed": "verlangsamt {{ratio}}×",
"fit_slowed_title": "Die synchronisierte Zeile war kürzer als ihr Zeitfenster — {{ratio}}× verlangsamt (Tonhöhe erhalten), damit die Sprache die originale Sprechzeit abdeckt statt ein Loch zu lassen",
"fit_compressed_title": "TTS-Audio macht {{pct}} % des Slots aus stark komprimiert.",
"fit_audio_title": "Audio passt in den Steckplatz.",
"fit_ratio_title": "TTS-Audio macht {{pct}} % des Slots aus.",
+2
View File
@@ -1202,6 +1202,8 @@
"fit_overflows_title": "Translated text was longer than the original slot by {{seconds}}s. The audio was hard-trimmed; shorten the text or switch Timing to \"Stretch Video\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Stretch Video mode: this segment's video was slowed to {{ratio}}× to fit the natural dub audio.",
"fit_slowed": "slowed {{ratio}}×",
"fit_slowed_title": "The dubbed line ran shorter than its slot — slowed {{ratio}}× (pitch preserved) so speech covers the original speaking time instead of leaving a hole",
"fit_compressed_title": "TTS audio is {{pct}}% of the slot — heavily compressed.",
"fit_audio_title": "Audio fit inside the slot.",
"fit_ratio_title": "TTS audio is {{pct}}% of the slot.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "El texto traducido era más largo que el espacio original en {{seconds}}s. El audio estaba muy recortado; acorte el texto o cambie el tiempo a \"Estirar vídeo\".",
"fit_stretched": "Vídeo {{ratio}}×",
"fit_stretched_title": "Modo Stretch Video: el video de este segmento se ralentizó a {{ratio}}× para adaptarse al audio doblado natural.",
"fit_slowed": "ralentizado {{ratio}}×",
"fit_slowed_title": "La línea doblada duró menos que su intervalo — se ralentizó {{ratio}}× (tono preservado) para que el habla cubra el tiempo original en vez de dejar un hueco",
"fit_compressed_title": "El audio TTS es el {{pct}}% de la ranura: muy comprimido.",
"fit_audio_title": "El audio encaja dentro de la ranura.",
"fit_ratio_title": "El audio TTS es el {{pct}}% de la ranura.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Le texte traduit était plus long que l'emplacement d'origine de {{seconds}}s. Laudio était dur ; raccourcissez le texte ou réglez le timing sur « Étirer la vidéo ».",
"fit_stretched": "Vidéo {{ratio}}×",
"fit_stretched_title": "Mode vidéo extensible : la vidéo de ce segment a été ralentie à {{ratio}}× pour s'adapter à l'audio naturel du doublage.",
"fit_slowed": "ralenti {{ratio}}×",
"fit_slowed_title": "La réplique doublée était plus courte que son créneau — ralentie {{ratio}}× (hauteur préservée) pour couvrir le temps de parole d'origine au lieu de laisser un trou",
"fit_compressed_title": "L'audio TTS représente {{pct}}% de l'emplacement — fortement compressé.",
"fit_audio_title": "L'audio s'adapte à l'intérieur de la fente.",
"fit_ratio_title": "L'audio TTS représente {{pct}}% de l'emplacement.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "अनुवादित पाठ मूल स्लॉट से {{seconds}}s अधिक लंबा था। ऑडियो को हार्ड-ट्रिम किया गया था; टेक्स्ट को छोटा करें या टाइमिंग को \"स्ट्रेच वीडियो\" पर स्विच करें।",
"fit_stretched": "वीडियो {{ratio}}×",
"fit_stretched_title": "स्ट्रेच वीडियो मोड: प्राकृतिक डब ऑडियो में फिट होने के लिए इस सेगमेंट के वीडियो को {{ratio}}× तक धीमा कर दिया गया था।",
"fit_slowed": "धीमा {{ratio}}×",
"fit_slowed_title": "डब की गई पंक्ति अपने स्लॉट से छोटी थी — {{ratio}}× धीमा किया गया (पिच सुरक्षित) ताकि वाणी मूल बोलने के समय को ढक ले और खाली जगह न छूटे",
"fit_compressed_title": "टीटीएस ऑडियो स्लॉट का {{pct}}% है - अत्यधिक संपीड़ित।",
"fit_audio_title": "ऑडियो स्लॉट के अंदर फ़िट हो जाता है।",
"fit_ratio_title": "टीटीएस ऑडियो स्लॉट का {{pct}}% है।",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Teks yang diterjemahkan lebih panjang dari slot aslinya sebanyak {{seconds}}s. Audionya terpotong dengan keras; persingkat teks atau alihkan Timing ke \"Regangkan Video\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Mode Peregangan Video: video segmen ini diperlambat ke {{ratio}}× agar sesuai dengan audio sulih suara alami.",
"fit_slowed": "diperlambat {{ratio}}×",
"fit_slowed_title": "Baris sulih suara lebih pendek dari slotnya — diperlambat {{ratio}}× (nada dipertahankan) agar ucapan menutupi waktu bicara asli, bukan meninggalkan celah",
"fit_compressed_title": "Audio TTS menempati {{pct}}% dari slot — sangat terkompresi.",
"fit_audio_title": "Audio pas di dalam slot.",
"fit_ratio_title": "Audio TTS adalah {{pct}}% dari slot.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Il testo tradotto era più lungo dello spazio originale di {{seconds}}s. L'audio è stato ridotto al minimo; accorcia il testo o imposta il timing su \"Stretch Video\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Modalità Stretch Video: il video di questo segmento è stato rallentato a {{ratio}}× per adattarsi all'audio naturale del doppiaggio.",
"fit_slowed": "rallentato {{ratio}}×",
"fit_slowed_title": "La battuta doppiata era più corta del suo intervallo — rallentata {{ratio}}× (tonalità preservata) così il parlato copre il tempo originale invece di lasciare un vuoto",
"fit_compressed_title": "L'audio TTS è il {{pct}}% dello slot: fortemente compresso.",
"fit_audio_title": "L'audio si inserisce all'interno dello slot.",
"fit_ratio_title": "L'audio TTS è il {{pct}}% dello slot.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "翻訳されたテキストが元のスロットより {{seconds}}s 長くなりました。音声は大幅にトリミングされました。テキストを短くするか、タイミングを「ビデオをストレッチ」に切り替えてください。",
"fit_stretched": "ビデオ {{ratio}}×",
"fit_stretched_title": "ストレッチ ビデオ モード: このセグメントのビデオは、自然なダブ オーディオに合わせて {{ratio}}× まで遅くされました。",
"fit_slowed": "減速 {{ratio}}×",
"fit_slowed_title": "吹き替えの台詞が元の時間より短かったため {{ratio}}× 減速(ピッチ維持)— 空白を残さず元の発話時間をカバーします",
"fit_compressed_title": "TTS オーディオはスロットの {{pct}}% であり、高度に圧縮されています。",
"fit_audio_title": "オーディオはスロット内に収まります。",
"fit_ratio_title": "TTS オーディオはスロットの {{pct}}% です。",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "번역된 텍스트가 원래 슬롯보다 {{seconds}}s 길었습니다. 오디오가 잘 다듬어졌습니다. 텍스트를 줄이거나 타이밍을 \"비디오 늘이기\"로 전환하세요.",
"fit_stretched": "비디오 {{ratio}}×",
"fit_stretched_title": "비디오 늘이기 모드: 자연스러운 더빙 오디오에 맞추기 위해 이 세그먼트의 비디오가 {{ratio}}× 속도로 느려졌습니다.",
"fit_slowed": "감속 {{ratio}}×",
"fit_slowed_title": "더빙된 대사가 슬롯보다 짧아 {{ratio}}× 감속(음높이 유지) — 공백 대신 원래 말하는 시간을 채웁니다",
"fit_compressed_title": "TTS 오디오는 슬롯의 {{pct}}%입니다. — 심하게 압축되었습니다.",
"fit_audio_title": "오디오는 슬롯 안에 맞습니다.",
"fit_ratio_title": "TTS 오디오는 슬롯의 {{pct}}%입니다.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "De vertaalde tekst was {{seconds}}s langer dan de oorspronkelijke tekst. De audio was hard getrimd; verkort de tekst of schakel Timing naar \"Video uitrekken\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Stretch Video-modus: de video van dit segment is vertraagd tot {{ratio}}× om te passen bij de natuurlijke dub-audio.",
"fit_slowed": "vertraagd {{ratio}}×",
"fit_slowed_title": "De nagesynchroniseerde zin was korter dan zijn slot — {{ratio}}× vertraagd (toonhoogte behouden) zodat de spraak de originele spreektijd dekt in plaats van een gat te laten",
"fit_compressed_title": "TTS-audio is {{pct}}% van de sleuf zwaar gecomprimeerd.",
"fit_audio_title": "Audio past in de sleuf.",
"fit_ratio_title": "TTS-audio is {{pct}}% van de sleuf.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Przetłumaczony tekst był dłuższy niż oryginalny fragment o {{seconds}}s. Dźwięk został mocno przycięty; skróć tekst lub przełącz synchronizację na „Rozciągnij wideo”.",
"fit_stretched": "Wideo {{ratio}}×",
"fit_stretched_title": "Tryb rozciągania wideo: wideo tego segmentu zostało spowolnione do {{ratio}}×, aby dopasować je do naturalnego dźwięku dubowego.",
"fit_slowed": "spowolniono {{ratio}}×",
"fit_slowed_title": "Zdubbingowana kwestia była krótsza niż jej slot — spowolniona {{ratio}}× (z zachowaniem wysokości), by mowa pokryła oryginalny czas wypowiedzi zamiast zostawiać dziurę",
"fit_compressed_title": "Dźwięk TTS stanowi {{pct}}% szczeliny — jest mocno skompresowany.",
"fit_audio_title": "Dźwięk mieści się w gnieździe.",
"fit_ratio_title": "Dźwięk TTS zajmuje {{pct}}% szczeliny.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "O texto traduzido era maior que o espaço original em {{seconds}}s. O áudio foi cortado; encurte o texto ou mude o tempo para \"Alongar vídeo\".",
"fit_stretched": "Vídeo {{ratio}}×",
"fit_stretched_title": "Modo Stretch Video: o vídeo deste segmento foi desacelerado para {{ratio}}× para se ajustar ao áudio de dublagem natural.",
"fit_slowed": "desacelerado {{ratio}}×",
"fit_slowed_title": "A fala dublada ficou mais curta que o seu intervalo — desacelerada {{ratio}}× (tom preservado) para cobrir o tempo de fala original em vez de deixar um buraco",
"fit_compressed_title": "O áudio TTS é {{pct}}% do slot fortemente compactado.",
"fit_audio_title": "O áudio cabe dentro do slot.",
"fit_ratio_title": "O áudio TTS é {{pct}}% do slot.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Переведенный текст был длиннее исходного слота на {{seconds}}s. Звук был жестко обрезан; сократите текст или переключите время на «Растянуть видео».",
"fit_stretched": "Видео {{ratio}}×",
"fit_stretched_title": "Режим растягивания видео: видео этого сегмента было замедлено до {{ratio}}×, чтобы соответствовать естественному звуку дубляжа.",
"fit_slowed": "замедлено {{ratio}}×",
"fit_slowed_title": "Дублированная реплика короче своего интервала — замедлена {{ratio}}× (высота тона сохранена), чтобы речь покрывала исходное время, а не оставляла провал",
"fit_compressed_title": "Звук TTS занимает {{pct}}% слота и сильно сжат.",
"fit_audio_title": "Аудио поместилось внутри слота.",
"fit_ratio_title": "Звук TTS занимает {{pct}}% слота.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Den översatta texten var längre än den ursprungliga platsen med {{seconds}}s. Ljudet var hårt trimmat; förkorta texten eller ändra Timing till \"Stretch Video\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Stretch Video-läge: detta segments video saktades ner till {{ratio}}× för att passa det naturliga dubbljudet.",
"fit_slowed": "saktad {{ratio}}×",
"fit_slowed_title": "Den dubbade repliken var kortare än sin lucka — saktad {{ratio}}× (tonhöjd bevarad) så talet täcker den ursprungliga taltiden i stället för att lämna ett hål",
"fit_compressed_title": "TTS-ljud är {{pct}}% av kortplatsen — kraftigt komprimerat.",
"fit_audio_title": "Ljudet passar in i öppningen.",
"fit_ratio_title": "TTS-ljud är {{pct}}% av kortplatsen.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "ข้อความที่แปลยาวกว่าช่องต้นฉบับ {{seconds}}s เสียงถูกตัดแต่งอย่างหนัก ย่อข้อความหรือเปลี่ยนระยะเวลาเป็น \"ยืดวิดีโอ\"",
"fit_stretched": "วิดีโอ {{ratio}}×",
"fit_stretched_title": "โหมดวิดีโอยืด: วิดีโอของส่วนนี้ช้าลงเป็น {{ratio}}× เพื่อให้พอดีกับเสียงพากย์ที่เป็นธรรมชาติ",
"fit_slowed": "ช้าลง {{ratio}}×",
"fit_slowed_title": "บทพากย์สั้นกว่าช่วงเวลา — ทำให้ช้าลง {{ratio}}× (คงระดับเสียง) เพื่อให้เสียงพูดครอบคลุมช่วงพูดเดิมแทนที่จะทิ้งช่องว่าง",
"fit_compressed_title": "เสียง TTS มีค่า {{pct}}% ของช่อง — บีบอัดอย่างหนัก",
"fit_audio_title": "เสียงพอดีกับช่อง",
"fit_ratio_title": "เสียง TTS คือ {{pct}}% ของช่อง",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Çevrilen metin orijinal alandan {{seconds}}s daha uzundu. Ses sert bir şekilde kesilmişti; metni kısaltın veya Zamanlamayı \"Videoyu Uzat\" olarak değiştirin.",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Uzatılmış Video modu: Bu bölümün videosu, doğal dub sese uyacak şekilde {{ratio}}× değerine yavaşlatıldı.",
"fit_slowed": "yavaşlatıldı {{ratio}}×",
"fit_slowed_title": "Dublajlı replik zaman aralığından kısa kaldı — {{ratio}}× yavaşlatıldı (perde korundu), böylece konuşma boşluk bırakmak yerine orijinal konuşma süresini kaplıyor",
"fit_compressed_title": "TTS sesi yuvanın %{{pct}}'si kadardır ve yoğun şekilde sıkıştırılmıştır.",
"fit_audio_title": "Ses yuvanın içine sığar.",
"fit_ratio_title": "TTS sesi yuvanın %{{pct}} kadarıdır.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Перекладений текст був довший за оригінальний слот на {{seconds}}s. Аудіо було жорстко обрізано; скоротіть текст або перемкніть параметри часу на «Розтягнути відео».",
"fit_stretched": "Відео {{ratio}}×",
"fit_stretched_title": "Режим розтягнутого відео: відео цього сегменту було сповільнено до {{ratio}}×, щоб відповідати природному дублюванню звуку.",
"fit_slowed": "уповільнено {{ratio}}×",
"fit_slowed_title": "Дубльована репліка коротша за свій інтервал — уповільнена {{ratio}}× (висота збережена), щоб мовлення покривало оригінальний час, а не лишало прогалину",
"fit_compressed_title": "Аудіо TTS займає {{pct}}% слота — сильно стиснуте.",
"fit_audio_title": "Аудіо вміщується в слот.",
"fit_ratio_title": "Аудіо TTS займає {{pct}}% слота.",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "Văn bản được dịch dài hơn vị trí ban đầu {{seconds}}s. Âm thanh đã được cắt bớt cứng; rút ngắn văn bản hoặc chuyển Thời gian thành \"Căng video\".",
"fit_stretched": "Video {{ratio}}×",
"fit_stretched_title": "Chế độ Kéo dài video: video của phân đoạn này được giảm tốc độ xuống {{ratio}}× để phù hợp với âm thanh lồng tiếng tự nhiên.",
"fit_slowed": "làm chậm {{ratio}}×",
"fit_slowed_title": "Câu lồng tiếng ngắn hơn khung thời gian — làm chậm {{ratio}}× (giữ cao độ) để lời nói phủ hết thời gian thoại gốc thay vì để khoảng trống",
"fit_compressed_title": "Âm thanh TTS chiếm {{pct}}% dung lượng khe — bị nén rất nhiều.",
"fit_audio_title": "Âm thanh vừa vặn bên trong khe cắm.",
"fit_ratio_title": "Âm thanh TTS chiếm {{pct}}% dung lượng khe.",
+2
View File
@@ -899,6 +899,8 @@
"fit_overflows_title": "翻译后的文本比原始文本长 {{seconds}}s。音频经过严格修剪;缩短文本或将计时切换为“拉伸视频”。",
"fit_stretched": "视频{{ratio}}×",
"fit_stretched_title": "拉伸视频模式:该片段的视频速度减慢至 {{ratio}}× 以适应自然的配音音频。",
"fit_slowed": "放慢 {{ratio}}×",
"fit_slowed_title": "配音台词比原时段短 — 放慢 {{ratio}}×(保持音高),让语音覆盖原说话时间而不留空洞",
"fit_compressed_title": "TTS 音频占插槽的 {{pct}}% — 被严重压缩。",
"fit_audio_title": "音频适合插槽内。",
"fit_ratio_title": "TTS 音频占插槽的 {{pct}}%。",
+2
View File
@@ -940,6 +940,8 @@
"fit_overflows_title": "翻譯後的文字比原始文字長 {{seconds}}s。音訊經過嚴格修剪;縮短文字或將計時切換為“拉伸視訊”。",
"fit_stretched": "視訊{{ratio}}×",
"fit_stretched_title": "拉伸視訊模式:此片段的視訊速度減慢至 {{ratio}}× 以適應自然的配音音訊。",
"fit_slowed": "放慢 {{ratio}}×",
"fit_slowed_title": "配音台詞比原時段短 — 放慢 {{ratio}}×(保持音高),讓語音覆蓋原說話時間而不留空洞",
"fit_compressed_title": "TTS 音訊佔插槽的 {{pct}}% — 被嚴重壓縮。",
"fit_audio_title": "音訊適合插槽內。",
"fit_ratio_title": "TTS 音訊佔插槽的 {{pct}}%。",
+4 -4
View File
@@ -2,7 +2,7 @@
"orig_duration": 6.0,
"segments": [
{
"audio_rate": 1.0,
"audio_rate": 0.85,
"effective_end": 2.95,
"index": 0,
"new_end": 2.95,
@@ -11,11 +11,11 @@
"orig_start": 0.0,
"overflow_s": 0.0,
"seg_id": "g0",
"status": "fits",
"status": "audio_slowed",
"video_ratio": 1.0
},
{
"audio_rate": 1.0,
"audio_rate": 0.85,
"effective_end": 6.0,
"index": 1,
"new_end": 6.0,
@@ -24,7 +24,7 @@
"orig_start": 3.0,
"overflow_s": 0.0,
"seg_id": "g1",
"status": "fits",
"status": "audio_slowed",
"video_ratio": 1.0
}
],
+2 -2
View File
@@ -28,7 +28,7 @@
"video_ratio": 1.17444
},
{
"audio_rate": 1.0,
"audio_rate": 0.85,
"effective_end": 5.0,
"index": 2,
"new_end": 5.2529,
@@ -37,7 +37,7 @@
"orig_start": 3.0,
"overflow_s": 0.0,
"seg_id": "g2",
"status": "fits",
"status": "audio_slowed",
"video_ratio": 1.0
}
],
+2 -2
View File
@@ -15,7 +15,7 @@
"video_ratio": 1.0
},
{
"audio_rate": 1.0,
"audio_rate": 0.85,
"effective_end": 4.0,
"index": 1,
"new_end": 4.0,
@@ -24,7 +24,7 @@
"orig_start": 2.0,
"overflow_s": 0.0,
"seg_id": "g1",
"status": "fits",
"status": "audio_slowed",
"video_ratio": 1.0
}
],
+2 -2
View File
@@ -15,7 +15,7 @@
"video_ratio": 2.0
},
{
"audio_rate": 1.0,
"audio_rate": 0.85,
"effective_end": 3.5,
"index": 1,
"new_end": 4.95,
@@ -24,7 +24,7 @@
"orig_start": 2.0,
"overflow_s": 0.0,
"seg_id": "g1",
"status": "fits",
"status": "audio_slowed",
"video_ratio": 1.0
}
],
+38 -4
View File
@@ -44,13 +44,46 @@ def _single_seg_plan(natural: float, *, slot: float = 1.0, params: FitParams | N
# ── Threshold boundaries ───────────────────────────────────────────────
def test_need_below_one_fits():
def test_need_below_one_is_slowed_toward_the_slot():
"""Underrun fill: a line shorter than its slot is slowed (pitch-preserving)
so speech covers the on-screen mouth time instead of leaving a hole of
thin bed residue (measured live: 8.8s of holes across 18.7s of speech)."""
p = _single_seg_plan(0.9)
sf = p.segments[0]
assert sf.status == "audio_slowed"
assert sf.audio_rate == pytest.approx(0.9) # exactly fills the slot
assert sf.video_ratio == 1.0
assert sf.overflow_s == 0.0
def test_underrun_fill_is_bounded_by_the_floor():
"""A drastically short line only slows to min_audio_rate — 0.6× speech
would sound wrong; a smaller hole remains, honestly."""
p = _single_seg_plan(0.6)
sf = p.segments[0]
assert sf.status == "audio_slowed"
assert sf.audio_rate == pytest.approx(FitParams().min_audio_rate)
def test_near_full_slots_are_left_alone():
"""Within UNDERRUN_TOLERANCE the hole is imperceptible — no ffmpeg pass."""
p = _single_seg_plan(0.97)
sf = p.segments[0]
assert sf.status == "fits"
assert sf.audio_rate == 1.0
assert sf.video_ratio == 1.0
assert sf.overflow_s == 0.0
def test_underrun_fill_disabled_via_min_audio_rate():
p = _single_seg_plan(0.6, params=FitParams(min_audio_rate=1.0))
sf = p.segments[0]
assert sf.status == "fits"
assert sf.audio_rate == 1.0
def test_empty_audio_is_not_slowed():
p = _single_seg_plan(0.0)
assert p.segments[0].status == "fits"
assert p.segments[0].audio_rate == 1.0
def test_need_exactly_one_fits():
@@ -155,7 +188,8 @@ def test_last_segment_absorbs_tail_to_video_end():
)
sf = p.segments[0]
assert sf.effective_end == pytest.approx(5.0)
assert sf.status == "fits" # 4.5 / 5.0
assert sf.status == "audio_slowed" # 4.5 / 5.0 → filled toward the slot
assert sf.audio_rate == pytest.approx(0.9)
assert p.total_duration == pytest.approx(5.0)
+6 -3
View File
@@ -269,8 +269,10 @@ def test_zero_and_negative_duration_segments_dont_crash(patched_generate):
def test_smart_fit_audio_only_stretch_keeps_original_duration(patched_generate):
run, model, job, job_dir = patched_generate
# seg0 [0,1] natural 0.5s → fits. seg1 [2,3] is last → slot extends to
# 4.0s (2.0s); natural 2.2s → need 1.1 → audio-only 1.1×, no video.
# seg0 [0,1] natural 0.5s → far short of its slack-extended ~1.95s slot →
# underrun fill slows it at the 0.85× floor (it still ends inside the
# slot). seg1 [2,3] is last → slot extends to 4.0s (2.0s); natural 2.2s
# → need 1.1 → audio-only 1.1×, no video.
segs = [
{"start": 0.0, "end": 1.0, "text": "0.5:hola"},
{"start": 2.0, "end": 3.0, "text": "2.2:buenos dias"},
@@ -279,7 +281,8 @@ def test_smart_fit_audio_only_stretch_keeps_original_duration(patched_generate):
assert done["timing_strategy"] == "smart_fit"
fs = done["fit_status"]
assert fs[0] == {"status": "fits"}
assert fs[0]["status"] == "audio_slowed"
assert fs[0]["audio_rate"] == pytest.approx(0.85, abs=1e-3)
assert fs[1]["status"] == "audio_stretched"
assert fs[1]["audio_rate"] == pytest.approx(1.1, abs=1e-3)
assert "video_ratio" not in fs[1]