* feat: onboarding demos, opt-in bug reporting, error-docs deeplinks + issue triage Working-tree snapshot bundling several in-flight workstreams (v0.3.0): - Onboarding/demo system: DemoPresetGrid, DictationDemo, DubbingDemo components + tests, render scripts (render_demos_omnivoice.py, build_demos.sh, build_dub_demo.sh), personalities preview URLs, alembic 0002 voice-profile demo fields. - Opt-in bug reporting: ReportBugButton (prefilled GitHub-issue URL path). - Error transparency UX: errorDocsMap deeplinks + BootstrapSplash/error wiring. - Dub workspace: DubSegmentRow/Table, WaveformTimeline, dubSlice tweaks. - Issue triage: .planning/issue-clusters/ (plan-01..05 root-cause masters, GH #128-#132). - CLAUDE.md: hard rule — everything ships on v0.3.0, no version bumps. KNOWN GAP (why this is a draft): the generated demo audio assets are NOT in this tree, and backend/assets/samples/demo_voice.wav is deleted. onboarding.py guards the missing file (skips seeding the demo profile with a warning), so no crash — but first-run Launchpad will be empty and /demo_audio/ preview URLs 404 until assets are regenerated via scripts/build_demos.sh. Do not merge before regenerating + committing the demo assets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dub): timing strategies — kill audio compression, add Concise + Stretch Video Replaces the current audio time-compression default (atempo squeeze to fit slot) that produced chipmunk/alien output on high-density target languages like Bengali. Two new user-selectable modes; legacy behaviour kept behind an explicit "Strict slot" choice. New `DubRequest.timing_strategy` enum (default "concise"): - "concise" Translator trims text to fit at natural rate; if it still overflows, hard-trim at slot with a fade so we never overlap the next speaker. Surface overflow_s per segment so the user can shorten the text. - "stretch_video" Audio plays at natural 1.0× rate. Backend computes a per-segment new timeline; persists a video_stretch_plan on the job. Mux step (dub_export) builds an ffmpeg trim+setpts+concat filter graph that stretches each segment's video portion to match the natural-rate dub audio. Gaps/pre-roll/tail pass through at 1.0×. Sub burn under stretch_video is skipped in one pass (cues would drift). - "strict_slot" Legacy atempo squeeze. Retained for back-compat. Director rate-bias side-effect (seg_speed *= bias) now gated on strict_slot only, so "urgent"/"slow" direction tokens keep their instruct effect in the new modes without chipmunking. Per-segment fit_status emitted in the SSE done event: {status: "fits" | "overflows" | "video_stretched", overflow_s?, stretch_ratio?} DubSegmentRow's "Sync: 100%" badge (which was lying — sync_ratio was always ~1.0 because the TTS loop pre-trimmed to slot) is replaced with a truthful "Fits / Overflows +Ns / Video 1.18×" label. Frontend: - prefsSlice.timingStrategy (persisted, store v3→v4 with safe migrate). - DubTab footer Segmented control: "Concise · Stretch Video · Strict slot". - useDubWorkflow passes timing_strategy on /dub/generate; consumes fit_status. Tests: tests/test_dub_timing_strategy.py — 13 cases covering schema defaults/validation, _build_video_stretch_filter_graph (pre-roll, gap, tail, empty-plan early return, post-subtitle chain-in), and _video_stretch_plan_for guards. 30/30 existing dub tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(waveform): surface missing source as "Source media missing" instead of code-4 black box When a project's underlying media file is gone (moved or deleted between save and reload) the <video> element fires MediaError code 4 and the companion audio fetch returns HTTP 404 — both were silently warned to the console while the user stared at an unresponsive black panel and an empty waveform. - WaveformTimeline now flips loadError when the video element rejects code 3 (decode) or 4 (src not supported), and tracks `sourceMissing` separately so the error UI can name the actual problem. - The audio decode fallback chain catches HTTP 404 specifically and treats it as source-missing instead of loading silent empty peaks — an empty waveform on a deleted source is more confusing than a clear "Re-upload the video to continue" message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(tray): "Show OmniVoice" reloads when the webview is blank When the dev Vite server restarts (or the main window is created before the backend is ready), the webview load fails and the window is left with `<body></body>` plus a "Could not connect to the server" console error. Clicking "Show OmniVoice" from the tray menu just re-showed the broken window — there was no recovery path short of quit+relaunch. Now the show handler runs a tiny eval after `show()`/`set_focus()` that calls `location.reload()` only when `document.body.childElementCount === 0`. A healthy window doesn't blink (body is non-empty); a blank one self-recovers as soon as the user clicks Show. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(#133): bug-report diagnostics field mapping + drop unused imports Address PR #133 review: - ReportBugButton: /system/info exposes `platform` + `device`, not `os`/`torch_device`/`gpu` — those reads silently dropped OS/GPU from every bug report. Map to the real fields (CodeRabbit). Also remove the dead `home` local in stripHome (CodeQL unused-variable). - DictationDemo: drop unused `Loader` import (CodeQL unused-import). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
155 lines
6.1 KiB
Python
155 lines
6.1 KiB
Python
"""Regression tests for the dub timing-strategy feature.
|
|
|
|
Covers:
|
|
- DubRequest schema defaults — `timing_strategy="concise"` is the new
|
|
safe default; back-compat `slot_fit="time_stretch"` is still accepted.
|
|
- `_build_video_stretch_filter_graph` — builds a deterministic ffmpeg
|
|
filter_complex graph that splits the source video into per-segment
|
|
chunks, setpts each, and concats. Includes gap and pre/tail handling.
|
|
- `_video_stretch_plan_for` — only returns a plan when the job's
|
|
`timing_strategy == "stretch_video"` AND a plan was persisted for
|
|
that lang_code, otherwise None.
|
|
|
|
Why these helpers and not the full mix loop: the mix loop lives inside
|
|
the `dub_generate` async generator, which needs a loaded TTS model to
|
|
exercise end-to-end. The helpers tested here are the new pure logic the
|
|
mode introduced — if they hold, the integration is essentially correct
|
|
modulo the actual ffmpeg call (which is exercised by the existing dub
|
|
smoke tests).
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from schemas.requests import DubRequest, DubSegment
|
|
from api.routers.dub_export import (
|
|
_build_video_stretch_filter_graph,
|
|
_video_stretch_plan_for,
|
|
)
|
|
|
|
|
|
# ── DubRequest schema defaults ────────────────────────────────────────
|
|
|
|
|
|
def _minimal_segs():
|
|
return [DubSegment(start=0.0, end=1.0, text="hi")]
|
|
|
|
|
|
def test_dubrequest_defaults_to_concise():
|
|
req = DubRequest(segments=_minimal_segs())
|
|
assert req.timing_strategy == "concise"
|
|
assert req.overflow_budget_s == 0.0
|
|
# slot_fit default retained for legacy callers that still send it.
|
|
assert req.slot_fit == "time_stretch"
|
|
|
|
|
|
@pytest.mark.parametrize("strategy", ["concise", "stretch_video", "strict_slot"])
|
|
def test_dubrequest_accepts_all_three_strategies(strategy):
|
|
req = DubRequest(segments=_minimal_segs(), timing_strategy=strategy)
|
|
assert req.timing_strategy == strategy
|
|
|
|
|
|
def test_dubrequest_rejects_unknown_strategy():
|
|
with pytest.raises(Exception):
|
|
DubRequest(segments=_minimal_segs(), timing_strategy="warp_speed")
|
|
|
|
|
|
# ── _build_video_stretch_filter_graph ─────────────────────────────────
|
|
|
|
|
|
def test_filter_graph_empty_plan_returns_empty_string_and_passthrough_label():
|
|
graph, label = _build_video_stretch_filter_graph(plan=[], orig_dur=10.0)
|
|
assert graph == ""
|
|
assert label == "[0:v]"
|
|
|
|
|
|
def test_filter_graph_single_segment_no_pre_or_tail():
|
|
"""If the seg covers the whole video, no extra chunks are emitted."""
|
|
plan = [{
|
|
"orig_start": 0.0, "orig_end": 5.0,
|
|
"new_start": 0.0, "new_end": 6.0,
|
|
"stretch_ratio": 1.2,
|
|
}]
|
|
graph, label = _build_video_stretch_filter_graph(plan, orig_dur=5.0)
|
|
assert label == "[vstretched]"
|
|
# One split node, one trim+setpts node, one concat — concat n=1.
|
|
assert "split=1" in graph
|
|
assert "trim=start=0.0000:end=5.0000" in graph
|
|
assert "setpts=1.200000*PTS" in graph
|
|
assert "concat=n=1:v=1:a=0[vstretched]" in graph
|
|
|
|
|
|
def test_filter_graph_pre_roll_gap_and_tail_all_at_native_rate():
|
|
"""Pre-roll, inter-segment gap, and tail should each be a 1.0x chunk."""
|
|
plan = [
|
|
{"orig_start": 1.0, "orig_end": 3.0, "new_start": 1.0, "new_end": 3.5, "stretch_ratio": 1.25},
|
|
{"orig_start": 4.0, "orig_end": 6.0, "new_start": 4.5, "new_end": 6.7, "stretch_ratio": 1.10},
|
|
]
|
|
graph, label = _build_video_stretch_filter_graph(plan, orig_dur=8.0)
|
|
assert label == "[vstretched]"
|
|
# Chunks: [0,1]@1.0 (pre-roll), [1,3]@1.25, [3,4]@1.0 (gap), [4,6]@1.10, [6,8]@1.0 (tail)
|
|
assert "split=5" in graph
|
|
assert "concat=n=5:v=1:a=0[vstretched]" in graph
|
|
# Native-rate chunks are emitted with setpts=1.000000 so the graph stays uniform.
|
|
assert graph.count("setpts=1.000000*PTS") == 3
|
|
assert "setpts=1.250000*PTS" in graph
|
|
assert "setpts=1.100000*PTS" in graph
|
|
|
|
|
|
def test_filter_graph_chains_after_subtitle_filter():
|
|
"""When `in_label` is supplied, the graph reads from that label instead of
|
|
the raw source — used when subtitles burn into [vsub] first."""
|
|
plan = [{
|
|
"orig_start": 0.0, "orig_end": 2.0,
|
|
"new_start": 0.0, "new_end": 2.5,
|
|
"stretch_ratio": 1.25,
|
|
}]
|
|
graph, label = _build_video_stretch_filter_graph(
|
|
plan, orig_dur=2.0, in_label="[vsub]",
|
|
)
|
|
assert label == "[vstretched]"
|
|
assert graph.startswith("[vsub]split=1")
|
|
|
|
|
|
# ── _video_stretch_plan_for ───────────────────────────────────────────
|
|
|
|
|
|
def test_plan_for_returns_none_when_strategy_is_concise():
|
|
job = {
|
|
"timing_strategy": "concise",
|
|
"video_stretch_plans": {"bn": {"plan": [{"orig_start": 0.0, "orig_end": 1.0,
|
|
"new_start": 0.0, "new_end": 1.0,
|
|
"stretch_ratio": 1.0}]}},
|
|
}
|
|
assert _video_stretch_plan_for(job, "bn") is None
|
|
|
|
|
|
def test_plan_for_returns_none_when_plan_missing_for_lang():
|
|
job = {
|
|
"timing_strategy": "stretch_video",
|
|
"video_stretch_plans": {"de": {"plan": [{"orig_start": 0.0, "orig_end": 1.0,
|
|
"new_start": 0.0, "new_end": 1.0,
|
|
"stretch_ratio": 1.0}]}},
|
|
}
|
|
assert _video_stretch_plan_for(job, "bn") is None
|
|
|
|
|
|
def test_plan_for_returns_entry_when_strategy_matches_and_plan_exists():
|
|
entry = {
|
|
"plan": [{"orig_start": 0.0, "orig_end": 1.0,
|
|
"new_start": 0.0, "new_end": 1.2,
|
|
"stretch_ratio": 1.2}],
|
|
"total_duration": 1.2,
|
|
"orig_duration": 1.0,
|
|
}
|
|
job = {"timing_strategy": "stretch_video", "video_stretch_plans": {"bn": entry}}
|
|
got = _video_stretch_plan_for(job, "bn")
|
|
assert got is entry
|
|
|
|
|
|
def test_plan_for_returns_none_when_video_stretch_plans_absent():
|
|
"""A job that ran strict_slot then was upgraded won't carry the plans
|
|
dict; the helper must tolerate that and return None."""
|
|
job = {"timing_strategy": "stretch_video"}
|
|
assert _video_stretch_plan_for(job, "bn") is None
|