Files
VoiceStudio/backend/schemas/requests.py
T
Palash DebnathandClaude Opus 4.8 8b00dc1f4f feat: onboarding demos, opt-in bug reporting, error-docs deeplinks + issue triage (#133)
* feat: onboarding demos, opt-in bug reporting, error-docs deeplinks + issue triage

Working-tree snapshot bundling several in-flight workstreams (v0.3.0):

- Onboarding/demo system: DemoPresetGrid, DictationDemo, DubbingDemo components
  + tests, render scripts (render_demos_omnivoice.py, build_demos.sh,
  build_dub_demo.sh), personalities preview URLs, alembic 0002 voice-profile
  demo fields.
- Opt-in bug reporting: ReportBugButton (prefilled GitHub-issue URL path).
- Error transparency UX: errorDocsMap deeplinks + BootstrapSplash/error wiring.
- Dub workspace: DubSegmentRow/Table, WaveformTimeline, dubSlice tweaks.
- Issue triage: .planning/issue-clusters/ (plan-01..05 root-cause masters,
  GH #128-#132).
- CLAUDE.md: hard rule — everything ships on v0.3.0, no version bumps.

KNOWN GAP (why this is a draft): the generated demo audio assets are NOT in
this tree, and backend/assets/samples/demo_voice.wav is deleted. onboarding.py
guards the missing file (skips seeding the demo profile with a warning), so no
crash — but first-run Launchpad will be empty and /demo_audio/ preview URLs
404 until assets are regenerated via scripts/build_demos.sh. Do not merge
before regenerating + committing the demo assets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dub): timing strategies — kill audio compression, add Concise + Stretch Video

Replaces the current audio time-compression default (atempo squeeze to fit
slot) that produced chipmunk/alien output on high-density target languages
like Bengali. Two new user-selectable modes; legacy behaviour kept behind
an explicit "Strict slot" choice.

New `DubRequest.timing_strategy` enum (default "concise"):
  - "concise"        Translator trims text to fit at natural rate; if it
                     still overflows, hard-trim at slot with a fade so we
                     never overlap the next speaker. Surface overflow_s
                     per segment so the user can shorten the text.
  - "stretch_video"  Audio plays at natural 1.0× rate. Backend computes a
                     per-segment new timeline; persists a video_stretch_plan
                     on the job. Mux step (dub_export) builds an ffmpeg
                     trim+setpts+concat filter graph that stretches each
                     segment's video portion to match the natural-rate dub
                     audio. Gaps/pre-roll/tail pass through at 1.0×.
                     Sub burn under stretch_video is skipped in one pass
                     (cues would drift).
  - "strict_slot"    Legacy atempo squeeze. Retained for back-compat.

Director rate-bias side-effect (seg_speed *= bias) now gated on strict_slot
only, so "urgent"/"slow" direction tokens keep their instruct effect in
the new modes without chipmunking.

Per-segment fit_status emitted in the SSE done event:
  {status: "fits" | "overflows" | "video_stretched", overflow_s?, stretch_ratio?}
DubSegmentRow's "Sync: 100%" badge (which was lying — sync_ratio was always
~1.0 because the TTS loop pre-trimmed to slot) is replaced with a truthful
"Fits / Overflows +Ns / Video 1.18×" label.

Frontend:
  - prefsSlice.timingStrategy (persisted, store v3→v4 with safe migrate).
  - DubTab footer Segmented control: "Concise · Stretch Video · Strict slot".
  - useDubWorkflow passes timing_strategy on /dub/generate; consumes fit_status.

Tests: tests/test_dub_timing_strategy.py — 13 cases covering schema
defaults/validation, _build_video_stretch_filter_graph (pre-roll, gap,
tail, empty-plan early return, post-subtitle chain-in), and
_video_stretch_plan_for guards. 30/30 existing dub tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(waveform): surface missing source as "Source media missing" instead of code-4 black box

When a project's underlying media file is gone (moved or deleted between
save and reload) the <video> element fires MediaError code 4 and the
companion audio fetch returns HTTP 404 — both were silently warned to
the console while the user stared at an unresponsive black panel and an
empty waveform.

- WaveformTimeline now flips loadError when the video element rejects
  code 3 (decode) or 4 (src not supported), and tracks `sourceMissing`
  separately so the error UI can name the actual problem.
- The audio decode fallback chain catches HTTP 404 specifically and
  treats it as source-missing instead of loading silent empty peaks —
  an empty waveform on a deleted source is more confusing than a clear
  "Re-upload the video to continue" message.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(tray): "Show OmniVoice" reloads when the webview is blank

When the dev Vite server restarts (or the main window is created before
the backend is ready), the webview load fails and the window is left
with `<body></body>` plus a "Could not connect to the server" console
error. Clicking "Show OmniVoice" from the tray menu just re-showed the
broken window — there was no recovery path short of quit+relaunch.

Now the show handler runs a tiny eval after `show()`/`set_focus()` that
calls `location.reload()` only when `document.body.childElementCount === 0`.
A healthy window doesn't blink (body is non-empty); a blank one
self-recovers as soon as the user clicks Show.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(#133): bug-report diagnostics field mapping + drop unused imports

Address PR #133 review:
- ReportBugButton: /system/info exposes `platform` + `device`, not
  `os`/`torch_device`/`gpu` — those reads silently dropped OS/GPU from every
  bug report. Map to the real fields (CodeRabbit). Also remove the dead
  `home` local in stripHome (CodeQL unused-variable).
- DictationDemo: drop unused `Loader` import (CodeQL unused-import).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 17:24:25 +05:30

117 lines
5.1 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
from pydantic import BaseModel, field_validator
from typing import List, Literal, Optional
from services.audio_dsp import EFFECT_PRESETS
class ExportRequest(BaseModel):
source_filename: str
destination_path: str
mode: str = "history"
class ExportRecordRequest(BaseModel):
filename: str
destination_path: str = "~/Downloads"
mode: str = "file"
class RevealRequest(BaseModel):
path: str
class DubSegment(BaseModel):
start: float
end: float
text: str
instruct: str = "" # Per-segment voice override
profile_id: str = "" # Per-segment voice profile
speed: Optional[float] = None
gain: Optional[float] = None # Per-segment volume (0.0 - 2.0, default 1.0)
target_lang: Optional[str] = None # Per-segment language override (ISO code)
effect_preset: str = "broadcast" # NEW: DSP preset id (default: broadcast)
@field_validator("effect_preset")
@classmethod
def validate_effect_preset(cls, v: str) -> str:
if v not in EFFECT_PRESETS:
raise ValueError(
f"Unknown effect preset: {v!r}. "
f"Valid: {list(EFFECT_PRESETS.keys())}"
)
return v
class DubRequest(BaseModel):
segments: List[DubSegment]
language: str = "Auto"
language_code: str = "und" # ISO 639-1 for ffmpeg metadata (e.g. "es", "fr", "de")
instruct: str = ""
num_step: int = 16
guidance_scale: float = 2.0
speed: float = 1.0
# Phase 4.1 — partial regen. Parallel lists by index with `segments`.
# When `regen_only` is set, only listed segment ids re-run TTS; others
# reuse their on-disk seg_N.wav. `segment_ids` lets the client bind
# each segment to a stable id across regen cycles.
segment_ids: Optional[List[str]] = None
regen_only: Optional[List[str]] = None
# Fast-preview mode for interactive edits. When true, TTS runs at
# num_step=8 (~2× faster, ~10-20% quality drop). Client is responsible
# for re-rendering preview segs at full quality before final export.
preview: Optional[bool] = False
# How to handle segs whose TTS audio is longer than its slot (the
# "ghost lang" overlap bug otherwise). Options:
# "time_stretch" — phase-vocoder stretch to fit, preserves pitch (default).
# "trim" — hard-clip to slot length + fade out (cheap, may cut mid-word).
# "off" — no fit; mix layers with += (legacy behaviour, may overlap).
# Legacy knob. When `timing_strategy` is set, it takes precedence and this is ignored.
slot_fit: Optional[str] = "time_stretch"
# High-level timing strategy. Replaces the audio-compression default that
# produced chipmunk/alien artefacts on high-density target languages
# (Bengali, Hindi, Arabic…). Three modes:
# "concise" — never compress TTS audio. Trim text up-front via
# speech_rate so it fits naturally; if it still
# overflows, hard-trim at slot with a short fade and
# surface fit_status="overflows" so the UI can prompt
# the user to shorten the segment. DEFAULT.
# "stretch_video" — never compress TTS audio. Re-lay the timeline so
# each segment's video portion is stretched (via
# ffmpeg setpts) to fit the natural-rate dub audio.
# Audio plays at 1.0×; total video duration grows.
# "strict_slot" — legacy: keep `slot_fit` semantics (atempo squeeze
# when audio > slot). Kept for back-compat.
timing_strategy: Optional[Literal["concise", "stretch_video", "strict_slot"]] = "concise"
# Per-job slip budget for "concise" mode. Hard-trim only kicks in once
# gap absorption + this much extra time has been consumed.
overflow_budget_s: Optional[float] = 0.0
class TranslateSegment(BaseModel):
id: str
text: str
target_lang: Optional[str] = None
class TranslateRequest(BaseModel):
segments: List[TranslateSegment]
target_lang: str # ISO 639-1 code like "es", "fr"
provider: Optional[str] = None
source_lang: Optional[str] = None # ISO 639-1; overrides job detection
job_id: Optional[str] = None # Dub job id, used to resolve detected source_lang
quality: Optional[str] = "fast" # "fast" (one-shot) | "cinematic" (reflect → adapt)
glossary: Optional[List[dict]] = None # [{"source": "...", "target": "...", "note": "..."}]
class DubIngestUrlRequest(BaseModel):
url: str
job_id: Optional[str] = None
# When true and the URL is a caption-bearing host (YouTube, Vimeo, TED…),
# ask yt-dlp to also download the original-language + any additional
# sub_langs as VTT. The UI uses this to seed a transcript without running
# Whisper, and optionally to skip the Translate step for languages that
# YouTube auto-translates for us.
fetch_subs: Optional[bool] = False
sub_langs: Optional[List[str]] = None
class ProjectSaveRequest(BaseModel):
name: str
video_path: Optional[str] = None
audio_path: Optional[str] = None
duration: Optional[float] = None
state: dict # Full JSON blob: segments, settings, tracks, etc.