Compare commits

...
26 Commits
Author SHA1 Message Date
453db55f12 release: freeze v0.3.11 — version bump, lockfiles, changelog (#970)
package.json + three mirrors -> 0.3.11 in lockstep; Cargo.lock/uv.lock/
bun.lock regenerated; CHANGELOG [Unreleased] -> [0.3.11] — 2026-07-05
with the multi-language-release headline; nine entries since v0.3.10.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 22:28:43 +05:30
620321a9cc feat(diagnostics): backend crashes become self-documenting — exit code + stderr tail surfaced and attached to bug reports (#969)
* feat(diagnostics): backend crashes become self-documenting — exit code + stderr tail surfaced and attached to bug reports

When the backend PROCESS died (native CUDA abort, OOM kill, DLL crash) the
user saw only "Can't reach the local OmniVoice backend" and the evidence
died with the process — every #941-class report needed a logs-please
round-trip nobody answers. The v0.3.9 guard fixed HANGS; this fixes the
class of invisible DEATHS:

- Rust (crash.rs): every unexpected child exit — detected by the startup
  health poll and the post-Ready supervisor — writes a rotating (last 3)
  JSON crash marker next to the backend logs: ts, exit code/signal,
  backend version, uptime, ~40-line stderr tail. Intentional shutdowns
  never forensicate: app-quit raises the quitting flag first (now also on
  macOS Cmd+Q via ExitRequested), and retry/clean-retry kills set a
  BACKEND_KILL_INTENDED flag cleared when the fresh child is tracked.
- Tauri commands get_last_backend_crash / acknowledge_backend_crash;
  ack is a persisted watermark, never a delete — bug reports still get
  the evidence after the user viewed it.
- Crash-loop escalation: the supervisor budget goes 5-in-60s → 3-in-10min
  so slow crash loops stop respawning and land on the Failed screen with
  the last exit code + stderr tail.
- Frontend: apiFetch's transport-failure path swaps the vague message for
  "the backend crashed (exit code X) N s ago…" when an unacknowledged
  marker exists, and BackendCrashNotice (banner + details dialog,
  i18n'd, ack-on-view) surfaces it even with no request in flight.
- Bug-report prefill gains a "Last backend crash" section (exit code +
  home-path-scrubbed stderr tail via the existing scrubText), so the next
  report arrives WITH the evidence.

Tests: cargo --lib 57 pass (marker rotation write-4-keep-3, ack
semantics, store IO, ExitStatus decomposition, 3-in-10min policy);
vitest 909 pass incl. crash-notice branch, client crash-message branch,
bug-report enrichment; legacy node:test 41 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add backend crash forensics under [Unreleased] (#969)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:36:47 +05:30
738d45f1c7 fix(ui): timeline box colors pre-blended in JS — visible on any WebView2 (#963) (#968)
* fix(ui): timeline box colors pre-blended in JS — visible on any WebView2, color-mix dependency removed (#963)

#951 moved the segment-box palette to `color-mix(in srgb, tint 45%,
var(--chrome-bg))` strings applied as inline styles. WebView2/Chromium
< 111 has no color-mix, so the CSSOM rejects the whole `background`
assignment — and since .seg-track__box declares no background of its
own, the boxes rendered fully transparent on pinned/enterprise WebView2
runtimes (the Windows installer never enforces a minimum runtime).

Fix the class, not the instance: no engine-dependent CSS may reach this
lane's inline styles. The 0.45·tint + 0.55·bg blend now happens in JS —
timeline.js keeps the tints as numeric [r,g,b], reads --chrome-bg off
the document root (fallback #0f1011), and emits literal `rgb(r, g, b)`
strings every engine parses. Pixel-identical to what color-mix painted.
Theme-awareness is preserved by re-blending when [data-theme] changes
on <html> (the seam App.jsx switches themes through), observed via
MutationObserver; SegmentTrack subscribes with useSyncExternalStore so
mounted boxes recolor live.

Guards updated: palette entries must match plain opaque rgb() (no
color-mix/var()/alpha), the default-theme blend is asserted against
independently computed literals, theme-change re-blend and rgb()/
garbage --chrome-bg parsing are covered, and SegmentTrack's rendered
inline background is asserted to be a literal rgb() — fails on any
reintroduction of engine-dependent CSS in this lane.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add WebView2 box-color fix under [Unreleased] (#968)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:28:19 +05:30
d959aae41b fix(dub): dialogue starts stop snapping to footsteps — sustained-energy onsets, bounded snap (#963) (#967)
* fix(dub): dialogue starts stop snapping to footsteps — sustained-energy onsets, bounded snap distance (#963)

Field report #963 (point 3): dubbed speakers start seconds early or late.
The reporter's own theory was right on the money — 'when a noise is heard
(a sigh or footsteps), it's interpreted as the start of the conversation.'
The #280 onset snapper took the FIRST 20 ms frame above an adaptive RMS
threshold as the speech onset, so any transient qualified; it also had no
snap-distance bound (a wrong onset could move a start by the whole segment
minus 0.3 s) and ran even when Demucs had failed and the 'vocals' track was
really the raw mix, where every ambient sound is a candidate.

Three layered guards, all pure NumPy (no new deps):

- Sustained energy: an onset only counts when >=160 ms of the following
  300 ms stays above the threshold. Footsteps/door thuds light up one or
  two frames and die; syllables keep the energy up.
- Bounded snap distance: shifts beyond 1.5 s are only trusted when the
  skipped span is (near-)silent — that is exactly the genuine #280
  whisper start-stretch on the vocals track (Demucs removed the music,
  leaving real silence), so long trims over silence still work in full.
  Long jumps over audible content (e.g. quiet speech under the relative
  threshold) are refused instead of playing the dub seconds late; an
  isolated transient in the span (<10% audible frames) doesn't block it.
- Source-aware: snapping now runs only on the separated vocals track.
  dub_core detects the Demucs fallback (vocals_path == audio_path, see
  dub_pipeline) at both call sites and passes separated_vocals=False on
  mixed audio, disabling snapping — whisper's own timestamps beat a
  confidently wrong snap when music/ambience is sustained energy too.

Tests (tests/test_onset_align.py, fail-before/pass-after): transient burst
rejected at detect- and snap-level, transient-only window yields no onset,
long jump over audible content refused, bounded shift over audible lead
still allowed, >1.5 s trim over true silence still snaps (#280 regression
guard), mixed-audio mode is a no-op. 28 pass in the file; full dub-adjacent
suites green.

Credit: theory and repro description by the #963 reporter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add onset-snap robustness under [Unreleased] (#967)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:15:14 +05:30
de99bc3bd5 fix(net): SOCKS-proxy users can synthesize again — ship socksio, cache-first model resolution (#959) (#966)
* fix(net): SOCKS-proxy users can synthesize again — ship socksio, cache-first model resolution, degrade LLM clients (#959)

Under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio installed, httpx
raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS proxy, but the
'socksio' package is not installed"). huggingface_hub's get_session()
builds exactly that client inside snapshot_download, so POST /generate
500'd with the bare message even for a fully installed model, and
preload_model's model_info probe hit the same error and silently
skipped warm-up. Latent since v0.3.5 — #947's fresh-process engine
spawning unmasked it in v0.3.10 by handing the user's proxy env
directly to a clean backend process.

Three layers, so the class (any session-construction failure) is dead,
not just the reported instance:

* Ship SOCKS support: socksio>=1.0 in [project] dependencies (pure
  Python, MIT, zero transitive deps) AND in backend.spec hiddenimports
  — httpx imports it lazily in try/except, so PyInstaller's tracer
  misses it and the frozen installers would stay broken without the
  explicit entry. uv.lock regenerated; `uv lock --check` and
  `uv sync --frozen` (the Docker/release bootstrap semantics) verified.

* Cache-first model resolution: from_pretrained's snapshot resolution
  extracted into _resolve_snapshot_dir() — local dir, else
  snapshot_download(local_files_only=True) (a complete cache resolves
  with NO HTTP session constructed), else the original network path.
  preload_model's failed network probe now falls back to a cache-only
  check and warms up anyway instead of silently skipping (honest log
  either way).

* Class guards: resolve_skill_client wraps OpenAI() construction —
  env-shaped construction failures degrade to the existing "LLM
  unavailable" contract instead of 500ing the calling feature; and
  core.failure learns SOCKS_PROXY_SUPPORT_MISSING with an actionable
  hint, appended on the raw-string surfaces (global 500 handler,
  model-install SSE) via the new append_hint().

Fail-before/pass-after verified by reverting the fix: 11 of the 12 new
tests fail pre-fix (the remaining one is the unchanged network-fallback
contract). 165 tests green across the touched suites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add SOCKS-proxy resilience under [Unreleased] (#966)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 20:00:08 +05:30
c47633a409 fix(settings): a saved LLM provider survives restart — explicit save activates, stale TRANSLATE_* prefs stop hijacking (#963) (#965)
* fix(settings): a saved LLM provider survives restart — explicit save activates, stale TRANSLATE_* prefs stop hijacking (#963)

"Ollama works until I restart OmniVoice" had three stacked causes:

1. Only "Save & use for translation" persisted the selection. Plain
   "Save" and "Test" sent make_active:false, and on restart
   active_provider_id() deliberately excludes local providers
   (Ollama/LM Studio) from auto-select — so a saved-and-tested Ollama
   was never resolved active again. The PUT handler now also claims the
   active slot on an explicit save when the user has never chosen a
   provider (new llm_providers.stored_active_provider_id(): the stored
   row only — no env pin, no legacy fallback, no auto-detect). An
   explicit prior choice is never stolen; an unconfigured provider
   can't claim the slot; make_active:true still flips.

2. Users of the retired (≤v0.3.7) Translation-LLM panel had
   env.TRANSLATE_* rows in prefs.json, re-imported into os.environ
   every launch — and a live TRANSLATE_BASE_URL resolves the active
   provider to "custom" ahead of auto-select on every restart. New
   startup migration (llm_providers.migrate_legacy_translate_prefs,
   run in main.py BEFORE the prefs→env import) moves those values into
   the custom provider's own settings-store rows (only where the store
   has no value yet) and deletes the prefs rows. Real process env vars
   are never touched; a failed store write keeps the prefs row and
   retries next boot. The legacy endpoint keeps working — via the
   store, without hijacking the active slot.

3. The panel read as "done" after a green Test even when another
   provider stayed active. It now shows a notice after save/Test when
   the edited provider is not the effective active one (suppressed
   while LLM_DEFAULT_PROVIDER pins the choice — the env banner already
   covers that).

Tests (fail-before): 7 new backend tests fail on the old code
(save-activates, never-steals, migration semantics, env untouched,
end-to-end ollama-beats-legacy-env) and the new panel test fails
without the notice; all pass after. Full LLM/settings suites, frontend
vitest (890), typecheck:ci, oxlint, oxfmt and vite build are green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add LLM-provider persistence fix under [Unreleased] (#965)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 19:47:54 +05:30
4aa9abe22a docs+scripts: install fixes — desktop-prod tauri resolution, Ubuntu white-screen guidance, honest GPU/prereq docs (#960 #961 #962) (#964)
* docs+scripts: install fixes — desktop-prod tauri resolution, Ubuntu white-screen guidance, honest GPU/prereq docs (#960 #961 #962)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the install-fixes batch under [Unreleased] (#964)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 19:34:01 +05:30
d90cfde1bb feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1) (#958)
* feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1)

Multi-language dubbing translated per language (#957) but stored everything
in single-slot state, so tracks silently destroyed each other's work:

P1.2 — per-language translation storage (additive):
- Frontend keeps every translation in s.translations[langCode] alongside the
  legacy s.text slot (still = the shown language). Translate All writes both;
  the new store action switchDubLangCode swaps text through the map on a
  user-driven language switch (non-destructive; restore paths keep the plain
  setter); manual edits / restore-original update the current language's
  entry; merge joins per-language texts, split drops them. Rides project
  save/load inside dubSegments — legacy projects behave exactly as before.
- Backend mirrors it as job["segments_i18n"] = {lang: {segKey: text}}
  (segKey = stable id, index for id-less legacy rows), written by
  _sync_job_segments; job["segments"] stays byte-identical for every existing
  consumer. /dub/srt|vtt?lang= and subtitle burn-in now emit THAT language's
  text when present — ExportModal's "all dubs" batch stops producing N
  identical files. Legacy jobs without the field fall back to today's output.

P1.3 — per-track WAV cache + fingerprints:
- Per-segment WAVs are language-keyed (seg_{lang}_{id}.wav). The partial-regen
  read path falls back to legacy seg_{id}.wav ONLY while the job has no
  other-language track — single-language jobs keep their whole on-disk cache;
  multi-track jobs stop splicing the last-generated language into the current
  track. Read-only endpoints (segment preview, clips zip) gained ?lang= with
  the permissive legacy fallback they always had.
- Fingerprints include the track language (segment_fingerprint(track_lang=…),
  /tools/incremental lang=…) and live in job["seg_hashes_by_lang"]; the flat
  job["seg_hashes"] stays as the current track's mirror so the done event,
  history restore and older frontends read it unchanged. A legacy flat map is
  attributed to the job's last-generated language (dropped when unknown) and
  reads stale once — the safe direction. seg_wav_kind is per-track too.
- The frontend stores fingerprints per language and judges "Regen N changed"
  against the ACTIVE track; project save/load and dub-history restore carry
  all tracks' hashes (segHashesByLang / seg_hashes_by_lang, additive).

Tests: fail-before regression coverage — two-track regen never splices the
other language's audio (sample-level assert on the mixed track), legacy
single-track cache reuse + multi-track gate, per-lang seg_hashes with flat
mirror + migration semantics, /dub/srt|vtt?lang= emitting different text per
track with legacy fallbacks, per-lang burn-in, /tools/incremental lang
scoping, and 14 frontend tests for translations round-trips, per-track
fingerprints and legacy-project behaviour. Full backend + frontend suites,
typecheck, lint and format:check green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add per-language storage + per-track caches under [Unreleased] (#958)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 03:05:40 +05:30
bd92729623 feat(dub): multi-language generate translates each language first + picks persist (P1) (#957)
* feat(dub): multi-language generate translates each language first + picks persist with the project

P1.1 — the "Generate N dubs" loop never translated: the backend synthesizes
segment text verbatim, so every multi-language track rendered the same words
and at most one was actually in its language. The loop now runs
translate → generate per pick:

- handleTranslateAll(langOverride?) accepts an optional ISO-code override
  (no-arg Translate All behavior unchanged; a click-event first arg is
  guarded). It resolves true only when a translation actually landed, and
  both it and handleDubGenerate snapshot segments from the store at call
  time — the click-time closures went stale the moment the previous pick's
  translate pass rewrote the segments.
- A pick whose translate fails (request error or all-segments-errored) is
  SKIPPED — never a wrong-language track — the batch continues, and the
  skipped languages are reported in a final toast.
- The redundant first translate is skipped only when pick 1 targets the
  language the editor text is already translated into; every later pick
  always translates.
- Honest progress: the pill shows "Translating → {lang} (i/N)…" before each
  generate, and the header CTA is inert while translating so a re-click
  can't start a second batch (belt: a ref guard in the loop).

P1.4 — multiLangMode/multiLangs move from DubTab component state into the
dub store slice and ride the project save/load payload (exportTracks too).
Additive and back-compat: legacy payloads default to off/empty and leave the
in-session exportTracks untouched (utils/projectState.js).

Tests (fail-before verified: 9 failures on the pre-fix code):
- handleTranslateAll override targets + return semantics + call-time
  segment snapshot (dubTranslateAllOverride.test.jsx)
- per-language translate-before-generate call order, skip-on-failure with
  continuation + skip-report toast, first-pick skip heuristic, unchanged
  single-language path (dubMultiLangGenerate.test.jsx)
- slice defaults/setters/reset, payload round-trip, legacy-payload defaults,
  App.jsx wiring guards (dubMultiLangPersist.test.js)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add multi-lang auto-translate under [Unreleased] (#957)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:49:41 +05:30
mergetestandClaude Fable 5 afe819f498 fix(ci): oxfmt the two #956 test files (unbreak main format check)
#956 merged with a red Tests gate — my merge script ran unconditionally
instead of aborting on the gate value; the failure was oxfmt-only on the
two new test files. Whitespace-only fix, tests re-verified green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:23:03 +05:30
6b91205036 fix(dub): completed tracks always show their tabs + history keeps its language (P0) (#956)
* fix(dub): completed tracks always show their tabs + history keeps its language (P0)

Root cause chain: the track switcher's visibility expression required
dubLangCode !== 'und' and ended in a tautology (dubTracks?.length > 0 ||
!!dubTracks), so it was effectively keyed to the language dropdown, not
the persisted tracks. History restore always handed the frontend 'und'
because the dub_history language/language_code COLUMNS froze at the
ingest-time "" — the save_job UPSERT never updated them after generation
set them on the job dict (only the job_data JSON carried the real value).
Net effect: a restored project with finished tracks showed no track tabs
until the user re-picked a language.

- DubTab: hasDubbedTrack = done && dubTracks.length > 0 (tracks only;
  also stops the tautology from showing a trackless switcher).
- DubTab auto-jump: membership-guarded — the preview only jumps to a
  language that has a track, else tracks[0]. Kills the preview-404 class
  (restores falling back to 'en' with tracks ['bn'] pointed the player
  at /dub/preview-video?lang=en).
- dub_pipeline.save_job UPSERT: language/language_code now update when
  non-empty (same CASE guard as content_hash), so new saves heal the
  frozen columns and empty re-saves can't clobber them back.
- App.restoreDubHistory: falls back to job_data's language/language_code
  so EXISTING rows in users' DBs restore correctly with no migration.
- P0.2 polish: track pills get duration + timing-strategy tooltips,
  hydrated lazily and failure-silently from the existing
  GET /dub/tracks/{job_id} via new api/dub.dubListTracks; all new
  strings through i18n (en.json).

Tests (fail-before/pass-after): DubTab-level visibility + auto-jump
membership-guard tests (3 of 4 fail pre-fix), pill-tooltip hydration
tests, and save_job language heal/no-clobber tests (heal fails pre-fix).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): open [Unreleased] with the dub track-tabs fix (#956)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:21:23 +05:30
1dfa032be4 feat(dub): project title first in the editor header, pipeline stages below (#955)
Header reordered per owner: row 1 = title (+ duration/segments) with the
action buttons, row 2 = the Upload→Export pipeline spine directly beneath
with a tight 2px gap (was: stepper and title side-by-side on one row).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:13:15 +05:30
fd7d20fe1e release: freeze v0.3.10 — version bump, lockfiles, changelog (#954)
package.json (source of truth) + the three mirrors -> 0.3.10, in lockstep;
Cargo.lock/uv.lock/bun.lock regenerated (one line each; bun --frozen-lockfile
verified). CHANGELOG [Unreleased] -> [0.3.10] — 2026-07-05 with the release
headline; nine fixes since v0.3.9, mostly same-day field-report turnarounds.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:57:17 +05:30
27ff5846c2 fix(audiobook): chapter render no longer crashes on mixed 1-D/2-D audio chunks (#897) (#953)
* fix(audiobook): chapter render no longer crashes on mixed 1-D/2-D audio chunks (#897)

Root cause: synthesize_chapter (backend/services/audiobook.py) built
inter-span pause silence as bare 1-D torch.zeros(n) while every real
engine's synth returns (1, samples) per the TTSBackend.generate contract
(OmniVoice's model.generate(...)[0] included) — so the chapter's final
hard concat in chunked_tts.concatenate_audio_chunks hit
torch.cat with mixed ranks and died with
'RuntimeError: Tensors must have same number of dimensions: got 1 and 2'.
Any chapter containing a [pause] span (Stories/audiobook longform)
crashed; existing tests missed it because their stub synth returned 1-D.
The crossfade branch had the same latent bug for mixed-rank chunks.

Fix, both layers:
- concatenate_audio_chunks now normalizes chunk shapes before any cat
  (_normalize_chunk_shapes): lower-rank chunks gain leading singleton
  dims to the highest rank present, then singleton channel dims
  broadcast to the widest channel count (mono follows stereo). Covers
  both the hard-cut and crossfade branches; homogeneous input passes
  through untouched, so all-1-D / all-2-D callers keep their exact
  output shapes. No future backend's output rank can re-break the join.
- synthesize_chapter materializes silence AFTER the loop, matching the
  rendered audio's channel dims / dtype / device — the same pattern
  generation.py's _render_with_pauses already uses for the single-shot
  path — so the data is rank-consistent at the source too. A
  silence-only chapter stays 1-D float32 as before.

Regression tests: mixed-rank hard-cut (both orders), mixed-rank
crossfade, mono->stereo broadcast, all-1-D/all-2-D shape stability, a
2-D-engine + [pause] chapter through synthesize_chapter (the exact #897
scenario), and a spy asserting the parts reaching the concat are
rank-homogeneous. All fail before the fix with the reported error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the audiobook pause-span concat fix under [Unreleased] (#953)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:40:35 +05:30
d14b37fab2 fix(dub): the speaker-count hint is honored on every diarization path + clone-purity guard (#952)
* fix(dub): the speaker-count hint is honored on every diarization path + clone-purity guard

The dub "Speakers" count reached _diarize() and then died on 3 of its 4
branches, so setting it changed nothing, speakers blended, and auto-clones
were cut from mixed-speaker audio ("made up" voices):

- FunASR inline-turns shortcut returned before the hint was ever consulted
  → now an explicit num_speakers routes the job through pyannote (the one
  engine that honors an exact count); turns stay the fast path only when no
  hint is set, and remain the fallback (with an honest "hint ignored"
  warning) when pyannote can't load or crashes mid-run.
- pyannote-unavailable fallback used a hardcoded 2-speaker silence-gap
  heuristic → assign_speakers_heuristic now takes num_speakers and cycles N
  labels on gap boundaries (1 → single speaker; None → legacy alternation),
  and the existing diarization warning says the hint is only approximately
  honored.
- pyannote-crash fallback dropped the hint the same way → same treatment.

No branch drops the hint silently anymore: every degraded path extends the
existing `warning` SSE payload (detail + a machine-readable speaker_hint
field) that the frontend already renders.

Parity + purity:
- POST /dub/transcribe/{job_id} (the CLI's endpoint) gains the same clamped
  num_speakers query param, forwarded to pyannote and the heuristic; the
  omnivoice-dub CLI gains --speakers N.
- Clone-purity guard: _pick_reference_slices rejects sub-1.5s slices, prefers
  slices not temporally adjacent (<0.3s) to another speaker's turn (scoring
  preference, not a hard filter), and extract_speaker_clones skips extraction
  entirely when labels came from the heuristic (labels_source kwarg threaded
  from _diarize; missing kwarg keeps the old behavior) — with a user-facing
  warning pointing at Settings → Models → pyannote.

Tests: fail-before/pass-after coverage in tests/test_speaker_hint.py (all
four _diarize branches driven through the real SSE stream), clone-purity
guards in tests/test_speaker_clone_purity.py, heuristic hint semantics in
tests/test_segmentation.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the speaker-hint + clone-purity fix under [Unreleased] (#952)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:53:47 +05:30
edf86c1800 fix(ui): dub timeline boxes can no longer flash invisible during playback (#373 class, completes #381) (#951)
* fix(ui): dub timeline boxes can no longer flash invisible during playback (#373 class, completes #381)

Root cause: the segment lane animated an inline
`transform: translateX(-scrollLeft)` on every playback tick, so Chromium
promoted it to a compositor layer — and on some Windows GPU/WebView2
driver combos, composited semi-transparent paints (the 0.45-alpha box
fills) flash invisible/visible while the layer moves, settling only when
paused. PR #381 removed `will-change` and raised the alpha, which only
dampened the symptom; the animated transform kept the lane composited.

Fix the class — no composited translucent paints on the lane, ever:

- Position boxes in pure layout: the lane transform is gone; each box's
  `left` is start·pxPerSec − scrollLeft (viewport coordinates). The
  virtualization window already derives from the same scrollLeft, so
  windowing stays consistent by construction. The selfScroll WebKit
  fallback keeps lane coordinates (its viewport is a real scroll
  container), unchanged.
- Belt-and-braces: REGION_COLORS are now fully opaque — each entry
  pre-blends the old 45% tint against the surface behind the lane
  (`--chrome-bg`, the .studio-panel background) via color-mix, which is
  pixel-identical to the previous alpha compositing (0.45·tint + 0.55·bg)
  in every theme, with zero alpha.

Regression tests (fail on pre-fix code): lane carries no transform at
rest and after a scroll update, box lefts are viewport-relative for a
scrolled view, no double-shift in the selfScroll fallback, and every
REGION_COLORS entry is alpha-free with the 45% ratio preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the timeline-box compositor fix under [Unreleased] (#951)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:41:16 +05:30
437a995a0f fix(translate): Cinematic/Autofit can no longer invent dialogue — divergence guard + pinned temperature (#950)
Root cause (v0.3.9 field report): the refine paths had no-op output guards.
Cinematic's ADAPT step only checked _looks_like_target_script, which returns
True unconditionally for every Latin-script target (no _SCRIPT_RANGES entry)
— so any non-empty LLM reply (hallucinated dialogue, refusals, commentary,
or the REFLECT critique itself) shipped as the dub line. Autofit's
adjust_for_slot accepted ANY non-empty reply, and its best-candidate tracker
(closest rate_ratio to 1.0) actively selected the most-padded output, while
_EXPAND_PROMPT invited invention with no ceiling. Both call paths also ran
at the provider-default temperature 1.0, unlike the working Fast path which
pins 0.2.

The fix, class-level:
- Shared divergence guard translator.refine_output_ok (length window
  0.4–2.5x, env-tunable via OMNIVOICE_REFINE_RATIO_MIN/MAX, with an
  absolute cap for short references; target-script check; critique-echo
  detection). Rejected ADAPT output degrades to the literal with
  error="adapt-diverged" (wrong-script keeps its adapt-wrong-script:<lang>
  marker), riding the existing degradation machinery unchanged.
- Autofit validates every reply against the ORIGINAL input text (divergence
  compounds across attempts otherwise); rejected candidates are discarded
  (attempt burned, graceful degradation to the input preserved) with
  error="fit-diverged"; lines under 15% of their slot skip LLM expansion
  entirely (fit-skip-short) — they could only "fill" the slot with
  fabricated dialogue.
- temperature=0.2 pinned on the cinematic (_chat) and fit (llm.chat) calls;
  chat/chat_messages gained an optional temperature param that is only sent
  when set, so refinement/director/glossary callers keep provider defaults.
- Prompts hardened: ADAPT forbids introducing facts/names/dialogue not in
  the source line; EXPAND forbids inventing information and more than
  doubling the line.
- speech_rate strict-mode docstring made honest: strict changes only the
  upper tolerance bound; expansion still runs (now guard-bounded).

Fail-before/pass-after regression tests for the reported bugs (10x runaway
ADAPT on an es target, critique echo, hallucinated slot-fill expansion,
refusal replies, tiny-line expansion skip, pinned temperature) plus the
previously-untested wrong-script fallback and legit-output acceptance.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:28:37 +05:30
03eb8f7d04 docs(readme): community table lists the Discord server's real channels (#949)
The table described aspirational channels (#showcase/#help/#feature-requests/#dev)
that don't exist on the server; it now matches reality (#announcements,
#releases+#changelog, #issues and #ideas forums, #discuss-ideas, #general).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 21:39:12 +05:30
beca8cfea8 fix(launcher): replace a stale old-version backend instead of attaching to it (#947)
* fix(launcher): replace a stale old-version backend instead of attaching to it

After an update, an orphaned backend from the PREVIOUS version can survive
holding the port. It still answers /system/info, so both attach paths
(lib.rs launch bootstrap + bootstrap.rs retry) treated it as "already
serving OmniVoice — attaching" and the new UI silently ran OLD backend
code: every fix in the update appeared to change nothing. Reported on
Discord as "a bound port which blocked the newer versions"; the app
already knew how to kill_orphan_on_port on both Unix (lsof) and Windows
(netstat) — it just never applied it to a healthy-but-stale backend.

The attach decision now compares versions: running_backend_version()
reads app_version from /system/info (string-sniff, no new deps), and
same_app_version() compares BASE versions (pre-release -N suffix
stripped, so a preview build 0.3.10-4 still attaches to its 0.3.10
backend). Same version → attach exactly as before. Different or missing
version → the orphan is killed and the bundled backend spawns. Foreign
processes keep the existing port_in_use take-ownership path; the
post-spawn health polls are untouched (we spawned that backend
ourselves).

Rust unit tests cover the /system/info parse shape and the
match/preview/stale/unversioned decisions; 51 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the stale-backend port-reclaim fix under [Unreleased] (#947)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:40:42 +05:30
32103ad7b7 docs(readme): charm + organization overhaul (Opal-style) — collapsibles + OpenAI-compatible API section (#945)
* docs(readme): charm + organization overhaul (Opal-style)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): restore inventory-exact feature names (docs-drift guard)

The charm pass sentence-cased five bold leads in the collapsed feature
list; scripts/check-docs-drift.py greps for the inventory's exact
title-case names. Restored: Vocal Isolation, Speaker Diarization,
Batch Queue, AI Watermark, GPU Auto-Detect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): dubbing screenshot shows a real completed dub (37 segs, EN→BN)

Replaces the empty drop-zone shot with the populated editor — video +
waveform + cast, 37 Bengali segment rows, DUB COMPLETE banner — captured
live from the v0.3.9 app; caption updated to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:36:34 +05:30
46c4445258 fix(ui): relative timestamps no longer render as "20617d ago" (seconds-vs-ms class) (#946)
Backend rows store timestamps as Unix SECONDS (time.time() REAL columns:
generation_history, dub_history, exports, longform jobs, projects), while
frontend-local records carry milliseconds (Date.now() story projects) or ISO
strings (transcriptions). Projects/OmniDrive fed the seconds straight into a
millisecond-based diff (fmtTime), so every generation-history card rendered
as ~1970 ("20617d ago") — and, because the same raw value drove the recency
sort, history items also sank to the bottom of the drive.

Fix the class, not the label: a single shared, unit-tolerant normalizer
(frontend/src/utils/relativeTime.js) now backs every relative-time call site.

- toMillis(ts): numbers < 1e12 are seconds (×1000), >= 1e12 already ms; ISO
  and numeric strings parse; Date instances pass through; null/0/undefined/
  garbage -> null. Backend storage format is untouched (backward compat).
- timeAgo(ts): "—" for missing stamps (never an epoch age), "just now" for
  future stamps within 1 min of clock skew, s/m/h/d buckets, short absolute
  date beyond 7 days.
- absoluteTime(ts): unit-safe tooltip text, '' when missing (no more
  "Jan 1 1970" titles on null rows).

Converted call sites: pages/Projects.jsx (drop local fmtTime + per-source
*1000 juggling; normalize ts once so sort and label agree), components/
Sidebar.jsx + components/WorkspaceProjects.jsx (drop duplicated local
timeAgo copies and caller-side *1000), pages/BatchQueue.jsx (drop local
formatAge; missing created_at used to render an epoch date), pages/
Transcriptions.jsx + components/TranscriptionPicker.jsx (parse via
toMillis, keep their i18n labels; unparseable stamps no longer render
"Invalid Date").

Tests (fail-before/pass-after): utils/relativeTime.test.js covers seconds/
ms/ISO/numeric-string/Date inputs, null/0 -> "—", clock-skew "just now",
and the 1970 regression (a seconds stamp from today must not render as
thousands of days ago); test/ProjectsRelativeTime.test.jsx guards the
OmniDrive wiring end-to-end (seconds created_at renders "2h ago", null
renders "—", mixed-unit sort orders by real recency). Full frontend suite:
106 files / 843 tests green; oxlint, oxfmt, typecheck:ci, node:test green.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:03:41 +05:30
8bbab3fcc3 fix(translate): Dub LLM engine runs on the configured LLM provider (new dub_translation skill) (#944)
* fix(translate): the Dub LLM engine now runs on the configured LLM provider

Picking "LLM (OpenAI-compatible)" in the Dub tab read only the raw
TRANSLATE_* env vars — completely bypassing Settings → LLM Providers, so a
provider the user had configured AND tested in-app silently didn't power
the engine (empty key → raw 401 per segment). The Cinematic refiner was
already rewired through LLM Skills (#910/#912); this closes the gap for
direct LLM translation:

* new "dub_translation" LLM skill (Settings → LLM Skills) — per-skill
  provider override → global active provider, same resolution as every
  other skill; disabled == unconfigured, no new degradation modes
* the provider=openai branch resolves through resolve_skill_client();
  TRANSLATE_BASE_URL/TRANSLATE_API_KEY/TRANSLATE_MODEL stay working as
  the power-user override (env-only setups see zero behavior change,
  except the stale gpt-3.5-turbo default is now gpt-4o-mini, matching
  the cinematic path)
* per-segment calls are now bounded by the LLM timeout (45s default via
  OMNIVOICE_LLM_TIMEOUT) instead of the SDK's 600s default
* fully unconfigured → an up-front actionable 400 naming Settings → LLM
  Providers / LLM Skills instead of a per-segment 401
* provider-store keys are resolved into the error scrubber so a provider
  echoing the key can't leak it (parity with the env-key scrub)
* translation_engines registry: honest notes + a configured/configured_via
  stamp on LLM entries so the Engine dropdown can show ready-vs-needs-setup
  before the user clicks Translate

Tests: 4 new (skills-resolved client wins with its model+timeout; 400s
name the right settings page for no_provider vs disabled; env fallback
keeps working incl. TRANSLATE_MODEL); skills registry coverage updated;
existing openai-branch tests routed deterministically through the env
branch via the shared fake helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the dub-translation provider wiring under [Unreleased] (#944)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:22:12 +05:30
9ff5f38cb3 docs(readme): cross-promote the maker's other local-first projects (Opal, memxt) (#943)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:09:46 +05:30
359c70bc21 docs(readme): use the Launchpad screenshot as the hero image (#942)
Swap the static social-preview banner for the live v0.3.9 Launchpad shot
and drop the now-duplicate Launchpad row from the gallery (shown once).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 18:56:21 +05:30
0579ec91ad docs(readme): Opal-style restyle + fresh v0.3.9 screenshots (#937)
- Emoji section headers with explicit <a id> anchors. Emoji breaks
  GitHub's auto-generated heading slugs, so every in-page nav target keeps
  a stable explicit anchor (verified all href="#..." resolve).
- Refresh the screenshot gallery. The prior set was from April, predating
  the launchpad / settings / dictation UI overhaul, so it misrepresented
  the app. Captured fresh at retina from the live v0.3.9 UI and led the
  gallery with the new Launchpad home: launchpad, studio, voice design,
  voice gallery, dubbing, engine-compatibility matrix, model store,
  embedded API reference (Scalar), and the in-app changelog reader.
- Fix the stale engine count (11 -> 14 TTS engines) in the comparison
  table, FAQ, and roadmap to match the engine table + backend registry.
- Use <kbd> keycaps for the dictation shortcut (Opal detail).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 18:38:47 +05:30
b6ec4e23f3 fix(engines): snapshot lazy registry keys so /engines can't 500 under concurrency (#940)
* fix(engines): snapshot lazy registry keys so /engines can't 500 under concurrency

`list_backends()` runs in a FastAPI threadpool and iterates the lazy TTS/ASR
registries via `items()` → `__iter__`, which held a *live* `dict.__iter__(self)`
open across each engine's slow `is_available()` probe. Meanwhile the lazy
`__getitem__` resolves a deferred entry by mutating the dict (`self[key] = cls`).
A second concurrent `/engines` request (or any ASR op) materializing the lazy
`faster-whisper-isolated` entry therefore changed the dict size mid-iteration:

    RuntimeError: dictionary changed size during iteration
      asr_backend.py:1729 list_backends → _REGISTRY.items()
      asr_backend.py:1665 __iter__ → for k in dict.__iter__(self)

Both `_LazyRegistry` (TTS) and `_LazyASRRegistry` (ASR) now snapshot their live
keys up front with `list(dict.__iter__(self))` — consumed atomically under the
GIL — so a concurrent lazy insert can no longer trip the iteration. The slow
per-engine probes then run over the snapshot, not the live iterator.

Deterministic fail-before/pass-after regression for both registries:
tests/backend/services/test_lazy_registry_concurrency.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(changelog): add the /engines concurrency fix under [Unreleased] (#940)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 16:53:57 +05:30
118 changed files with 7129 additions and 822 deletions
+4 -1
View File
@@ -118,9 +118,12 @@ jobs:
working-directory: frontend
run: bun run format:check
# `bun run test` (frontend/package.json), not `bunx vitest` — bunx
# resolves by npm package name and can miss workspace-hoisted bins,
# then falls back to fetching from npm (#962 class).
- name: Run Vitest (frontend)
working-directory: frontend
run: bunx vitest run
run: bun run test
# Legacy node:test runner for tests/frontend/*.test.mjs
- name: Run frontend node:test (legacy)
+34
View File
@@ -6,6 +6,40 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
The bundled TTS model package (`pyproject.toml`) is versioned independently.
## [0.3.11] — 2026-07-05
The multi-language release — dubbing into several languages at once is finally a mature, honest workflow: **"Generate N dubs" now translates each language before rendering it** (with visible per-language progress), **switching languages never destroys your work** (every track keeps its own text, subtitles, and audio cache), completed tracks always show their tabs, and dialogue stops starting seconds early because of footsteps — a community reporter's theory, confirmed exactly. Around it, a reliability sweep driven by same-day field reports: your LLM provider finally survives a restart, SOCKS-proxy users can synthesize again (installed models now load without touching the network at all), timeline boxes are visible on every WebView2 runtime, running from source works again — and when the backend crashes, **it now tells you the exit code and attaches the evidence to your bug report automatically**.
### Added
- **Backend crashes are now self-documenting.** When the local backend process dies (a native GPU abort, an out-of-memory kill), the app used to show only "Can't reach the backend" — undiagnosable without logs nobody sends. The launcher now records every unexpected backend death (exit code, how long it ran, the last 40 log lines), tells you honestly that it *crashed* and is restarting, offers a "View crash details" panel, attaches the evidence to in-app bug reports automatically (paths scrubbed), and stops silent crash-loops after 3 deaths in 10 minutes with the details on screen. Intentional shutdowns, restarts, and app quits are never misreported as crashes. (#969)
- **"Generate N dubs" now actually translates each language first.** Multi-language generation used to synthesize every track from whatever text was in the editor — so at most one of your N dubs was really in its language. The batch now runs translate → generate per language with a visible "Translating → Bengali (2/3)…" phase, skips (and reports) any language whose translation fails instead of rendering a wrong-language track, and your multi-language picks and export-track selection are saved with the project instead of vanishing on tab switch. (#957)
- **Switching dub languages no longer destroys your work — every track keeps its own text and audio.** Translations are now stored per language (switching the target swaps the editor text non-destructively; manual edits stay with their language), subtitles export each track's own text instead of N identical files, burned-in subs match their track, and the per-segment audio cache is keyed by language — "Regen changed" can no longer splice another language's audio into the track you're rebuilding, and staleness is tracked per track. Fully backward-compatible: existing projects and caches keep working; a pre-upgrade project's first "Regen changed" simply regenerates cleanly once. (#958)
### Fixed
- **Timeline segment boxes are visible on every WebView2 runtime.** The v0.3.10 flicker fix switched box colors to a newer CSS feature (`color-mix`) applied as an inline style — on WebView2 runtimes older than ~March 2023 (pinned enterprise/offline installs) that renders as *fully transparent*, turning "flickering boxes" into "no boxes at all" while looking perfect on up-to-date machines. Colors are now pre-blended in plain JavaScript to universally-supported `rgb()` values — pixel-identical on modern runtimes, theme-aware, and guarded by a test that fails if an engine-dependent color ever reaches the timeline again. (#968)
- **Dubbed dialogue stops starting seconds early because of footsteps.** Dialogue starts are snapped to the first detected sound — and a single 20 ms burst (footsteps, a door, a sigh) counted as "speech", with no limit on how far a start could jump, and the snap even ran on the raw mix when vocal separation had failed. Onsets now require sustained speech-like energy, long jumps are only allowed across genuinely silent spans (so the original fix for whisper's stretched starts keeps working), and snapping turns off entirely when vocals weren't separated. Credit to the community reporter whose "footsteps theory" was exactly right. (#967)
- **Completed dub tracks always show their video tabs.** Opening a project with a finished dubbed track hid the Original/track switcher until you re-selected the language — visibility was keyed to the language dropdown instead of the project's tracks, and restored projects couldn't set the language because the history database froze it at empty forever. Tabs now render from the tracks themselves, history keeps its language (existing projects heal without migration), restoring a project can no longer 404 the video preview, and track pills gained duration/timing tooltips plus an accurate now-playing indicator. (#956)
- **Running from source works again, and the install docs stop lying.** `bun run desktop-prod` broke when the frontend became a workspace (`bunx` could fetch the wrong "tauri" package from npm — fixed everywhere including CI); the Linux white-screen guidance now leads with the variable that actually fixes modern Ubuntu (`WEBKIT_DISABLE_DMABUF_RENDERER=1`, with the exact `EGL_BAD_PARAMETER` error quoted); Windows docs now state plainly that GPU acceleration is NVIDIA-only there; the Linux docs document the ROCm support that already shipped (the "planned follow-up" note was stale); and prerequisites are split installer-vs-source with git and curl included. (#964)
- **Your LLM provider now survives a restart.** Setting up Ollama (or any provider), testing it, and saving looked like it worked — then a restart forgot the selection: only the separate "Save & use for translation" button ever persisted it, and a leftover setting from the retired (≤0.3.7) translation panel could silently steal the choice back to "Custom" on every launch. An explicit save now activates the provider when none was chosen yet, the leftover legacy settings are migrated into the Custom provider once and removed, and the panel says "Saved — not yet used for translation" instead of staying silent when your edit isn't the active provider. (#965)
- **SOCKS-proxy users can synthesize again — and an installed model can never again be blocked by a broken network stack.** With a system-wide SOCKS proxy set, clicking Synthesize 500'd with a raw "socksio not installed" error: loading an already-downloaded model still constructed a network session first, which failed at creation. The app now ships SOCKS support (including in the packaged installers), resolves installed models **cache-first** (no network session when the files are already on disk — the local-first guarantee at the loader level), warms up at startup even when the online check fails, degrades LLM extras instead of crashing on proxy errors, and classifies the error with an actionable hint if it ever does surface. (#966)
## [0.3.10] — 2026-07-05
The listening release — nine fixes in twenty-four hours, almost all driven by your v0.3.9 field reports (several with same-day turnaround). The dubbing pipeline stops lying: **Cinematic and Autofit can no longer invent dialogue**, the **speaker count you set is honored on every path** (and auto-cloning stops fabricating voices from guessed labels), and the timeline stops flashing invisible on Windows. Audiobook chapters with pauses render again. And one fix everyone should want: **updating can no longer leave you secretly running the old version** — a leftover backend from a previous install holding the port is now detected and replaced at launch. Plus: the Dub tab's LLM engine finally runs on the provider you configured in Settings, history timestamps stop reading "20617d ago", and the Engines page can't crash under concurrent load.
### Fixed
- **Audiobook/Stories chapters with a `[pause]` no longer fail to render.** Pause spans were built as 1-D silence while every TTS engine returns 2-D audio, so the chapter concatenation crashed with `Tensors must have same number of dimensions` — any chapter containing a pause failed on every attempt (reported with a precise trace in #897). Silence now matches the rendered audio's shape at the source, and the chunk concatenator defensively normalizes mixed ranks (including honest mono→stereo broadcast) so no engine can re-trigger the class. (#953)
- **Cinematic and Autofit dubbing can no longer invent dialogue.** The refine and slot-fit passes accepted any non-empty LLM reply for Latin-script languages — hallucinated lines, refusals, or the critique itself could ship as the dub. Every reply is now checked against the original line (length window, target script, critique echo — tunable via `OMNIVOICE_REFINE_RATIO_MIN/MAX`), rejected output falls back to the literal translation with an `adapt-diverged`/`fit-diverged` marker, lines too short to honestly fill their slot skip LLM expansion entirely, and both passes pin `temperature=0.2` like the Fast path. (#950)
- **Dub timeline boxes can no longer flash invisible during playback.** On some Windows GPU/WebView2 driver combos the segment boxes under the video vanished and reappeared while playing (first reported in #373; the earlier fix was incomplete) — the timeline lane still animated a CSS transform every playback tick, keeping the translucent boxes on a composited layer that the driver mis-painted. Boxes are now positioned in pure layout with fully opaque theme-aware fills (pixel-identical colors), removing the glitch class on every platform. (#951)
- **The dub "Speakers" count now actually does something — on every path.** The hint only reached pyannote; the common fallbacks silently ignored it (the no-diarization heuristic was hardcoded to alternate two speakers, and the FunASR shortcut never consulted it). The heuristic now cycles the requested count, an explicit count routes through pyannote when available, every path that can only approximate (or must ignore) the setting says so in a visible warning, and the legacy endpoint + a new CLI `--speakers` flag accept it too. Auto voice-cloning also stops fabricating voices from guessed labels: reference slices under 1.5s are rejected, slices bordering another speaker's turn are avoided, and cloning is skipped with an honest warning when speaker labels came from the gap heuristic instead of real diarization. (#952)
- **Settings → Engines can no longer 500 under concurrent loads.** The lazy TTS/ASR engine registries held a *live* dictionary iterator open across each engine's `is_available()` probe while `list_backends()` ran in a FastAPI threadpool — so a second concurrent `/engines` request materializing a lazy engine entry (`self[key] = cls`) mutated the dict mid-iteration and crashed the request with `RuntimeError: dictionary changed size during iteration`. Both registries now snapshot their keys before iterating (atomic under the GIL), immune to a concurrent insert; regression-tested for TTS and ASR. (#940)
- **The Dub tab's LLM translation engine now runs on your configured LLM provider.** Picking "LLM (OpenAI-compatible)" silently required three hand-set environment variables even when a provider was already configured and tested in Settings → LLM Providers; it now resolves through a new "Dub translation" LLM skill (route it to any provider — remote or local — in Settings → LLM Skills, independently of Cinematic refinement), keeps the `TRANSLATE_*` env vars as a power-user override, bounds every call with the LLM timeout instead of the SDK's 600-second default, tells the Engine dropdown whether the engine is actually ready (and via which provider), and — when nothing is configured — returns a clear pointer to Settings → LLM Providers instead of a raw 401 per segment. (#944)
- **Timestamps no longer show "20617d ago" in OmniDrive/Projects.** The backend stores record times in Unix seconds while some views assumed milliseconds, so generation-history cards rendered as ~1970 ("20617d ago") and sorted last; every relative-time label (OmniDrive, sidebar history, dub projects, batch queue, transcriptions) now goes through one unit-tolerant formatter, and records missing a timestamp show "—" instead of an epoch age.
- **Updating can no longer leave you secretly running the old version.** If a backend from a previous version was still holding the port (an orphan that survived an update), the new app "attached" to it because it answered health checks — so every fix in the update appeared to change nothing (the reported "bound port blocked the newer version"). The launcher now compares the running backend's version against the app before attaching: same version attaches as before, a stale one is killed and the bundled backend is started in its place — on macOS, Windows, and Linux. (#947)
## [0.3.9] — 2026-07-04
The dictation release — and a deep reliability pass driven by live-testing the entire app. **Dictation is rebuilt end-to-end**: instant feedback with a live waveform, words that commit about half a second after you stop speaking, clean punctuation, and text insertion that never lies about success. **LLM providers get one-click connection testing** with real diagnostics and model discovery, in all 21 languages. The app now **always opens maximized**, bottom buttons **can't hide under the footer** at small window sizes, and a wave of "out of memory / can't reach the backend / stuck at preparing" reports were traced to their real causes and fixed — including the silent VRAM crash on 8 GB cards, dead-IPC startup hangs after a Windows BSOD, and misleading error labels. Intel-Mac support status is now stated honestly, Confucius4-TTS is validated end-to-end, and Parakeet — roughly 20× faster than the default transcriber on CPU — is unlocked for every machine.
+1
View File
@@ -18,6 +18,7 @@ Thanks for your interest in improving OmniVoice Studio! This guide covers everyt
### Prerequisites
- [Git](https://git-scm.com/)
- `curl` (used by the Bun / uv / rustup install one-liners on macOS and Linux)
- [Bun](https://bun.sh/) (frontend package manager)
- [uv](https://docs.astral.sh/uv/) (Python environment manager)
- [ffmpeg](https://ffmpeg.org/) (audio/video processing)
+230 -149
View File
@@ -10,6 +10,7 @@
<a href="#why-ovs">Why OVS</a> ·
<a href="#tts-engines">TTS Engines</a> ·
<a href="#asr-engines">ASR Engines</a> ·
<a href="#openai-api">API</a> ·
<a href="#sponsors">Sponsors</a> ·
<a href="#sponsor--donate">Donate</a> ·
<a href="#contributing">Contributing</a> ·
@@ -31,11 +32,21 @@
<br/>
<div align="center">
<img src=".github/assets/social-preview.png" alt="OmniVoice Studio — The open-source ElevenLabs alternative" width="100%"/>
<img src="docs/screenshot-launchpad.png" alt="OmniVoice Studio — Launchpad" width="100%"/>
</div>
> **Your voice is the most personal data you have. So why rent it back from a cloud?** Every mainstream voice tool ships your audio to someone else's server and bills you monthly for the privilege. OmniVoice Studio flips that: clone, design, dub, and dictate on your own hardware — 646 languages, no meter running, nothing leaving your machine.
<div align="center">
| 🔑 No API keys | 🙅 No accounts | ☁️ No cloud | 💳 No subscription |
|:---:|:---:|:---:|:---:|
| nothing to paste in | nothing to sign up for | your audio stays home | it's your computer |
</div>
> [!WARNING]
> **OmniVoice Studio is in active beta.** Things may break between releases. For the latest features and fixes, clone the repo and run from source rather than using pre-built installers. Bug reports and PRs are very welcome [open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) or [join Discord](https://discord.gg/bzQavDfVV9).
> **OmniVoice Studio is in active beta.** Things may break between releases — for the latest features and fixes, clone the repo and run from source rather than the pre-built installers. Bug reports and PRs are very welcome: [open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) or [join Discord](https://discord.gg/bzQavDfVV9).
<div align="center">
<br/>
@@ -47,7 +58,68 @@
<br/>
## Features
<a id="screenshots"></a>
## 📸 See it in action
<table>
<tr>
<td align="center" width="50%">
<img src="docs/screenshot-studio.png" alt="Studio" width="100%"/>
<br/><b>Studio</b><br/>
<sub>Generate &amp; clone in one workspace — a 3-second clip mirrors any voice, 646 languages, zero-shot.</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="Voice Design" width="100%"/>
<br/><b>Voice Design</b><br/>
<sub>Build new voices from scratch — gender, age, accent, pitch, emotion, dialect.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="Voice Gallery" width="100%"/>
<br/><b>Voice Gallery</b><br/>
<sub>Browse ready-made archetype voices with language filters — or build your own library.</sub>
</td>
<td align="center">
<img src="docs/screenshot-dub.png" alt="Video Dubbing" width="100%"/>
<br/><b>Video Dubbing</b><br/>
<sub>A real dub, end to end: 37 segments transcribed, translated to Bengali, re-voiced, and timed — ready to export as MP4.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-engines.png" alt="Settings — Engines" width="100%"/>
<br/><b>Settings → Engines</b><br/>
<sub>The engine compatibility matrix — 14 TTS engines with per-engine GPU preflight, no silent CPU fallback.</sub>
</td>
<td align="center">
<img src="docs/screenshot-settings.png" alt="Settings — Models" width="100%"/>
<br/><b>Settings → Models</b><br/>
<sub>One-click model store — auto-detects your platform (CUDA / MPS / CPU) and recommends the right models.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-openapi.png" alt="Settings — API Reference" width="100%"/>
<br/><b>API Reference</b><br/>
<sub>The full local REST API, embedded — every endpoint documented with copy-paste client snippets.</sub>
</td>
<td align="center">
<img src="docs/screenshot-updates.png" alt="Settings — What's New" width="100%"/>
<br/><b>What's New</b><br/>
<sub>In-app changelog reader — see exactly what shipped in each release without leaving the app.</sub>
</td>
</tr>
</table>
---
<a id="features"></a>
## ✨ Features
The eight headliners — and twelve more waiting under the fold.
<table>
<tr>
@@ -75,76 +147,44 @@
</td>
<td align="center" valign="top">
<h3>⌨️ Dictation Widget</h3>
<p><code>⌘+⇧+Space</code> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
<p><kbd>⌘</kbd>+<kbd>⇧</kbd>+<kbd>Space</kbd> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
</td>
<td align="center" valign="top">
<h3>🔊 Vocal Isolation</h3>
<p>Demucs-powered. Splits speech<br/>from music, <b>keeps the background</b>.</p>
</td>
<td align="center" valign="top">
<h3>👥 Speaker Diarization</h3>
<p>Pyannote + WhisperX.<br/><b>Auto-identifies</b> who said what.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>📦 Batch Queue</h3>
<p>Drop <b>50 videos</b>, walk away.<br/>Progress bars per job.</p>
</td>
<td align="center" valign="top">
<h3>🤖 MCP Server</h3>
<p>Use OmniVoice from <b>Claude</b>,<br/>Cursor, or any MCP client.</p>
</td>
<td align="center" valign="top">
<h3>🛡️ AI Watermark</h3>
<p>AudioSeal (Meta). <b>Invisible</b>,<br/>survives compression.</p>
</td>
<td align="center" valign="top">
<h3>🔬 Diagnostics</h3>
<p>Self-check, error journal,<br/>scrubbed <b>diagnostic bundle</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🔐 100% Local</h3>
<p>No keys, no cloud, no accounts.<br/><b>Your machine only</b>.</p>
</td>
<td align="center" valign="top">
<h3>⚡ GPU Auto-Detect</h3>
<p>CUDA · MPS · ROCm · CPU.<br/>≤8 GB? <b>Auto-offloads</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧩 Extensible</h3>
<p>Subclass <code>TTSbackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧭 Engine Routing</h3>
<p>Preflight GPU check per engine.<br/><b>No silent CPU fallback</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🎒 Portable Personas</h3>
<p>Export voices as <code>.ovsvoice</code><br/>bundles — identity + <b>watermark</b>.</p>
</td>
<td align="center" valign="top">
<h3>♾️ Unlimited TTS</h3>
<p>Sentence-chunked generation.<br/><b>No length cap</b>. Streaming via WS.</p>
</td>
<td align="center" valign="top">
<h3>🌐 Remote Backend</h3>
<p>Point UI at a remote server.<br/>Tailscale-friendly. <b>Bearer auth</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧠 Dictation + LLM</h3>
<p>Local LLM cleanup of transcripts.<br/>Optional echo <b>cancellation</b>.</p>
<h3>🤖 MCP Server</h3>
<p>Use OmniVoice from <b>Claude</b>,<br/>Cursor, or any MCP client.</p>
</td>
</tr>
</table>
<details>
<summary><b>…and 12 more</b> — isolation, diarization, batch, watermarking, diagnostics, and friends</summary>
<br/>
- 🔊 **Vocal Isolation** — Demucs-powered: splits speech from music and keeps the background bed.
- 👥 **Speaker Diarization** — Pyannote + WhisperX auto-identify who said what.
- 📦 **Batch Queue** — drop 50 videos, walk away; per-job progress bars.
- 🛡️ **AI Watermark** — AudioSeal (Meta): invisible, survives compression.
- 🔬 **Diagnostics** — self-check suite, error journal, scrubbed diagnostic bundles.
-**GPU Auto-Detect** — CUDA · MPS · ROCm (Linux, opt-in) · CPU; ≤8 GB VRAM auto-offloads.
- 🧭 **Engine routing** — preflight GPU check per engine; no silent CPU fallback.
- 🧩 **Extensible** — subclass `TTSBackend`, add any engine in ~50 lines.
- 🎒 **Portable personas** — export voices as `.ovsvoice` bundles: identity + watermark.
- ♾️ **Unlimited TTS** — sentence-chunked generation, no length cap, streaming via WebSocket.
- 🌐 **Remote backend** — point the UI at a remote server; Tailscale-friendly, bearer auth.
- 🧠 **Dictation + LLM** — local-LLM cleanup of transcripts, optional echo cancellation.
</details>
---
## Quickstart
<a id="quickstart"></a>
## ⚡ Quickstart
<div align="center">
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
@@ -157,14 +197,19 @@
<sub><b>Intel Macs are not supported for the local backend:</b> the app UI installs, but the Python backend cannot run because PyTorch no longer ships Intel-Mac (x86_64) wheels (<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>) — see <a href="docs/install/macos.md">docs/install/macos.md</a>.</sub>
</div>
Per-OS install guides — pick yours and follow it end-to-end:
Pick your OS and follow the guide end-to-end:
- **macOS** — [docs/install/macos.md](docs/install/macos.md)
- **Windows** — [docs/install/windows.md](docs/install/windows.md)
- **Linux** — [docs/install/linux.md](docs/install/linux.md)
- **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
- 🍎 **macOS** — [docs/install/macos.md](docs/install/macos.md)
- 🪟 **Windows** — [docs/install/windows.md](docs/install/windows.md)
- 🐧 **Linux** — [docs/install/linux.md](docs/install/linux.md)
- 🐳 **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
Stuck? Run the built-in self-check first — **Settings → About → "Run
<details>
<summary><b>🧰 Stuck? Self-checks, tokens &amp; restricted networks</b></summary>
<br/>
Run the built-in self-check first — **Settings → About → "Run
self-check"** in the app, or `uv run python backend/main.py --diagnose` from
a checkout (`--deep` also test-loads the active engine). Then see
[docs/install/troubleshooting.md](docs/install/troubleshooting.md) for the
@@ -179,57 +224,13 @@ diarization-specific gating, see
speed, the ⚡ fast-download (Xet) status, and restricted-network / mirror
options, see [docs/downloading-models.md](docs/downloading-models.md).
## Screenshots
<table>
<tr>
<td align="center" width="50%">
<img src="docs/screenshot-clone.png" alt="Voice Clone" width="100%"/>
<br/><b>Voice Clone</b><br/>
<sub>Drop a 3-second clip → mirror any voice. 646 languages, zero-shot.</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="Voice Design" width="100%"/>
<br/><b>Voice Design</b><br/>
<sub>Build new voices from scratch — gender, age, accent, pitch, style.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-dub.png" alt="Video Dubbing" width="100%"/>
<br/><b>Video Dubbing</b><br/>
<sub>Upload or paste a YouTube URL. Transcribe, translate, re-voice, export.</sub>
</td>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="Voice Gallery" width="100%"/>
<br/><b>Voice Gallery</b><br/>
<sub>Search YouTube, browse categories, download clips, build your library.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-settings.png" alt="Settings — Models" width="100%"/>
<br/><b>Settings → Models</b><br/>
<sub>15 models. One-click install. Auto-detects your platform (CUDA / MPS / CPU).</sub>
</td>
<td align="center">
<img src="docs/screenshot-libraryprojects.png" alt="Projects" width="100%"/>
<br/><b>Projects</b><br/>
<sub>Dub projects, voice profiles, generation history, exports — all searchable.</sub>
</td>
</tr>
<tr>
<td align="center" colspan="2">
<img src="docs/screenshot-logs.png" alt="Settings — Logs" width="100%"/>
<br/><b>Settings → Logs</b><br/>
<sub>Live backend, frontend, and Tauri runtime logs. Filter, refresh, clear.</sub>
</td>
</tr>
</table>
</details>
---
## Why OVS?
<a id="why-ovs"></a>
## 💡 Why OmniVoice?
ElevenLabs charges **$5$330/mo** and processes your audio on their servers. OmniVoice Studio runs **on your hardware, with no usage limits.**
@@ -243,15 +244,15 @@ ElevenLabs charges **$5$330/mo** and processes your audio on their servers. O
| **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
| **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
| **API Keys** | Required | Not needed |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm · CPU |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm (Linux) · CPU |
| **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
| **TTS Engines** | 1 | **11** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3) |
| **TTS Engines** | 1 | **14** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS) |
| **ASR Engines** | 1 | **9** (WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR, isolated Faster-Whisper, sherpa-onnx live dictation) |
| **MCP Server** | ❌ | ✅ Use from Claude, Cursor, any MCP client |
| **Self-check** | ❌ | ✅ Diagnostics suite, error journal, scrubbed debug bundles |
| **Customizable** | ❌ Closed | ✅ Fork it, extend it, ship it |
OmniVoice Studio gives you professional-grade AI tools without the subscription or the cloud.
Professional-grade voice AI, minus the subscription and the cloud.
<div align="center">
<br/>
@@ -262,7 +263,7 @@ OmniVoice Studio gives you professional-grade AI tools without the subscription
---
## System Requirements
## 🖥️ System Requirements
| | **Minimum** | **Recommended** |
|---|---|---|
@@ -271,17 +272,27 @@ OmniVoice Studio gives you professional-grade AI tools without the subscription
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
| **Python** | 3.10+ (managed by `uv`) | 3.113.12 |
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm |
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm (Linux only) |
> [!TIP]
> On GPUs with **≤8 GB VRAM**, OmniVoice automatically offloads TTS to CPU during transcription — no config needed. A dedicated GPU is not required; the entire pipeline runs on CPU (just slower).
> [!NOTE]
> **AMD GPUs:** ROCm acceleration is **Linux-only and opt-in** — pick **"AMD GPU (ROCm)"** on the first-run setup screen or set `OMNIVOICE_TORCH_VARIANT=rocm` ([docs/install/linux.md](docs/install/linux.md#amd-gpu-rocm)). **On Windows, AMD GPUs (incl. Ryzen AI iGPUs) run CPU-only**: PyTorch has no Windows ROCm wheels, so Windows GPU acceleration is NVIDIA/CUDA-only ([docs/install/windows.md](docs/install/windows.md#gpu-support)).
> [!IMPORTANT]
> **macOS Intel (x86_64) is unsupported for the local backend:** the app UI installs, but the Python backend cannot run because PyTorch no longer ships Intel-Mac wheels ([#889](https://github.com/debpalash/OmniVoice-Studio/issues/889)). Intel-Mac users can still point the UI at a remote backend on another machine — see [docs/install/macos.md](docs/install/macos.md).
### TTS Engines
<a id="tts-engines"></a>
OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is always available; additional engines are opt-in and auto-detected. Switch engines in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
### 🗣️ TTS Engines
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
<details>
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
<br/>
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
@@ -304,9 +315,18 @@ OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is al
>
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
### ASR Engines
</details>
OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictation, video dubbing, and subtitle generation — all fully local. **WhisperX** is the cross-platform default; the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
<a id="asr-engines"></a>
### 🎧 ASR Engines
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
<details>
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
<br/>
| Engine | `OMNIVOICE_ASR_BACKEND` | Languages | Best for |
|--------|-------------------------|:---------:|----------|
@@ -324,9 +344,11 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
> **GPU without efficient float16?** On older NVIDIA GPUs (Maxwell/Pascal, GTX 16xx) or after a CTranslate2/cuDNN mismatch, the CTranslate2 ASR engines (WhisperX, Faster-Whisper) can't run `float16` and OmniVoice automatically retries on `int8` — no config needed. If transcription still fails, pin the compute type with the `ASR_COMPUTE_TYPE` env var (escape hatch): `ASR_COMPUTE_TYPE=int8` (or `float32` for CPU). Set it to `int8` and restart the backend.
</details>
---
## Architecture
## 🏗️ Architecture
```
┌─────────────────────────────────────────────────────────────┐
@@ -344,11 +366,50 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
CUDA / MPS / ROCm / CPU (auto-detected + routed)
```
<a id="openai-api"></a>
## 🔌 OpenAI-compatible API
Already have a script, agent, or tool that speaks OpenAI's audio API? Point it at `http://localhost:3900/v1` — no key needed, no code changes. The backend ships a drop-in surface for the audio endpoints, wired to whichever TTS/ASR engine you have active (and yes, `voice` accepts your cloned voice-profile IDs).
| Endpoint | What it does |
|---|---|
| `POST /v1/audio/speech` | TTS — text in; `mp3` / `wav` / `flac` / `opus` / `pcm` out. `tts-1` / `tts-1-hd` map to your active engine; OpenAI voice names (`alloy`, …) are accepted. |
| `POST /v1/audio/transcriptions` | STT — audio file in; `json`, `text`, `verbose_json`, `srt`, or `vtt` out. `whisper-1` maps to your active ASR engine. |
| `GET /v1/audio/voices` | OmniVoice extension — lists every voice profile and engine, so clients can discover your clones. |
```sh
curl http://localhost:3900/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model": "tts-1", "voice": "alloy", "input": "Generated on my own hardware.", "response_format": "wav"}' \
--output speech.wav
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3900/v1", api_key="none") # any string works — nothing checks it
result = client.audio.transcriptions.create(model="whisper-1", file=open("clip.wav", "rb"))
print(result.text)
```
Want the whole surface (100+ endpoints)? The full REST API reference is embedded in the app — **Settings → OpenAPI Reference** (Scalar-powered), or the `{}` button in the footer.
---
## Roadmap
## 🗺️ Roadmap
### ✅ Shipped
### 🔜 Up Next
- 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 🌐 **Hosted Demo** — try OmniVoice without installing anything
- 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
<details>
<summary><b>✅ Everything shipped so far</b> — the receipts, by category</summary>
<br/>
| Category | Features |
|----------|----------|
@@ -359,7 +420,7 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
| **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
| **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
| **ASR** | 9 engines (WhisperX, Faster-Whisper, isolated Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet TDT, Moonshine, FunASR/SenseVoice, sherpa-onnx live dictation), crash-isolated subprocess backend |
| **TTS** | 11 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3), engine routing with GPU preflight |
| **TTS** | 14 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS), engine routing with GPU preflight |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading, engine routing (no silent CPU fallback), diagnostics suite & error journal, restricted-network mirror support |
| **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system, UI scale fix for Linux/WebKitGTK |
@@ -372,16 +433,13 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
| **Remote Backend** | Point the desktop UI at a remote backend URL with bearer auth (Tailscale-documented) |
| **Reliability** | Stall watchdog on bootstrap splash, per-engine GPU compatibility matrix, actionable errors for non-executable engine binaries, setuptools auto-repair |
### 🔜 Up Next
- 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 🌐 **Hosted Demo** — try OmniVoice without installing anything
- 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
</details>
---
## Sponsor / Donate
<a id="sponsor--donate"></a>
## 💜 Sponsor / Donate
OmniVoice Studio is built by one developer using Claude Code and AI agents — and the agent bills are real. Over the last three months I've spent thousands of dollars on Claude subscriptions to keep the features shipping, the bugs fixed, and your issues answered. If OmniVoice has created value for you, helping cover those bills means I can keep developing full-time.
@@ -402,7 +460,9 @@ OmniVoice Studio is built by one developer using Claude Code and AI agents — a
</div>
### Sponsors
<a id="sponsors"></a>
### 🌟 Sponsors
OmniVoice is **free** and **AGPL-3.0** — no paid tier, no SaaS revenue. Sponsors keep development going, and in return get a logo slot here, in the app, and (for top tiers) on the project website. It's a thank-you, never a paywall. **[See tiers & become a sponsor →](SPONSORS.md)**
@@ -420,29 +480,37 @@ OmniVoice is **free** and **AGPL-3.0** — no paid tier, no SaaS revenue. Sponso
---
## Community
## 💬 Community
<div align="center">
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/💬_Discord-Join_Community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord" /></a>
<br/>
<sub>We respond to setup questions within hours, not days.</sub>
</div>
<details>
<summary><b>What happens in there</b></summary>
<br/>
| Channel | What happens there |
|---------|--------------------|
| `#showcase` | Members share their dubs, clones, and voice designs |
| `#help` | Setup issues, GPU troubleshooting, model questions |
| `#feature-requests` | Vote on what gets built next |
| `#dev` | Architecture discussions, PR reviews, engine integrations |
| `#announcements` | Release notes, breaking changes, early access |
| `#announcements` | Release news and the big moments — new versions land here first |
| `#releases` + `#changelog` | Every build and exactly what's inside it |
| `#issues` | Bug reports as forum posts — triaged straight into GitHub issues |
| `#ideas` | Feature requests, discussed and voted on |
| `#discuss-ideas` | Design talk before things get built |
| `#general` | Setup help, GPU troubleshooting, and showing off your dubs |
**[→ Join the Discord](https://discord.gg/bzQavDfVV9)** — we respond to setup questions within hours, not days.
</details>
---
## Contributing
<a id="contributing"></a>
We welcome contributions of all kinds — bug fixes, new TTS engine adapters, UI improvements, docs, and translations.
## 🤝 Contributing
Yes please — bug fixes, new TTS engine adapters, UI improvements, docs, translations. All of it.
- 📖 Read the **[Contributing Guide](CONTRIBUTING.md)** for setup, code style, and PR workflow
- 🐛 Browse [good first issues](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue)
@@ -450,7 +518,7 @@ We welcome contributions of all kinds — bug fixes, new TTS engine adapters, UI
---
## FAQ
## FAQ
<details>
<summary><b>Is this really as good as ElevenLabs?</b></summary>
@@ -485,12 +553,14 @@ Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are availab
<details>
<summary><b>Can I add my own TTS engine?</b></summary>
<br/>
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary. Eleven engines are built in: OmniVoice, CosyVoice 3, GPT-SoVITS, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, KittenTTS, Sherpa-ONNX, plus lazy-registered IndexTTS 2, OmniVoice GGUF, and Supertonic 3. See the <a href="#tts-engines">TTS Engines</a> section for details.
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary. Fourteen engines are built in: OmniVoice, CosyVoice 3, GPT-SoVITS, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, KittenTTS, Sherpa-ONNX, plus lazy-registered IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, and Confucius4-TTS. See the <a href="#tts-engines">TTS Engines</a> section for details.
</details>
---
## License
<a id="license"></a>
## 📜 License
OmniVoice Studio is free and open-source software under the [**GNU Affero General Public License v3.0 (AGPL-3.0)**](https://www.gnu.org/licenses/agpl-3.0.html).
@@ -502,7 +572,7 @@ The bundled `omnivoice/` TTS model by Han Zhu remains Apache-2.0 upstream. See [
---
## Acknowledgments
## 🙏 Acknowledgments
OmniVoice Studio is built on the shoulders of exceptional open-source work:
@@ -521,6 +591,17 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
---
## 🧰 More local open-source from the maker
Like the local-first philosophy? It runs in the family:
| Project | What it is |
|---------|------------|
| [**Opal**](https://github.com/debpalash/Opal) 💠 | **Play everything.** The evolved media player for the next decades of entertainment — video, anime, comics, torrents, Jellyfin/Plex, with local AI built in. |
| [**memxt**](https://github.com/debpalash/memxt) 🧠 | **The fastest benchmarked open-source AI memory system.** 100% local memory for AI agents, with MCP support. |
---
<div align="center">
<br/>
+8
View File
@@ -33,6 +33,14 @@ hiddenimports = [
'uvicorn.lifespan', 'uvicorn.lifespan.on',
'fastapi', 'fastapi.responses', 'starlette',
'multipart',
# SOCKS proxy support (#959). httpx imports socksio lazily inside a
# try/except (only when a socks5:// proxy env var is set), so
# PyInstaller's static tracer never sees it — without this entry the
# frozen installers keep raising "Using SOCKS proxy, but the 'socksio'
# package is not installed" on every model load under a SOCKS proxy,
# even though pyproject.toml ships the package. Guarded by
# tests/test_socks_proxy.py.
'socksio',
# Core
'uuid', 'asyncio',
+243 -64
View File
@@ -375,6 +375,31 @@ _CHUNK_TRANSCRIBE_ATTEMPTS = max(1, int(os.environ.get("OMNIVOICE_TRANSCRIBE_CHU
_sse_event = dub_pipeline.sse_event
_prep_event_helper = dub_pipeline.prep_event # alias; we keep the module-local _prep_event below for the inline one-liner shape
#: User-facing warning emitted when auto voice cloning is skipped because the
#: speaker labels came from the silence-gap heuristic (see _diarize /
#: extract_speaker_clones — gap-based labels routinely mix two people's audio
#: into one reference, which is how "made up" clone voices happen).
CLONE_SKIP_HEURISTIC_MSG = (
"auto voice cloning skipped: speaker labels are gap-based estimates — "
"set up diarization (Settings → Models → pyannote) for per-speaker clones"
)
def _clamp_num_speakers(value) -> Optional[int]:
"""Clamp the user's speaker-count hint to a sane 120 range.
Shared by the SSE and legacy transcribe endpoints so the two can't drift.
None / non-int / out-of-range → None (auto-detect), so a bad query string
can never break a diarization call.
"""
if value is None:
return None
try:
value = int(value)
except (TypeError, ValueError):
return None
return value if 1 <= value <= 20 else None
@router.get("/dub/transcribe-stream/{job_id}")
async def dub_transcribe_stream(
@@ -393,15 +418,14 @@ async def dub_transcribe_stream(
pyannote auto-detects the count — but its auto-detect can collapse a
multi-speaker clip to a single speaker (issue #274). When the user knows
the exact count, supplying it forces pyannote to return that many speakers.
On paths that can't honor the hint exactly (inline ASR turns, the
silence-gap heuristic) it is never silently dropped: the heuristic cycles
the requested count and a `warning` SSE event tells the user how far the
labels can be trusted.
"""
# Clamp to a sane range; ignore anything non-positive / absurd so a bad
# query string can never break the diarization call. None → auto-detect.
if num_speakers is not None:
try:
num_speakers = int(num_speakers)
num_speakers = num_speakers if 1 <= num_speakers <= 20 else None
except (TypeError, ValueError):
num_speakers = None
num_speakers = _clamp_num_speakers(num_speakers)
job = _get_job(job_id)
@@ -428,6 +452,11 @@ async def dub_transcribe_stream(
asr_audio_target = job.get("vocals_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
asr_audio_target = job.get("audio_path")
# #963: onset snapping is only trustworthy on the Demucs vocals
# track. When separation failed/was skipped, dub_pipeline sets
# vocals_path to the mixed audio_path — so compare paths instead
# of trusting the key's presence.
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
preflight_error = "No audio available for transcription."
else:
@@ -625,12 +654,19 @@ async def dub_transcribe_stream(
# leading music/silence (classic case: speech begins at 0:03,
# transcript says 0.0 → the dub plays 3 s early). Snap starts
# forward to the actual speech onset. `audio_np` is the same
# track ASR ran on — vocals.wav when Demucs succeeded.
# track ASR ran on — vocals.wav when Demucs succeeded. #963:
# when it didn't (mixed audio), snapping is disabled — every
# footstep/sigh/score cue is a false onset candidate there.
try:
snap_segment_starts(chunk_segs, audio_np, sr)
snap_segment_starts(chunk_segs, audio_np, sr,
separated_vocals=asr_on_vocals)
except Exception as e:
logger.warning("onset alignment skipped for chunk %d: %s", i, e)
chunk_segs = assign_speakers_heuristic(chunk_segs)
# Provisional per-chunk labels for the streaming UI only — the
# final diarization pass below overwrites them. Honor the user's
# speaker-count hint here too so the interim view doesn't flip
# between 2 and N speakers.
chunk_segs = assign_speakers_heuristic(chunk_segs, num_speakers)
for s in chunk_segs:
s["id"] = f"s{next_seg_id:05x}"
s["text_original"] = s.get("text", "")
@@ -684,33 +720,110 @@ async def dub_transcribe_stream(
return
def _diarize():
"""Returns (segments, warning_payload_or_None).
"""Returns (segments, warning_payload_or_None, labels_source).
`labels_source` records where the speaker labels came from —
`"pyannote"` | `"turns"` | `"heuristic"` — so downstream
auto-clone extraction can refuse to cut reference audio from
gap-based estimates (a mixed-speaker reference is how "made up"
clone voices happen).
`warning_payload` is a structured dict
`{detail, error_class, docs_url}` whenever we silently fell back
to the silence-gap heuristic (no HF_TOKEN, model unavailable,
license not accepted, or pyannote raised). The heuristic only
detects speaker turns from >1.2s silences, so a rapid-fire
man↔woman exchange will read as one speaker. Issue #78 — we
attach an `error_class` so the front-end's errorDocsMap can
render a "See docs" deeplink instead of a dead-end toast.
license not accepted, or pyannote raised) — or whenever the
user's `num_speakers` hint could not be honored exactly. The
heuristic only detects speaker turns from >1.2s silences, so a
rapid-fire man↔woman exchange will read as one speaker. Issue
#78 — we attach an `error_class` so the front-end's errorDocsMap
can render a "See docs" deeplink instead of a dead-end toast.
"""
# The active ASR backend already diarized inline (FunASR cam++):
# use its speaker turns directly and skip pyannote entirely (#182).
if asr_speaker_turns:
logger.info("Using inline ASR diarization (%d turns); skipping pyannote.", len(asr_speaker_turns))
assigned = assign_speakers_from_turns(all_segments, asr_speaker_turns)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
return resplit_segments_by_turns(assigned, all_words, asr_speaker_turns), None
from services.model_manager import (
DIARIZATION_ERR_LICENSE,
DIARIZATION_ERR_NO_TOKEN,
)
from core import error_docs_map
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
def _hint_suffix() -> str:
"""Honest caveat appended to heuristic-fallback warnings when a
multi-speaker hint is set: the count is now honored, but the
heuristic can't attribute voices. (A hint of 1 IS fully
honored — one label — so it needs no caveat.)"""
if not num_speakers or num_speakers < 2:
return ""
return (
f" Your speaker-count setting ({num_speakers}) is only "
f"approximately honored: the heuristic cycles "
f"{num_speakers} speaker labels on silence gaps instead "
f"of recognizing voices, so lines may be attributed to "
f"the wrong speaker."
)
def _use_turns(crash: Exception | None = None, err_sentinel=None):
"""Label from the ASR backend's inline speaker turns; warn when
that means the user's explicit count can't be enforced."""
logger.info(
"Using inline ASR diarization (%d turns)%s.",
len(asr_speaker_turns),
"" if crash else "; skipping pyannote",
)
assigned = assign_speakers_from_turns(all_segments, asr_speaker_turns)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
resplit = resplit_segments_by_turns(assigned, all_words, asr_speaker_turns)
if not num_speakers:
return resplit, None, "turns"
error_class = (
"HF_AUTH_FAILED"
if err_sentinel == DIARIZATION_ERR_NO_TOKEN
else "PYANNOTE_LICENSE_REQUIRED"
)
if crash:
detail = (
f"Speaker diarization crashed mid-run "
f"({type(crash).__name__}); falling back to the ASR "
f"engine's built-in speaker turns. Speaker-count hint "
f"ignored: the detected count may differ from the "
f"{num_speakers} you set."
)
else:
detail = (
f"Speaker-count hint ignored: pyannote diarization is "
f"unavailable, so the ASR engine's built-in speaker "
f"turns were used and the detected count may differ "
f"from the {num_speakers} you set. Set up diarization "
f"(Settings → Models → pyannote) to enforce an exact "
f"speaker count."
)
return resplit, {
"detail": detail,
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
"speaker_hint": {"requested": num_speakers, "status": "ignored"},
}, "turns"
# The active ASR backend already diarized inline (FunASR cam++):
# its turns are the fast path and skip pyannote entirely (#182) —
# but ONLY when the user didn't set an explicit speaker count.
# Inline turns are labeled per-30s-chunk and can't be forced to N
# speakers, so a set num_speakers prefers pyannote — the one
# engine that honors an exact count. When pyannote can't load,
# the turns are still the best labels available; use them and say
# so instead of silently eating the hint.
diar_pipe = None
err_sentinel = None
if asr_speaker_turns:
if num_speakers:
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
if not diar_pipe:
return _use_turns(err_sentinel=err_sentinel)
logger.info(
"num_speakers=%d set: preferring pyannote over %d inline "
"ASR turns (only pyannote honors an exact count).",
num_speakers, len(asr_speaker_turns),
)
else:
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
if not diar_pipe:
# Phase 1 AUTH-01: ask the resolver (App → Env → HF-CLI),
# not just the env var. This is the #35 fix — users who
@@ -761,13 +874,20 @@ async def dub_transcribe_stream(
f"heuristic; rapid speaker turns may be merged."
)
error_class = "PYANNOTE_LICENSE_REQUIRED"
warning = {
"detail": detail + _hint_suffix(),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
}
if num_speakers:
warning["speaker_hint"] = {
"requested": num_speakers,
"status": "approximate" if num_speakers > 1 else "honored",
}
return (
assign_speakers_heuristic(all_segments),
{
"detail": detail,
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
},
assign_speakers_heuristic(all_segments, num_speakers),
warning,
"heuristic",
)
try:
# Pass the user's speaker-count hint through to pyannote when
@@ -782,9 +902,14 @@ async def dub_transcribe_stream(
assigned = assign_speakers_from_diarization(all_segments, diar)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
return resplit_segments_by_diarization(assigned, all_words, diar), None
return resplit_segments_by_diarization(assigned, all_words, diar), None, "pyannote"
except Exception as e:
logger.error(f"Diarization failed: {e}")
# Inline ASR turns beat the silence-gap heuristic as a crash
# fallback (this path is reachable with turns present since a
# set num_speakers routes turns-jobs through pyannote).
if asr_speaker_turns:
return _use_turns(crash=e)
# Mid-run failure — classify against the same sentinels so a
# post-load 401 (rare but possible after a token rotation)
# still gets the right docs deeplink.
@@ -795,36 +920,50 @@ async def dub_transcribe_stream(
if err_class_post == DIARIZATION_ERR_LICENSE
else "PYANNOTE_LICENSE_REQUIRED" # LOAD failures land here too
)
warning = {
"detail": (
f"Speaker diarization crashed mid-run "
f"({type(e).__name__}); falling back to a silence-gap "
f"heuristic. Rapid speaker turns may be merged."
+ _hint_suffix()
),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
}
if num_speakers:
warning["speaker_hint"] = {
"requested": num_speakers,
"status": "approximate" if num_speakers > 1 else "honored",
}
return (
assign_speakers_heuristic(all_segments),
{
"detail": (
f"Speaker diarization crashed mid-run "
f"({type(e).__name__}); falling back to a silence-gap "
f"heuristic. Rapid speaker turns may be merged."
),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
},
assign_speakers_heuristic(all_segments, num_speakers),
warning,
"heuristic",
)
fut_diar = loop.run_in_executor(_gpu_pool, _diarize)
final_segs = None
diar_warning = None
labels_source = "heuristic"
while True:
done, pending = await asyncio.wait([fut_diar], timeout=5.0)
if done:
final_segs, diar_warning = done.pop().result()
final_segs, diar_warning, labels_source = done.pop().result()
break
yield _sse_event("ping", {})
if diar_warning:
logger.warning("diarization fallback: %s", diar_warning.get("detail"))
yield _sse_event("warning", {
payload = {
"detail": diar_warning.get("detail"),
"source": "diarization",
"error_class": diar_warning.get("error_class"),
"docs_url": diar_warning.get("docs_url"),
})
}
# Machine-readable trail of what happened to the user's
# speaker-count hint (the `detail` text carries the human story).
if diar_warning.get("speaker_hint"):
payload["speaker_hint"] = diar_warning["speaker_hint"]
yield _sse_event("warning", payload)
job["segments"] = final_segs
@@ -836,17 +975,37 @@ async def dub_transcribe_stream(
try:
from services.speaker_clone import extract_speaker_clones, auto_profile_id
vocals_for_clone = job.get("vocals_path") or asr_audio_target
fut_clones = loop.run_in_executor(
_cpu_pool, extract_speaker_clones,
vocals_for_clone, final_segs, os.path.dirname(vocals_for_clone),
)
clones = None
while True:
done, pending = await asyncio.wait([fut_clones], timeout=5.0)
if done:
clones = done.pop().result()
break
yield _sse_event("ping", {})
clones = {}
if labels_source == "heuristic":
# Clone-purity guard: heuristic labels are silence-gap
# estimates, not voice identity — a per-speaker reference cut
# from them routinely concatenates two people's audio and the
# clone sounds "made up". Skip auto-clones and say so instead
# of shipping bad ones. (extract_speaker_clones enforces the
# same guard internally; this branch exists to surface the
# warning to the user.)
logger.info(
"auto speaker clones skipped (labels_source=heuristic, job=%s)",
job_id,
)
yield _sse_event("warning", {
"detail": CLONE_SKIP_HEURISTIC_MSG,
"source": "speaker_clone",
})
else:
fut_clones = loop.run_in_executor(
_cpu_pool, lambda: extract_speaker_clones(
vocals_for_clone, final_segs,
os.path.dirname(vocals_for_clone),
labels_source=labels_source,
),
)
while True:
done, pending = await asyncio.wait([fut_clones], timeout=5.0)
if done:
clones = done.pop().result()
break
yield _sse_event("ping", {})
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
# own reference from the vocals so the dub of each line matches the
# prosody of its source line. Short lines fall back to the
@@ -960,18 +1119,28 @@ async def dub_transcribe_stream(
@router.post("/dub/transcribe/{job_id}")
async def dub_transcribe(job_id: str):
async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
"""Legacy synchronous transcribe (kept for the headless CLI).
`num_speakers` mirrors the SSE endpoint's query param (same 120 clamp):
an exact speaker count forwarded to pyannote, or cycled by the silence-gap
heuristic when pyannote is unavailable. None → auto-detect.
"""
num_speakers = _clamp_num_speakers(num_speakers)
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
_model = await get_model()
def _transcribe():
asr_audio_target = job.get("vocals_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
asr_audio_target = job.get("audio_path")
# #963: same source-awareness as the SSE endpoint — vocals_path
# falls back to the mixed audio_path when Demucs failed/skipped.
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
import torch
detected_lang = None
@@ -1015,10 +1184,13 @@ async def dub_transcribe(job_id: str):
segments = segment_transcript(result, duration=job.get("duration", 0.0), scene_cuts=scene_cuts)
# #280: snap segment starts forward to the actual speech onset so the
# dub doesn't begin seconds before the original speaker does.
# dub doesn't begin seconds before the original speaker does. #963:
# only on the separated vocals track — on mixed audio every ambient
# sound is a false onset candidate, so snapping is disabled.
try:
audio_for_onset, onset_sr = sf.read(asr_audio_target, dtype="float32")
snap_segment_starts(segments, audio_for_onset, onset_sr)
snap_segment_starts(segments, audio_for_onset, onset_sr,
separated_vocals=asr_on_vocals)
except Exception as e:
logger.warning("onset alignment skipped: %s", e)
@@ -1026,13 +1198,20 @@ async def dub_transcribe(job_id: str):
if diar_pipe:
try:
diar_target = job.get("vocals_path") or job.get("audio_path")
diarization = diar_pipe(diar_target)
# Same hint pass-through as the SSE endpoint (#274): omit the
# kwarg entirely when unset so we don't depend on it existing
# in every pyannote build.
if num_speakers:
logger.info("Diarizing with num_speakers=%d (user hint)", num_speakers)
diarization = diar_pipe(diar_target, num_speakers=num_speakers)
else:
diarization = diar_pipe(diar_target)
segments = assign_speakers_from_diarization(segments, diarization)
except Exception as e:
logger.error(f"Pyannote diarization failed during inference: {e}. Falling back to heuristic.")
segments = assign_speakers_heuristic(segments)
segments = assign_speakers_heuristic(segments, num_speakers)
else:
segments = assign_speakers_heuristic(segments)
segments = assign_speakers_heuristic(segments, num_speakers)
# Previously ran `segment_for_subtitles(segments)` here. Removed 2026-04-21 —
# that splitter enforces Netflix's 17 CPS reading-speed ceiling which
+78 -15
View File
@@ -170,8 +170,42 @@ async def dub_list_tracks(job_id: str):
return {"tracks": job.get("dubbed_tracks", {})}
def _segments_for_lang(job: dict, lang: "str | None") -> list:
"""Job segments with `text` overlaid from ``job["segments_i18n"][lang]``.
P1.2 ``job["segments"]`` is single-slot: it holds whichever language was
generated LAST, so exporting subtitles for track A after generating track B
emitted B's text under A's language label (the "N identical subtitle
files" class). ``segments_i18n`` ({lang: {segKey: text}}, written by
``dub_generate._sync_job_segments``) preserves each generated track's text;
this overlays it non-destructively when present.
Back-compat: no lang requested, no ``segments_i18n`` on the job (predates
the field), no entry for this lang, or no text for a given segment each
falls back to the segment as-is, i.e. exactly today's behaviour.
Segment keys are the stable id (str) with the list index (str) as the
legacy fallback, mirroring how the map is written.
"""
segments = job.get("segments", [])
if not lang:
return segments
i18n = job.get("segments_i18n")
lang_texts = i18n.get(lang) if isinstance(i18n, dict) else None
if not isinstance(lang_texts, dict) or not lang_texts:
return segments
out = []
for i, seg in enumerate(segments):
key = str(seg.get("id")) if seg.get("id") is not None else str(i)
txt = lang_texts.get(key)
if txt is None:
txt = lang_texts.get(str(i))
out.append(dict(seg, text=txt) if isinstance(txt, str) and txt.strip() else seg)
return out
def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
fitted_segments: "list[dict] | None" = None) -> str | None:
fitted_segments: "list[dict] | None" = None,
lang: "str | None" = None) -> str | None:
"""Build a temp SRT from job segments for use with ffmpeg's subtitles filter.
Returned path is already ffmpeg-filter-safe (plain ASCII basename under exports_dir).
@@ -181,8 +215,11 @@ def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
fitted timeline when provided, cue times come from there instead of
the original ``job["segments"]`` timings, so burned subs track the
retimed video / fitted audio rather than the source timeline.
``lang`` (P1.2): burn the named track's text (see ``_segments_for_lang``)
instead of whatever language generated last.
"""
segments = job.get("segments", [])
segments = _segments_for_lang(job, lang)
if not segments:
return None
if fitted_segments:
@@ -486,7 +523,9 @@ async def dub_download(
# Smart Fit: cue times come from the fitted timeline — that's where the
# dubbed audio actually sits, whether or not the video retime succeeds.
fitted_segments = _fitted_segments_for(job, default_track) if default_track and default_track != "original" else None
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments) if burn_subs else None
# Burn the DEFAULT track's text (P1.2) — it's the audio the viewer hears.
_burn_lang = default_track if default_track and default_track != "original" else None
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments, lang=_burn_lang) if burn_subs else None
# ── Smart Fit video retime (two-tier) ─────────────────────────────────
# Tier 1 (≤48 chunks): single filter_complex graph inlined into the mux
@@ -1125,20 +1164,40 @@ async def dub_get_audio(job_id: str):
raise HTTPException(status_code=404, detail="Audio file not found")
return FileResponse(audio, media_type="audio/wav")
def _seg_wav_candidates(job: dict, lang: "str | None", seg_keys: tuple) -> list:
"""Per-segment WAV name candidates, language-keyed first (P1.3).
Generation writes ``seg_{lang}_{id}.wav`` now; ``lang`` defaults to the
job's last-generated track. Legacy un-keyed names (``seg_{id}.wav`` /
``seg_{index}.wav``) stay as fallbacks so jobs rendered by previous
builds keep serving their audio these read-only endpoints keep the
permissive fallback that matches their historic behaviour (the strict
single-track gate lives on the generate splice path, where a wrong-
language read would be baked into a track).
"""
lang = lang or job.get("language_code")
keys = []
if lang:
keys.extend(f"{lang}_{k}" for k in seg_keys)
keys.extend(seg_keys)
return keys
@router.get("/dub/preview/{job_id}/{segment_index}")
async def dub_preview_segment(job_id: str, segment_index: int):
async def dub_preview_segment(job_id: str, segment_index: int, lang: str = Query(None)):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Resolve the stable-id-named WAV via the render manifest; fall back to the
# legacy index name for jobs rendered before id-based naming (#185). Each
# candidate is realpath-normalised and containment-checked BEFORE any
# filesystem access, so the guard dominates every path sink.
# Resolve the stable-id-named WAV via the render manifest — language-keyed
# name first (P1.3), then the legacy id/index names for jobs rendered
# before per-language (and before id-based, #185) naming. Each candidate
# is realpath-normalised and containment-checked BEFORE any filesystem
# access, so the guard dominates every path sink.
order = job.get("seg_order") or []
seg_id = order[segment_index] if 0 <= segment_index < len(order) else segment_index
base = os.path.realpath(DUB_DIR)
seg_path = None
for _sid in (seg_id, segment_index):
for _sid in _seg_wav_candidates(job, lang, (seg_id, segment_index)):
cand = os.path.realpath(dub_seg_path(job_id, _sid))
if cand.startswith(base + os.sep) and os.path.exists(cand):
seg_path = cand
@@ -1342,13 +1401,16 @@ def _fitted_cue_times(job: dict, lang: str | None) -> list | None:
async def dub_export_srt(
job_id: str,
dual: bool = False,
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
segments = job.get("segments", [])
# P1.2 — text follows the REQUESTED track, not whichever language was
# generated last (job["segments"] is single-slot). Legacy jobs without
# segments_i18n fall back to today's behaviour.
segments = _segments_for_lang(job, lang)
if not segments:
raise HTTPException(status_code=400, detail="No transcript segments available")
@@ -1391,13 +1453,14 @@ def _format_vtt_time(seconds):
async def dub_export_vtt(
job_id: str,
dual: bool = False,
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
segments = job.get("segments", [])
# Same per-track text resolution as /dub/srt (see comment there, P1.2).
segments = _segments_for_lang(job, lang)
if not segments:
raise HTTPException(status_code=400, detail="No transcript segments available")
@@ -1427,7 +1490,7 @@ async def dub_export_vtt(
@router.get("/dub/export-segments/{job_id}")
async def dub_export_segments_zip(job_id: str):
async def dub_export_segments_zip(job_id: str, lang: str = Query(None)):
import zipfile
job = _get_job(job_id)
if not job:
@@ -1445,7 +1508,7 @@ async def dub_export_segments_zip(job_id: str):
seg_id = order[i] if i < len(order) else i
# realpath + containment guard before any filesystem access.
seg_path = None
for _sid in (seg_id, i):
for _sid in _seg_wav_candidates(job, lang, (seg_id, i)):
cand = os.path.realpath(dub_seg_path(job_id, _sid))
if cand.startswith(base + os.sep) and os.path.exists(cand):
seg_path = cand
+118 -20
View File
@@ -87,6 +87,62 @@ def _sync_job_segments(job: dict, req: DubRequest) -> None:
merged.append(row)
job["segments"] = merged
# P1.2 — per-language text, additively. `job["segments"]` stays the flat
# single-slot map every existing consumer reads (last generated language);
# `job["segments_i18n"]` preserves EACH generated track's text so
# /dub/srt|vtt?lang= can emit that language instead of N identical files.
# Shape: { langCode: { segKey: text } } where segKey is the segment's
# stable id (str) or, for id-less legacy segments, its list index (str).
# The whole per-language map is rebuilt on every generate of that language
# (the request always carries the full segment list), so deleted segments
# never linger. Jobs predating this field simply lack it — every reader
# falls back to `job["segments"]`.
lang = (req.language_code or "und").strip() or "und"
i18n = job.setdefault("segments_i18n", {})
i18n[lang] = {
(str(row["id"]) if row.get("id") is not None else str(i)): row["text"]
for i, row in enumerate(merged)
}
def _seg_hashes_by_lang(job: dict) -> dict:
"""Per-language segment fingerprints: { langCode: { segId: hash } }.
Additive migration (P1.3): jobs written by previous builds carry ONE flat
`seg_hashes` map that was overwritten by whichever language generated
last. That flat map can only describe the job's last-generated track, so
it is attributed to `job["language_code"]` (which generate has always
kept in lock-step with the last run). When even that is unknown the
legacy hashes are dropped segments then read as stale and regenerate
cleanly, which is safer than guessing a language and splicing wrong-track
audio. Note the legacy hashes also predate language-scoped fingerprints
(see services.incremental.segment_fingerprint), so they compare stale
once regardless carrying them over just preserves the job shape.
"""
by_lang = job.get("seg_hashes_by_lang")
if not isinstance(by_lang, dict):
by_lang = {}
legacy = job.get("seg_hashes")
prev_lang = job.get("language_code")
if isinstance(legacy, dict) and legacy and prev_lang:
by_lang[prev_lang] = dict(legacy)
job["seg_hashes_by_lang"] = by_lang
return by_lang
def _legacy_seg_cache_ok(job: dict, lang_code: str) -> bool:
"""May this run reuse legacy un-keyed ``seg_<id>.wav`` files?
Only when no OTHER language's audio could be sitting in them: the job has
no dubbed track in a different language. Single-language jobs rendered by
previous builds therefore keep their whole on-disk cache; the moment a
job carries a second language the un-keyed files are ambiguous (they hold
whichever language wrote them last) and must never be spliced into a
track again the P1.3 cross-contamination class.
"""
tracks = job.get("dubbed_tracks") or {}
return not any(lc != lang_code for lc in tracks)
router = APIRouter()
@@ -107,6 +163,18 @@ async def dub_generate(job_id: str, req: DubRequest):
all_segment_wavs = []
sync_scores = []
# Track language for this run. Everything per-track — the per-segment
# WAV cache, fingerprints, seg_wav_kind — is keyed by it (P1.3) so a
# multi-language job's tracks can't cross-contaminate.
lang_code = req.language_code or "und"
def _seg_lang_path(seg_key) -> str:
# Per-language per-segment WAV: seg_{lang}_{id}.wav. Built through
# dub_seg_path so the sanitisation + DUB_DIR containment guard
# apply to the combined key. Legacy un-keyed seg_{id}.wav files
# remain readable via the gated fallback (_legacy_seg_cache_ok).
return dub_seg_path(job_id, f"{lang_code}_{seg_key}")
# Throttle the device cache flush. empty_cache() is a synchronous
# device stall, so calling it every segment (as the old code did)
# serialised the GPU loop; the batched-I/O design it replaced kept
@@ -236,7 +304,16 @@ async def dub_generate(job_id: str, req: DubRequest):
# double-compress. Force one full regen; afterwards seg_wav_kind is
# "natural" and partial regen / fit-only re-mix (regen_only=[]) work.
# Jobs predating this field have unknown kind → also regen once.
if strategy == "smart_fit" and regen_only is not None and job.get("seg_wav_kind") != "natural":
# P1.3: the kind is per-track now (each language renders under its own
# strategy); the flat job["seg_wav_kind"] is only consulted for jobs
# written before the per-language map existed — once the map is
# present, a language without an entry has unknown-kind WAVs (or none
# at all) and must regen once, exactly like the pre-field case.
_kind_map = job.get("seg_wav_kind_by_lang")
_wav_kind = (
_kind_map.get(lang_code) if isinstance(_kind_map, dict) else job.get("seg_wav_kind")
)
if strategy == "smart_fit" and regen_only is not None and _wav_kind != "natural":
regen_only = None
# Manifest: stable segment id per current index. Per-segment WAVs are
# named by stable id (dub_seg_path) so regen reuses the right audio after
@@ -283,12 +360,19 @@ async def dub_generate(job_id: str, req: DubRequest):
# Partial regen: if this segment isn't in the allow-list, reuse its
# previously-rendered WAV so the final mix still covers the timeline.
if regen_only is not None and seg_id not in regen_only:
seg_wav_path = dub_seg_path(job_id, seg_id)
if not os.path.exists(seg_wav_path):
# Back-compat: jobs rendered before id-named files used seg_{index}.wav.
_legacy = dub_seg_path(job_id, i)
if os.path.exists(_legacy):
seg_wav_path = _legacy
# This track's own cache first (seg_{lang}_{id}.wav). Legacy
# un-keyed files (seg_{id}.wav / seg_{index}.wav) are reused
# ONLY when no other-language track exists on the job — a
# multi-track job's un-keyed files hold whichever language
# rendered last, and splicing them here was exactly how
# "Regen N changed" mixed language B into track A (P1.3).
seg_wav_path = _seg_lang_path(seg_id)
if not os.path.exists(seg_wav_path) and _legacy_seg_cache_ok(job, lang_code):
for _legacy_key in (seg_id, i):
_legacy = dub_seg_path(job_id, _legacy_key)
if os.path.exists(_legacy):
seg_wav_path = _legacy
break
if os.path.exists(seg_wav_path):
try:
_t_cache_0 = time.perf_counter()
@@ -584,6 +668,9 @@ async def dub_generate(job_id: str, req: DubRequest):
# and job flush to the batch-write phase after the GPU loop.
_seg_fp = None
try:
# track_lang scopes the hash to THIS track (P1.3); the
# client-side recompute (/tools/incremental) sends the
# same code, so parity (#281 class) holds per language.
_seg_fp = segment_fingerprint({
"text": seg.text,
"target_lang": getattr(seg, "target_lang", None),
@@ -592,7 +679,7 @@ async def dub_generate(job_id: str, req: DubRequest):
"speed": getattr(seg, "speed", None),
"direction": getattr(seg, "direction", None),
"effect_preset": getattr(seg, "effect_preset", None),
})
}, track_lang=lang_code)
except Exception as e:
logger.debug("seg fingerprint skipped for %s: %s", seg_id, e)
@@ -601,7 +688,7 @@ async def dub_generate(job_id: str, req: DubRequest):
# RVC needs the WAV on disk, so write it immediately only
# when RVC is active (uncommon path).
if rvc_is_enabled():
seg_wav_path = dub_seg_path(job_id, seg_id)
seg_wav_path = _seg_lang_path(seg_id)
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
try:
await loop.run_in_executor(_gpu_pool, apply_rvc, seg_wav_path)
@@ -619,15 +706,16 @@ async def dub_generate(job_id: str, req: DubRequest):
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'RVC skipped: {str(e)[:120]}'})}\n\n"
# Watermark this FRESH TTS output exactly once, right before it
# is persisted. The same seg_<id>.wav is BOTH the downloadable
# per-segment file AND the assembly input for the final track,
# so marking it here (and nowhere else) gives the downloadable
# WAV its mark back and the final mix inherits it — no double-
# mark. Cached-reuse audio is already marked; silence/zero slots
# carry no speech to mark, so neither is re-watermarked.
# is persisted. The same seg_{lang}_{id}.wav is BOTH the
# downloadable per-segment file AND the assembly input for the
# final track, so marking it here (and nowhere else) gives the
# downloadable WAV its mark back and the final mix inherits it —
# no double-mark. Cached-reuse audio is already marked;
# silence/zero slots carry no speech to mark, so neither is
# re-watermarked.
audio_tensor = embed_watermark(audio_tensor, _model.sampling_rate)
seg_wav_path = dub_seg_path(job_id, seg_id)
seg_wav_path = _seg_lang_path(seg_id)
try:
# Keep the existing per-segment WAV contract for previews
# and partial regeneration, but do not keep the tensor in RAM.
@@ -663,12 +751,19 @@ async def dub_generate(job_id: str, req: DubRequest):
# Per-segment WAVs were written during the loop to keep RAM bounded.
# Flush only lightweight fingerprints/quality metadata here.
_t_diskw_0 = time.perf_counter()
hashes = job.setdefault("seg_hashes", {})
# P1.3 — fingerprints live per language so each track's staleness is
# judged against ITS OWN last generate. The flat job["seg_hashes"] is
# kept as a mirror of the CURRENT track's map: every existing consumer
# (the `done` event, dub-history restore, older frontends) already
# treats it as "the hashes of the language generated last", which is
# exactly what it now provably contains.
hashes = _seg_hashes_by_lang(job).setdefault(lang_code, {})
quality_map = job.setdefault("seg_num_step", {})
for (_si, _sr, _sid, _fp, _nstep) in _pending_seg_writes:
if _fp is not None:
hashes[_sid] = _fp
quality_map[_sid] = _nstep
job["seg_hashes"] = dict(hashes)
# Single job flush instead of one per 8 segments.
_save_job(job_id, job)
_t_diskw = time.perf_counter() - _t_diskw_0
@@ -759,7 +854,6 @@ async def dub_generate(job_id: str, req: DubRequest):
# not from the plan — so subtitles land exactly on the audio.
fitted_cues: list[dict] = []
lang_code = req.language_code or "und"
track_path = os.path.join(DUB_DIR, job_id, f"dubbed_{lang_code}.wav")
os.makedirs(os.path.dirname(track_path), exist_ok=True)
@@ -1049,8 +1143,12 @@ async def dub_generate(job_id: str, req: DubRequest):
job["dubbed_tracks"][lang_code]["fit_fp"] = fit_fp
# Record what kind of per-segment WAVs are on disk so a later
# smart_fit run knows whether partial regen / fit-only re-mix can
# reuse them ("natural") or must regen once ("slotted").
job["seg_wav_kind"] = "slotted" if strategy == "strict_slot" else "natural"
# reuse them ("natural") or must regen once ("slotted"). Per-track
# (P1.3) — each language renders under its own strategy; the flat
# field stays in lock-step for older readers.
_kind = "slotted" if strategy == "strict_slot" else "natural"
job.setdefault("seg_wav_kind_by_lang", {})[lang_code] = _kind
job["seg_wav_kind"] = _kind
_save_job(job_id, job)
_t_total = time.perf_counter() - _t_start
+57 -7
View File
@@ -308,14 +308,63 @@ async def dub_translate(req: TranslateRequest):
# first is fine — the refine LLM is a separate network provider.
return await _maybe_cinematic(translated, req, src_lang, loop)
# OpenAI / Ollama Local LLM Translation
# LLM translation — resolves through the LLM Skills registry: per-skill
# "Dub translation" override → global active provider (Settings → LLM
# Providers). The keys users configure + test in the app now actually
# power this engine; the raw TRANSLATE_* env vars stay working as a
# power-user override so pre-skills setups see zero behavior change.
if provider == "openai":
base_url = os.environ.get("TRANSLATE_BASE_URL")
model_name = os.environ.get("TRANSLATE_MODEL", "gpt-3.5-turbo")
from openai import OpenAI
# max_retries=0: a 429 + long Retry-After must not let one segment's
# SDK call sleep+retry and blow the overall translate wall time.
client = OpenAI(base_url=base_url, api_key=api_key or "local", max_retries=0)
from services import llm_skills
llm_timeout = llm_skills._default_timeout()
handle = None
try:
handle = llm_skills.resolve_skill_client("dub_translation")
except Exception: # noqa: BLE001 — resolution must never 500 a translate
logger.exception("dub_translation skill resolution failed; trying env fallback")
if handle is not None:
client = handle.client
model_name = handle.model
llm_timeout = handle.timeout
# The provider-store key never touches env; resolve it so the
# error scrubber below can redact it if a provider echoes it.
try:
from services import llm_providers
api_key = llm_providers.resolve_api_key(
llm_skills.effective_provider("dub_translation")) or api_key
except Exception: # noqa: BLE001 — scrub-key resolution is best-effort
pass
elif os.environ.get("TRANSLATE_BASE_URL") or api_key:
# Legacy env-only setup (no provider configured in-app).
from openai import OpenAI
# max_retries=0: a 429 + long Retry-After must not let one segment's
# SDK call sleep+retry and blow the overall translate wall time.
client = OpenAI(base_url=os.environ.get("TRANSLATE_BASE_URL"),
api_key=api_key or "local", max_retries=0)
model_name = os.environ.get("TRANSLATE_MODEL", "gpt-4o-mini")
else:
# Nothing configured anywhere — name the exact next step instead
# of letting an empty key surface as a raw 401 per segment.
try:
reason = llm_skills.resolve_skill("dub_translation").reason
except Exception: # noqa: BLE001
reason = None
if reason == "disabled":
friendly = (
"The LLM translation engine is turned off — enable the "
"'Dub translation' skill in Settings → LLM Skills, or "
"pick another engine in the Engine dropdown."
)
else:
friendly = (
"The LLM translation engine has no provider configured. "
"Add and test one in Settings → LLM Providers (it powers "
"this engine; route it per-skill in Settings → LLM "
"Skills), or set TRANSLATE_BASE_URL + TRANSLATE_API_KEY "
"+ TRANSLATE_MODEL. Or pick another engine in the "
"Engine dropdown."
)
return JSONResponse(status_code=400, content={"error": friendly})
def _build_prompt(src_code: str, tgt_code: str) -> str:
"""Build a system prompt that resists hallucinations on small
@@ -377,6 +426,7 @@ async def dub_translate(req: TranslateRequest):
res = client.chat.completions.create(
model=model_name,
temperature=0.2, # less drift than default 1.0
timeout=llm_timeout, # bound per call (OMNIVOICE_LLM_TIMEOUT, 45s default)
messages=[
{"role": "system", "content": sys_for_attempt},
{"role": "user", "content": seg.text},
+13 -2
View File
@@ -282,7 +282,8 @@ def save_llm_provider(provider_id: str, body: _LLMProviderBody):
A None field is left unchanged; an empty api_key clears the stored key.
"""
from services import llm_providers
if llm_providers.get_provider(provider_id) is None:
p = llm_providers.get_provider(provider_id)
if p is None:
raise HTTPException(status_code=404, detail=f"unknown provider {provider_id!r}")
if body.api_key is not None:
llm_providers.save_key(provider_id, body.api_key.strip())
@@ -290,7 +291,17 @@ def save_llm_provider(provider_id: str, body: _LLMProviderBody):
provider_id, base_url=body.base_url, model=body.model,
account_id=body.account_id,
)
if body.make_active:
# An explicit save also claims the active slot when the user has never
# chosen a provider (#963). Without this, a saved-and-tested local
# provider (Ollama/LM Studio) evaporates on restart: active_provider_id()
# deliberately excludes local providers from auto-select, so the plain
# "Save" left nothing persisted to resolve. Gated on the STORED selection
# only — an explicit prior choice is never stolen by a plain save, and an
# unconfigured provider can't claim the slot.
if body.make_active or (
llm_providers.stored_active_provider_id() is None
and llm_providers.is_configured(p)
):
llm_providers.set_active_provider(provider_id)
return list_llm_providers()
+5 -3
View File
@@ -530,14 +530,16 @@ async def install_model(req: InstallModelRequest):
_install_cooldowns[req.repo_id] = _time_fail.time()
# #874: when the install failed because the configured HF mirror is
# unreachable, name the mirror + the setting instead of leaking the
# raw connectivity error. No-op for every other failure.
from core.failure import append_hf_mirror_hint
# raw connectivity error. #959: likewise for the SOCKS-proxy class
# (missing socksio fails the download's session construction).
# No-op for every other failure.
from core.failure import append_hint
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
"downloaded": 0, "total": 0, "pct": 0.0,
"phase": "install_error",
"error": append_hf_mirror_hint(str(e)),
"error": append_hint(str(e)),
})
finally:
_cancelled.discard(req.repo_id)
+5
View File
@@ -77,6 +77,10 @@ async def probe(req: ProbeReq):
class IncrementalReq(BaseModel):
segments: list[dict]
stored_hashes: Optional[dict[str, str]] = None
# P1.3 — the ACTIVE track's language code. When set, fingerprints are
# scoped to that language (pass that language's stored hashes alongside);
# omitted → legacy language-agnostic hashing, kept for old callers.
lang: Optional[str] = None
@router.post("/tools/incremental")
@@ -84,6 +88,7 @@ def plan_incremental(req: IncrementalReq):
return incremental.plan_incremental(
req.segments,
stored_hashes=req.stored_hashes or {},
track_lang=req.lang,
)
+37
View File
@@ -41,6 +41,7 @@ _HINTS: dict[str, str] = {
"COMPUTE_TYPE_UNSUPPORTED": "Your GPU doesn't support float16 — OmniVoice retried on int8. If transcription still fails, set OMNIVOICE/ASR_COMPUTE_TYPE=int8 or use CPU.",
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
"OS_INVALID_ARGUMENT": "The OS rejected a file operation (Errno 22 / invalid argument) — in the transcribe path this is the temporary WAV write before ASR. It's almost always the temp directory: missing, read-only, on a full or removed drive, or blocked by antivirus. Check that your system TEMP/TMP folder exists and is writable and the drive has free space (add an OmniVoice antivirus exclusion if you use one), then retry.",
"SOCKS_PROXY_SUPPORT_MISSING": "A SOCKS proxy is configured in your environment (ALL_PROXY/HTTPS_PROXY=socks5://…) and the backend's HTTP client is missing SOCKS support. Newer OmniVoice builds ship SOCKS support (the socksio package) — update the app. If you still see this, unset ALL_PROXY/HTTPS_PROXY for OmniVoice, or run `uv pip install 'httpx[socks]'` in the backend venv, then restart.",
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
@@ -168,6 +169,33 @@ def append_hf_mirror_hint(text: str) -> str:
return f"{text}{hint}" if hint else text
# Classes whose hint is safe to attach on the CONTEXT-FREE surfaces (the
# global 500 handler in main.py, the model-install SSE in setup/download.py),
# where all we have is a raw error string with no stage. Only classes whose
# classify() trigger is unmistakable belong here — e.g. VIDEO_DOWNLOAD_NETWORK
# must NOT be added: its bare "timed out" trigger would stamp a "video server"
# hint on a model-load timeout that leaks through the 500 handler.
_CONTEXT_FREE_HINT_CLASSES = frozenset({
"SOCKS_PROXY_SUPPORT_MISSING",
})
def append_hint(text: str) -> str:
"""``"{text}{hint}"`` for raw-string surfaces (the global 500 handler,
the model-install SSE): the dynamic mirror hint (#874) when that class
applies, else a context-free static class hint (#959). ``text`` unchanged
otherwise a no-op for every other error. Never raises."""
try:
hint = hf_mirror_hint(text)
if not hint:
topic = classify(text)
if topic in _CONTEXT_FREE_HINT_CLASSES:
hint = _HINTS.get(topic, "")
except Exception:
return text
return f"{text}{hint}" if hint else text
def classify(reason: str) -> str:
"""Map a failure reason to a docs-taxonomy key, or "" when unknown.
@@ -220,6 +248,15 @@ def classify(reason: str) -> str:
)
):
return "TRANSFORMERS_IMPORT"
# #959: httpx raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS
# proxy, but the 'socksio' package is not installed. Make sure to install
# httpx using `pip install httpx[socks]`.") when ALL_PROXY/HTTPS_PROXY is
# socks5:// and socksio isn't importable. It surfaced from
# huggingface_hub's get_session() inside model load — a bare 500 on
# /generate with no next step. Checked BEFORE the HF-auth/mirror rules so
# a message that also carries HF wording still names this class.
if "socks proxy" in low or "socksio" in low:
return "SOCKS_PROXY_SUPPORT_MISSING"
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
):
+1 -1
View File
@@ -24,7 +24,7 @@ from pathlib import Path
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.3.9"
_FALLBACK_VERSION = "0.3.11"
def _fallback_version() -> str:
+17 -6
View File
@@ -155,6 +155,16 @@ from logging.handlers import RotatingFileHandler
# written to prefs.json so they survive backend restarts. Read them back
# here — before any user code reads os.environ — so the values are available
# from startup.
#
# Legacy (≤v0.3.7) Translation-LLM rows (env.TRANSLATE_*) must migrate into
# the custom LLM provider's settings store BEFORE the re-import below — once
# TRANSLATE_BASE_URL lands in os.environ it hijacks the LLM provider
# selection for the whole session (#963). Real env vars are untouched.
try:
from services.llm_providers import migrate_legacy_translate_prefs
migrate_legacy_translate_prefs()
except Exception:
pass # never block startup on the migration; it retries next launch
_PERSISTED_ENV_PREFIX = "env."
try:
from core.prefs import _load as _load_all_prefs
@@ -718,13 +728,14 @@ async def global_exception_handler(request: Request, exc: Exception):
# #874: a model download that failed because the CONFIGURED Hugging Face
# mirror (HF_ENDPOINT) is unreachable used to leak the raw transformers
# message ("We couldn't connect to 'https://hf-mirror.com' …") as the 500
# detail with no next step. Appending the shared mirror hint HERE covers
# every route that can leak a model-load/download error (generate, dub,
# archetypes, …), not just TTS generate. append_hf_mirror_hint is a no-op
# for every other error and never raises.
from core.failure import append_hf_mirror_hint
# detail with no next step. #959: same story for the SOCKS-proxy class
# ("Using SOCKS proxy, but the 'socksio' package is not installed").
# Appending the shared hints HERE covers every route that can leak a
# model-load/download error (generate, dub, archetypes, …), not just TTS
# generate. append_hint is a no-op for every other error and never raises.
from core.failure import append_hint
return JSONResponse(
{"detail": append_hf_mirror_hint(str(exc)), "error_class": _entry.get("error_class")},
{"detail": append_hint(str(exc)), "error_class": _entry.get("error_class")},
status_code=500,
headers=headers,
)
+7 -1
View File
@@ -1662,7 +1662,13 @@ class _LazyASRRegistry(dict):
def __iter__(self):
seen = set()
for k in dict.__iter__(self):
# Snapshot the live keys before yielding — see _LazyRegistry.__iter__ in
# tts_backend.py. A concurrent lazy __getitem__ inserts into self, and
# list_backends() runs in a FastAPI threadpool, so a *live* dict iterator
# held open across the per-engine is_available() probes would raise
# "dictionary changed size during iteration". list() consumes it
# atomically under the GIL, closing the window.
for k in list(dict.__iter__(self)):
seen.add(k)
yield k
for k in self._LAZY:
+21 -8
View File
@@ -104,9 +104,10 @@ def synthesize_chapter(
):
"""Render a chapter's spans to one waveform via an injected ``synth``.
``synth(text, voice_id, speed)`` returns a 1-D float32 audio tensor for a
span of text in the given voice (``speed`` may be ``None`` for the engine
default). Long spans are split with the ``chunked_tts`` splitter and
``synth(text, voice_id, speed)`` returns a float32 audio tensor 1-D
``(samples,)`` or ``(channels, samples)``; real engines emit ``(1, samples)``
per the ``TTSBackend`` contract (#897) — for a span of text in the given
voice (``speed`` may be ``None`` for the engine default). Long spans are split with the ``chunked_tts`` splitter and
crossfaded; inter-span ``pause_ms_after`` becomes silence. ``lexicon`` (when
given) respells each span's text before chunking so the engine pronounces
tricky words correctly; a ``None``/empty lexicon is a no-op pass-through.
@@ -118,23 +119,35 @@ def synthesize_chapter(
from services.chunked_tts import concatenate_audio_chunks, split_text_into_chunks
from services.pronunciation import apply_lexicon
parts: list = []
items: list = [] # ("a", tensor) for audio, ("s", n_samples) for silence
for span in spans:
if span.text:
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
if len(rendered) == 1:
parts.append(rendered[0])
items.append(("a", rendered[0]))
elif rendered:
parts.append(concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms))
items.append(("a", concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)))
if span.pause_ms_after > 0:
n = int(sample_rate * span.pause_ms_after / 1000.0)
if n > 0:
parts.append(torch.zeros(n, dtype=torch.float32))
items.append(("s", n))
if not parts:
if not items:
return torch.zeros(0, dtype=torch.float32), 0.0
# Engines return (1, samples) per the TTSBackend contract while a bare
# zeros(n) is 1-D — mixing the two crashed the final concat (#897). So
# materialize inter-span silence AFTER the loop, matching the rendered
# audio's channel dims / dtype / device (same pattern as generation.py's
# _render_with_pauses). A silence-only chapter stays 1-D float32 as before.
ref = next((t for kind, t in items if kind == "a"), None)
parts: list = [
val if kind == "a"
else (torch.zeros(val, dtype=torch.float32) if ref is None
else torch.zeros(*ref.shape[:-1], val, dtype=ref.dtype, device=ref.device))
for kind, val in items
]
# Hard-concat spans + silences (crossfading silence would bleed the gap).
audio = parts[0] if len(parts) == 1 else concatenate_audio_chunks(parts, sample_rate, crossfade_ms=0)
return audio, audio.shape[-1] / float(sample_rate)
+32 -2
View File
@@ -183,14 +183,43 @@ def _safe_hard_cut(segment: str, max_chars: int) -> int:
return cut
def _normalize_chunk_shapes(chunks: list) -> list:
"""Coerce mixed-rank / mixed-channel chunks to one concat-compatible shape.
Engines return ``(1, samples)`` per the ``TTSBackend.generate`` contract,
but silence buffers and some model paths hand over bare ``(samples,)``
tensors ``torch.cat`` then dies with "Tensors must have same number of
dimensions" (#897). Promote lower-rank chunks with leading singleton dims
to the highest rank present, then broadcast singleton channel dims up to
the widest channel count (mono follows stereo). Rank-homogeneous,
channel-homogeneous input is returned untouched, so all-1-D / all-2-D
callers keep their exact output shape; a genuine channel conflict
(e.g. 2 vs 3 channels) still raises, which is the honest outcome.
"""
target = max(c.dim() for c in chunks)
if any(c.dim() != target for c in chunks):
promoted = []
for c in chunks:
while c.dim() < target:
c = c.unsqueeze(0)
promoted.append(c)
chunks = promoted
if target > 1:
lead = tuple(max(c.shape[i] for c in chunks) for i in range(target - 1))
chunks = [c if tuple(c.shape[:-1]) == lead else c.expand(*lead, -1)
for c in chunks]
return chunks
def concatenate_audio_chunks(chunks: list, sample_rate: int,
crossfade_ms: int = DEFAULT_CROSSFADE_MS):
"""Join per-chunk waveforms with a linear crossfade on the sample axis.
``chunks`` are torch tensors as returned by the engine (1-D, or N-D with
samples on the last axis matching what ``_render_with_pauses`` handles).
Crossfade overlap is clamped to the shorter neighbor; ``crossfade_ms=0``
is a hard concat.
Mixed ranks / mono-vs-multichannel chunks are normalized to one shape
first (#897), so no producer can crash the concat. Crossfade overlap is
clamped to the shorter neighbor; ``crossfade_ms=0`` is a hard concat.
"""
import torch
@@ -199,6 +228,7 @@ def concatenate_audio_chunks(chunks: list, sample_rate: int,
return torch.zeros(1, dtype=torch.float32)
if len(chunks) == 1:
return chunks[0]
chunks = _normalize_chunk_shapes(chunks)
crossfade_samples = int(sample_rate * crossfade_ms / 1000)
result = chunks[0]
+10
View File
@@ -187,6 +187,14 @@ def put_job(job_id: str, job: dict) -> None:
def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0, content_hash: str = "") -> None:
"""Persist dub job state to SQLite so it survives restarts. Uses UPSERT
on `id` so repeated saves in a session keep the latest snapshot.
language / language_code / content_hash only update when the incoming
value is non-empty: the ingest-time insert runs before the target
language is known (both columns ""), generation sets them on the job
dict, and a later save from a job that lost them (e.g. hydrated from an
old row) must not clobber the healed columns back to "". The frontend
keys history restore off language_code, so a frozen "" hid finished
tracks until the user re-picked a language.
"""
try:
segments = job.get("segments") or []
@@ -200,6 +208,8 @@ def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0,
filename=excluded.filename,
duration=excluded.duration,
segments_count=excluded.segments_count,
language=CASE WHEN excluded.language != '' THEN excluded.language ELSE dub_history.language END,
language_code=CASE WHEN excluded.language_code != '' THEN excluded.language_code ELSE dub_history.language_code END,
tracks=excluded.tracks,
job_data=excluded.job_data,
content_hash=CASE WHEN excluded.content_hash != '' THEN excluded.content_hash ELSE dub_history.content_hash END""",
+22 -2
View File
@@ -49,7 +49,7 @@ def _canon_value(field: str, value):
return value
def segment_fingerprint(seg: dict) -> str:
def segment_fingerprint(seg: dict, track_lang: str | None = None) -> str:
"""Deterministic hash of the inputs that actually affect TTS output.
Any change to `_GEN_INPUT_FIELDS` flips the hash and the segment becomes
@@ -61,8 +61,20 @@ def segment_fingerprint(seg: dict) -> str:
so a fingerprint computed from the generate request (server defaults
filled in) matches one recomputed later from the client's raw segment
state the root cause of #281's "1 edit re-dubs all N lines".
``track_lang`` (P1.3) is the TRACK's language code (`req.language_code`,
e.g. "es"). It is part of the fingerprint because the same segment text
renders different audio per language without it, a bn hash could
vouch for an es WAV on a multi-track job. It is only mixed in when
provided, so hashes computed by legacy callers (and hashes stored by
previous builds, which never carried a language) keep their old values;
a legacy hash therefore never matches a lang-scoped fingerprint and the
segment reads as stale the safe direction (one clean regen, never a
wrong-language splice).
"""
payload = {k: _canon_value(k, seg.get(k)) for k in _GEN_INPUT_FIELDS}
if track_lang:
payload["track_lang"] = str(track_lang)
blob = json.dumps(payload, sort_keys=True, ensure_ascii=False)
return hashlib.sha1(blob.encode("utf-8"), usedforsecurity=False).hexdigest()[:16]
@@ -120,6 +132,7 @@ def plan_incremental(
segments: list[dict],
*,
stored_hashes: dict[str, str] | None = None,
track_lang: str | None = None,
) -> dict:
"""Return `{stale, fresh, total, fingerprints}` where:
@@ -133,6 +146,13 @@ def plan_incremental(
`stored_hashes` may come from the caller's own bookkeeping (e.g. the
`dub_history.job_data["seg_hashes"]` we'll start writing in Phase 4.5).
When missing, every segment is considered stale (first run).
`track_lang` (P1.3) scopes the plan to ONE dub track: pass the track's
language code together with THAT language's stored hashes
(`job_data["seg_hashes_by_lang"][lang]`) so staleness is judged against
the active track, never against whatever language was generated last.
Must match the language the generate run hashed with, or every segment
reads stale (#281 parity class).
"""
stored = stored_hashes or {}
stale: list[str] = []
@@ -142,7 +162,7 @@ def plan_incremental(
sid = str(seg.get("id", ""))
if not sid:
continue
fp = segment_fingerprint(seg)
fp = segment_fingerprint(seg, track_lang=track_lang)
fingerprints[sid] = fp
prev = stored.get(sid)
if prev == fp:
+17 -3
View File
@@ -54,9 +54,12 @@ class LLMBackend(ABC):
def model_name(self) -> str: ...
@abstractmethod
def chat(self, *, system: str, user: str, timeout: Optional[float] = None) -> str:
def chat(self, *, system: str, user: str, timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
"""One-shot chat completion. Returns the assistant content string.
Raises on failure callers decide whether to fallback gracefully.
``temperature`` is only sent to the provider when set callers that
leave it None keep the provider default (existing behavior).
"""
@@ -132,32 +135,43 @@ class OpenAICompatBackend(LLMBackend):
self._client = OpenAI(max_retries=0, **kw)
return self._client
def chat(self, *, system: str, user: str, timeout: Optional[float] = None) -> str:
def chat(self, *, system: str, user: str, timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
return self.chat_messages(
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
],
timeout=timeout,
temperature=temperature,
)
def chat_messages(self, *, messages: list[dict], timeout: Optional[float] = None) -> str:
def chat_messages(self, *, messages: list[dict], timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
"""One-shot completion over a full message list.
Additive surface for callers that need structured few-shot turns
(dictation refinement, Wave 2.1) small local models pattern-match
and echo inline examples, so examples must arrive as prior chat
turns, not inside the system prompt.
``temperature`` is only forwarded when set (Cinematic/Autofit pin 0.2
the provider default of 1.0 makes local models drift and invent);
every other caller leaves it None and keeps the provider default.
"""
if timeout is None:
try:
timeout = float(os.environ.get("OMNIVOICE_LLM_TIMEOUT", "45"))
except ValueError:
timeout = 45.0
kw = {}
if temperature is not None:
kw["temperature"] = temperature
res = self._get_client().chat.completions.create(
model=self.model_name,
timeout=timeout,
messages=messages,
**kw,
)
return (res.choices[0].message.content or "").strip()
+84 -3
View File
@@ -24,10 +24,13 @@ users.
"""
from __future__ import annotations
import logging
import os
from dataclasses import dataclass
from typing import Optional
logger = logging.getLogger("omnivoice.llm_providers")
# Settings-store row names (non-secret overrides live in the plaintext table;
# keys live in the encrypted secret table under ``llm_key.<id>``).
_ACTIVE_PROVIDER_KEY = "llm.active_provider"
@@ -248,6 +251,19 @@ def is_configured(p: Provider) -> bool:
# ── Active provider selection ─────────────────────────────────────────────
def stored_active_provider_id() -> Optional[str]:
"""The user's explicitly-persisted selection ONLY — no env pin, no legacy
TRANSLATE_* fallback, no auto-detect.
``None`` means the user has never chosen a provider. This is what gates
save-activates in the settings router (#963): an explicit save may claim
the *empty* slot, but must never steal it from a made choice.
"""
from services import settings_store
stored = settings_store.get_text(_ACTIVE_PROVIDER_KEY)
return stored if stored and stored in _BY_ID else None
def active_provider_id() -> Optional[str]:
"""The provider Cinematic/Autofit should use.
@@ -255,12 +271,11 @@ def active_provider_id() -> Optional[str]:
configured provider None. Legacy ``TRANSLATE_BASE_URL`` users with no
explicit selection resolve to ``custom`` (its envs are TRANSLATE_*).
"""
from services import settings_store
env_pick = os.environ.get("LLM_DEFAULT_PROVIDER")
if env_pick and env_pick in _BY_ID:
return env_pick
stored = settings_store.get_text(_ACTIVE_PROVIDER_KEY)
if stored and stored in _BY_ID:
stored = stored_active_provider_id()
if stored:
return stored
# Legacy: a lone TRANSLATE_BASE_URL means the old single-endpoint setup.
if os.environ.get("TRANSLATE_BASE_URL"):
@@ -354,3 +369,69 @@ def describe(p: Provider) -> dict:
d["account_id"] = resolve_account_id(p)
d["account_from_env"] = bool(p.account_env and os.environ.get(p.account_env))
return d
# ── Legacy TRANSLATE_* prefs migration (#963) ──────────────────────────────
# prefs.json row → the custom-provider field it becomes.
_LEGACY_TRANSLATE_PREFS: tuple[tuple[str, str], ...] = (
("env.TRANSLATE_BASE_URL", "base_url"),
("env.TRANSLATE_MODEL", "model"),
("env.TRANSLATE_API_KEY", "api_key"),
)
def migrate_legacy_translate_prefs() -> bool:
"""Move the retired (≤v0.3.7) Translation-LLM panel's prefs rows into the
``custom`` provider's own settings-store rows, then delete them.
Those ``env.TRANSLATE_*`` rows in prefs.json are re-imported into
``os.environ`` on every launch (main.py), and a live ``TRANSLATE_BASE_URL``
makes :func:`active_provider_id` resolve to ``custom`` ahead of the stored
selection fallbacks silently hijacking the active slot on every restart
(issue #963, "Ollama works until I restart"). Must run BEFORE main.py's
prefsenv import so the rows never reach the environment.
Semantics:
* Each value is copied only where the store has no value yet a user's
later edit of the custom provider always wins over legacy leftovers.
* The prefs row is deleted afterwards either way, so it can never be
re-imported as env again (the migration is one-shot per row).
* Real process env vars are NEVER touched a shell/.env
``TRANSLATE_BASE_URL`` keeps its documented override behavior.
* A row whose store write fails is kept in prefs (it still works via the
env import this launch and the migration retries next launch).
Returns True if any prefs row was migrated/removed.
"""
from core import prefs
from services import settings_store
changed = False
for prefs_key, field in _LEGACY_TRANSLATE_PREFS:
try:
raw = prefs.get(prefs_key)
except Exception:
logger.exception("legacy TRANSLATE prefs read failed (%s)", prefs_key)
return changed
if raw is None:
continue
val = str(raw).strip()
try:
if val:
if field == "base_url":
if not settings_store.get_text(_BASE_URL_KEY + "custom"):
save_overrides("custom", base_url=val)
elif field == "model":
if not settings_store.get_text(_MODEL_KEY + "custom"):
save_overrides("custom", model=val)
else: # api_key — encrypted store, never overwrite an existing one
if not _key_in_store("custom"):
save_key("custom", val)
prefs.delete(prefs_key)
changed = True
except Exception:
# Store not ready (e.g. settings table missing) — keep the prefs
# row so the legacy env import still works and we retry next boot.
logger.exception("legacy TRANSLATE prefs migration failed (%s)", prefs_key)
return changed
+22 -3
View File
@@ -5,8 +5,10 @@ the Settings → LLM Skills panel can (a) toggle it and (b) route it to a
specific provider (a local Ollama/LM Studio vs a remote key) instead of
everything riding the one global active provider.
The five consumption points today:
The six consumption points today:
dub_translation api/routers/dub_translate.py (the Dub tab's direct
"LLM" translation engine; provider=openai branch)
cinematic_translation services/translator.py (Cinematic + Autofit
REFLECT/ADAPT rewrite; dub_translate quality gate)
slot_fitting services/speech_rate.py (trim/expand a line to its
@@ -68,8 +70,9 @@ def _skill(sid: str) -> LLMSkill:
# Display order in the settings panel: the dub pipeline first (translation →
# fit → glossary → direction), then dictation.
# refine → fit → glossary → direction), then dictation.
_SKILLS: tuple[LLMSkill, ...] = (
_skill("dub_translation"),
_skill("cinematic_translation"),
_skill("slot_fitting"),
_skill("glossary_extract"),
@@ -239,8 +242,24 @@ def resolve_skill_client(skill_id: str) -> Optional[SkillClient]:
# skill's wall-clock budget (the cinematic pass budget, the glossary call
# timeout) from inside one request. Fail fast — the per-call timeout and the
# pass-level budget are the only bounds we want. Mirrors OpenAICompatBackend.
#
# #959 class guard: OpenAI() eagerly builds its httpx client, which can
# raise AT CONSTRUCTION for environment-shaped reasons — the reported one
# is httpx's ImportError under ALL_PROXY/HTTPS_PROXY=socks5:// without
# socksio; a malformed proxy URL or broken cert bundle fails the same way.
# The contract here is already "None == LLM unavailable, degrade" — a bad
# proxy env must degrade the skill, never 500 the calling feature.
try:
client = OpenAI(max_retries=0, **kw)
except Exception as exc:
logger.warning(
"LLM client construction failed for skill %s (provider %s): %s"
"treating the skill as unavailable.",
skill_id, res.provider.id, exc,
)
return None
return SkillClient(
client=OpenAI(max_retries=0, **kw),
client=client,
model=llm_providers.resolve_model(res.provider),
provider_id=res.provider.id,
timeout=_default_timeout(),
+37 -4
View File
@@ -1022,6 +1022,22 @@ async def get_model():
return model
def _checkpoint_in_local_cache(checkpoint: str) -> bool:
"""True when ``checkpoint`` is loadable with NO network: an existing local
directory, or a COMPLETE HF cache snapshot. ``snapshot_download(...,
local_files_only=True)`` never constructs an HTTP session, so a broken
proxy env (#959: ``ALL_PROXY``/``HTTPS_PROXY=socks5://`` without socksio)
can't false-negative this probe. Never raises."""
if os.path.isdir(checkpoint):
return True
try:
from huggingface_hub import snapshot_download
snapshot_download(checkpoint, local_files_only=True)
return True
except Exception:
return False
async def preload_model():
"""Background model warm-up — call from lifespan startup.
@@ -1042,10 +1058,27 @@ async def preload_model():
try:
from huggingface_hub import model_info
model_info(checkpoint, timeout=5)
except Exception:
# Model not downloaded yet — skip preload
logger.info("Preload skipped: %s not available locally.", checkpoint)
return
except Exception as probe_err:
# The probe failing does NOT mean the model isn't installed — it
# means the Hub API wasn't reachable from this process. The #959
# class: under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio,
# hf_hub's get_session() raises ImportError AT CLIENT CONSTRUCTION;
# same story for offline mode, DNS, or firewall failures. Fall back
# to a cache-only probe (no HTTP session involved) and warm up
# anyway when the model is locally present, instead of silently
# skipping and letting the first /generate eat the full load.
if not _checkpoint_in_local_cache(checkpoint):
logger.info(
"Preload skipped: %s not available locally (network probe "
"failed: %s: %s).",
checkpoint, type(probe_err).__name__, probe_err,
)
return
logger.warning(
"Network probe for %s failed (%s: %s) — model found in the "
"local cache; warming up from cache.",
checkpoint, type(probe_err).__name__, probe_err,
)
logger.info("Preloading TTS model in background…")
_last_used = time.time()
+111 -7
View File
@@ -10,8 +10,8 @@ plays the moment the video begins and everything feels desynchronised.
``snap_segment_starts`` post-processes segments against the actual audio
(ideally the Demucs-isolated vocals track, which the dub pipeline already
produces): for each segment it scans the waveform inside ``[start, end]``
for the first frame whose RMS rises above an adaptive threshold and moves
``start`` forward to just before that onset.
for the first *sustained* rise of frame RMS above an adaptive threshold
and moves ``start`` forward to just before that onset.
Design constraints:
@@ -23,6 +23,27 @@ Design constraints:
(no frame above the absolute floor) are left untouched.
* **Pure NumPy.** No model, no platform-specific code identical
behaviour on macOS / Windows / Linux, trivially unit-testable.
Robustness against non-speech onsets (#963): a field report showed dubbed
lines starting seconds off because "when a noise is heard (a sigh or
footsteps), it's interpreted as the start of the conversation". Three
guards address that class of failure:
* **Sustained energy.** A frame only counts as an onset when the energy
stays up for a speech-like duration (``SUSTAIN_MIN_S`` within the
following ``SUSTAIN_WINDOW_S``). Footsteps/door thuds/clicks light up
one or two 20 ms frames and die; syllables keep the energy up.
* **Bounded snap distance.** Shifts beyond ``MAX_SNAP_S`` are only
trusted when everything being skipped is (near-)silence the genuine
#280 whisper start-stretch, where Demucs removed the leading music and
left real silence on the vocals track. Jumping far over *audible*
content (e.g. quiet speech sitting under the relative threshold) would
play the dub seconds late, so it is refused.
* **Source-aware.** Snapping only runs on a separated vocals track
(``separated_vocals=True``). On mixed/original audio Demucs skipped
or failed music, ambience and room tone are all legitimate sustained
energy, so any detected "onset" is as likely the score as the speaker;
whisper's own timestamps beat a confidently wrong snap.
"""
from __future__ import annotations
@@ -51,6 +72,23 @@ RELATIVE_THRESHOLD = 0.10
# the whole window is treated as silence and left alone (we'd only be
# snapping to noise).
ABS_RMS_FLOOR = 1e-3
# An onset must be *sustained* to count as speech (#963): within the
# SUSTAIN_WINDOW_S that follows a candidate frame, at least SUSTAIN_MIN_S
# worth of frames must also sit above the threshold. A ~100 ms footstep
# burst fails this; real speech (syllables every few hundred ms) passes.
SUSTAIN_WINDOW_S = 0.30
SUSTAIN_MIN_S = 0.16
# Snaps larger than this are only trusted when the skipped span is
# (near-)silence — see _region_mostly_silent (#963).
MAX_SNAP_S = 1.5
# The skipped span counts as "mostly silent" when at most this fraction of
# its frames is audible. Non-zero so an isolated transient bleeding through
# separation (a footstep) doesn't block a genuine long silence-trim…
SKIPPED_AUDIBLE_FRAC = 0.10
# …where "audible" = above max(ABS_RMS_FLOOR, this fraction of the span's
# own peak); the relative term keeps a slightly raised residual noise floor
# from reading as content.
SKIPPED_FLOOR_PEAK_FRAC = 0.02
def _frame_rms(x: np.ndarray, frame_len: int) -> np.ndarray:
@@ -70,6 +108,15 @@ def detect_speech_onset(
) -> float | None:
"""Return the absolute time (s) of the first speech-like frame inside
``[start_s, end_s]``, or ``None`` when the window is empty / silent.
"Speech-like" requires *sustained* energy (#963): within the
``SUSTAIN_WINDOW_S`` look-ahead after a candidate frame, at least
``SUSTAIN_MIN_S`` worth of frames must also exceed the threshold.
Short broadband transients footsteps, door thuds, mouse clicks
light up one or two 20 ms frames and then die, so they no longer read
as "the conversation started here"; real speech keeps the energy up
across syllables. A candidate too close to the window's end to prove
sustain is rejected (conservative: the ASR timestamp stands).
"""
if sr <= 0 or end_s <= start_s:
return None
@@ -86,10 +133,21 @@ def detect_speech_onset(
if peak < ABS_RMS_FLOOR:
return None # whole window is effectively silent
threshold = max(RELATIVE_THRESHOLD * peak, ABS_RMS_FLOOR)
above = np.nonzero(rms >= threshold)[0]
if above.size == 0:
above = rms >= threshold
candidates = np.nonzero(above)[0]
if candidates.size == 0:
return None
return start_s + float(above[0]) * (frame_len / sr)
frame_s = frame_len / sr
win_frames = max(1, int(round(SUSTAIN_WINDOW_S / frame_s)))
need_frames = max(1, int(round(SUSTAIN_MIN_S / frame_s)))
# counts[k] = above-threshold frames within rms[c : c + win_frames]
# for candidate c — O(n) via a cumulative sum, no per-candidate scan.
cum = np.concatenate(([0], np.cumsum(above)))
counts = cum[np.minimum(candidates + win_frames, above.size)] - cum[candidates]
sustained = candidates[counts >= need_frames]
if sustained.size == 0:
return None # only transient bursts in this window
return start_s + float(sustained[0]) * frame_s
# Hysteresis for full-track onset listing: after a frame crosses the
@@ -138,21 +196,60 @@ def detect_speech_onsets(audio: np.ndarray, sr: int) -> list[float]:
return onsets
def _region_mostly_silent(
audio: np.ndarray,
sr: int,
start_s: float,
end_s: float,
) -> bool:
"""True when ``[start_s, end_s]`` contains (almost) no audible content.
Gates long snaps (> ``MAX_SNAP_S``, #963): jumping far forward is only
trustworthy when everything being skipped is silence the genuine
whisper start-stretch of #280, where Demucs stripped the leading music
and left real silence on the vocals track. A small fraction of audible
frames is tolerated so an isolated transient bleeding through
separation (a footstep) doesn't block the trim; *sustained* audible
content e.g. quiet speech sitting below the relative onset
threshold does block it, because skipping past it would desync the
dub by the full jump.
"""
i0 = max(0, int(start_s * sr))
i1 = min(len(audio), int(end_s * sr))
if i1 <= i0:
return True
rms = _frame_rms(audio[i0:i1], max(1, int(FRAME_S * sr)))
if rms.size == 0:
return True
floor = max(ABS_RMS_FLOOR, SKIPPED_FLOOR_PEAK_FRAC * float(rms.max()))
return float((rms >= floor).mean()) <= SKIPPED_AUDIBLE_FRAC
def snap_segment_starts(
segments: Sequence[dict],
audio: np.ndarray,
sr: int,
*,
min_shift_s: float = MIN_SHIFT_S,
separated_vocals: bool = True,
) -> int:
"""Snap each segment's ``start`` forward to the actual speech onset.
Mutates the segment dicts in place (the shape the dub pipeline passes
around). Returns the number of segments adjusted.
``audio`` should be mono float; the Demucs vocals track gives the best
signal but the mixed track still beats nothing.
``audio`` should be the mono-float **separated vocals** track. When the
caller only has mixed/original audio (Demucs skipped or failed), pass
``separated_vocals=False``: snapping is then disabled entirely (#963) —
on a mixed track music, ambience and footsteps are all sustained energy,
so a detected "onset" is as likely the score as the speaker, and
whisper's own timestamps beat a confidently wrong snap.
"""
if not separated_vocals:
logger.info(
"onset-align: skipped — audio is not a separated vocals track "
"(Demucs unavailable/failed); keeping ASR timestamps as-is")
return 0
if sr <= 0 or audio is None or len(audio) == 0:
return 0
if audio.ndim > 1:
@@ -174,6 +271,13 @@ def snap_segment_starts(
shift = new_start - start
if shift < min_shift_s:
continue
if shift > MAX_SNAP_S and not _region_mostly_silent(audio, sr, start, onset):
# Long jump over audible content (#963): the "onset" is more
# likely a louder late event than the true start — quiet speech
# under the relative threshold would be skipped wholesale and
# the dub would play seconds LATE. Bounded corrections are fine;
# unbounded ones only over true silence (the #280 case).
continue
# Preserve a minimum playable duration.
new_start = min(new_start, end - MIN_SEG_DUR_S)
if new_start - start < min_shift_s:
+33 -6
View File
@@ -532,14 +532,41 @@ def assign_speakers_from_turns(
return segments
def assign_speakers_heuristic(segments: List[dict]) -> List[dict]:
"""Two-speaker alternation based on silence gaps."""
current = 1
def assign_speakers_heuristic(
segments: List[dict], num_speakers: Optional[int] = None
) -> List[dict]:
"""Silence-gap speaker assignment (used when no diarization model runs).
Base signal: a gap > SPEAKER_GAP seconds between consecutive segments is
treated as a speaker change. Without a ``num_speakers`` hint this keeps
the legacy behavior alternate between exactly two labels. With a hint:
* ``num_speakers=1`` every segment gets ``"Speaker 1"``.
* ``num_speakers>=2`` labels round-robin across N speakers at each
gap boundary, so the user's requested count is represented instead of
being silently capped at 2.
Limits (be honest with callers): this honors the *count*, not voice
identity. The rotation order is arbitrary (a returning speaker gets the
next label in the cycle, not their own), rapid exchanges with no
> SPEAKER_GAP pause still collapse into one label, and N is an upper
bound audio with fewer gap boundaries than N yields fewer labels.
Real per-speaker attribution needs pyannote (or an inline-diarizing ASR
backend); callers should warn the user accordingly (see dub_core).
Invalid hints (non-int, < 1) fall back to the legacy two-speaker cycle.
"""
try:
n = int(num_speakers) if num_speakers is not None else 2
except (TypeError, ValueError):
n = 2
if n < 1:
n = 2
current = 0 # zero-based rotation index; rendered one-based below
last_end = 0.0
for i, s in enumerate(segments):
if i > 0 and (s["start"] - last_end) > SPEAKER_GAP:
current = 2 if current == 1 else 1
s["speaker_id"] = f"Speaker {current}"
if i > 0 and n > 1 and (s["start"] - last_end) > SPEAKER_GAP:
current = (current + 1) % n
s["speaker_id"] = f"Speaker {current + 1}"
last_end = s["end"]
return segments
+101 -13
View File
@@ -42,11 +42,25 @@ IDEAL_REF_DURATION_S = 8.0 # target window — long enough for prosody, short e
# is the empirical floor below which our zero-shot clone gets unstable.
MIN_SEGMENT_REF_DURATION_S = 3.0
# Clone-purity guards (speaker-hint fix): a per-speaker reference cut from
# mislabeled or boundary-adjacent audio mixes two people's voices and the
# resulting clone sounds "made up".
# * A slice below MIN_SLICE_DURATION_S is too short to be a reliable
# single-speaker sample (and diarization boundary jitter dominates it).
# * A slice whose edges come within ADJACENT_TURN_GUARD_S of a *different*
# speaker's turn risks bleeding that speaker's audio across the imprecise
# boundary — deprioritized (scoring preference, not a hard filter, so
# extraction still succeeds on dense dialogue).
MIN_SLICE_DURATION_S = 1.5
ADJACENT_TURN_GUARD_S = 0.3
def extract_speaker_clones(
vocals_path: str,
segments: list[dict],
out_dir: str,
*,
labels_source: str | None = None,
) -> dict[str, dict]:
"""Build a per-speaker reference sample from `vocals_path` + `segments`.
@@ -63,7 +77,20 @@ def extract_speaker_clones(
Speakers whose segments total < MIN_REF_DURATION_S are skipped we'd
rather fall back to the default TTS voice than ship a bad clone.
``labels_source`` records where the ``speaker_id`` labels came from
(``"pyannote"`` | ``"turns"`` | ``"heuristic"``; ``None`` = unknown,
treated as trusted for backward compatibility). ``"heuristic"`` labels
are silence-gap *estimates*, not voice identity a reference cut from
them routinely concatenates two people's audio, so extraction is skipped
entirely (the caller warns the user and falls back to the default voice).
"""
if labels_source == "heuristic":
logger.info(
"speaker_clone: skipping auto-clone extraction — speaker labels "
"are gap-based heuristic estimates, not voice identity"
)
return {}
if not vocals_path or not os.path.exists(vocals_path):
logger.info("speaker_clone: no vocals track at %s; skipping", vocals_path)
return {}
@@ -88,7 +115,12 @@ def extract_speaker_clones(
out: dict[str, dict] = {}
for speaker_id, items in by_speaker.items():
chosen = _pick_reference_slices(items)
chosen = _pick_reference_slices(
items,
speaker_id=speaker_id,
all_segments=segments,
labels_source=labels_source,
)
if not chosen:
logger.info(
"speaker_clone: %s has <%ss of usable audio; will fall back to default voice",
@@ -194,31 +226,87 @@ def extract_segment_refs(
# ── Internals ───────────────────────────────────────────────────────────────
def _pick_reference_slices(items: list[tuple[int, dict]]) -> list[tuple[int, dict]]:
def _adjacent_to_other_speaker(
seg: dict, speaker_id: str, all_segments: list[dict] | None
) -> bool:
"""True when `seg`'s edges come within ADJACENT_TURN_GUARD_S of (or
overlap) a segment attributed to a *different* speaker a boundary where
imprecise diarization timestamps risk bleeding the other voice into the
reference slice."""
if not all_segments:
return False
s0 = float(seg.get("start", 0.0))
s1 = float(seg.get("end", 0.0))
for other in all_segments:
if other is seg:
continue
if (other.get("speaker_id") or "Speaker 1") == speaker_id:
continue
o0 = float(other.get("start", 0.0))
o1 = float(other.get("end", 0.0))
# Signed gap between the two spans; negative = overlap.
if max(o0 - s1, s0 - o1) < ADJACENT_TURN_GUARD_S:
return True
return False
def _pick_reference_slices(
items: list[tuple[int, dict]],
*,
speaker_id: str | None = None,
all_segments: list[dict] | None = None,
labels_source: str | None = None,
) -> list[tuple[int, dict]]:
"""Select the subset of a speaker's segments to use as reference audio.
Strategy: take the single longest segment; if it's short, accumulate the
next longest ones in original order until we clear IDEAL_REF_DURATION_S.
Cap at MAX_REF_DURATION_S. Return [] if we can't reach MIN_REF_DURATION_S.
Strategy: rank candidates clean-first (not temporally adjacent to a
different speaker's turn — see ``_adjacent_to_other_speaker``), longest
first within each tier, and accumulate until IDEAL_REF_DURATION_S is
cleared. Adjacency is a scoring preference, NOT a hard filter on dense
dialogue where every slice borders another speaker, extraction still
succeeds using the adjacent ones. Two hard guards protect clone purity:
* slices shorter than MIN_SLICE_DURATION_S are rejected outright
(boundary jitter dominates them, so they're the likeliest to carry a
second speaker's audio);
* ``labels_source="heuristic"`` returns [] gap-based labels are not
voice identity, so no slice of them is safe to clone from.
Cap at MAX_REF_DURATION_S. Return [] if we can't reach
MIN_REF_DURATION_S. When ``all_segments``/``speaker_id`` are not
provided (legacy callers), adjacency scoring degrades to duration-only
the pre-guard behavior.
"""
if not items:
return []
if labels_source == "heuristic":
return []
if speaker_id is None:
speaker_id = items[0][1].get("speaker_id") or "Speaker 1"
# Longest-first candidates. Keep original indices so we can preserve order.
by_dur = sorted(
def _dur(pair) -> float:
return max(0.0, float(pair[1].get("end", 0.0)) - float(pair[1].get("start", 0.0)))
# Rank: clean (non-adjacent) before adjacent, longest first within each
# tier. Keep original indices so we can restore transcript order below.
ranked = sorted(
items,
key=lambda pair: (pair[1].get("end", 0.0) - pair[1].get("start", 0.0)),
reverse=True,
key=lambda pair: (
_adjacent_to_other_speaker(pair[1], speaker_id, all_segments),
-_dur(pair),
),
)
picked: list[tuple[int, dict]] = []
total = 0.0
for idx, seg in by_dur:
dur = max(0.0, float(seg.get("end", 0.0)) - float(seg.get("start", 0.0)))
if dur <= 0.0:
for idx, seg in ranked:
dur = _dur((idx, seg))
if dur < MIN_SLICE_DURATION_S:
continue
if total + dur > MAX_REF_DURATION_S and picked:
break
# Ranking is no longer duration-monotonic, so a later (shorter or
# adjacent) slice may still fit — skip, don't stop.
continue
picked.append((idx, seg))
total += dur
if total >= IDEAL_REF_DURATION_S:
+60 -10
View File
@@ -18,6 +18,10 @@ import logging
from typing import Iterable, Optional
from services.llm_backend import get_active_llm_backend, OffBackend
# Shared LLM-output divergence guard (length window + target-script +
# critique-echo). Lives in translator; translator never imports this module,
# so there is no import cycle.
from services.translator import refine_output_ok
logger = logging.getLogger("omnivoice.speech_rate")
@@ -91,9 +95,15 @@ _EXPAND_PROMPT = """\
You are a dubbing writer. The user will give you a translated line + the exact
time slot it must fit. The current line is TOO SHORT add natural filler or
gently flesh out the thought while keeping the meaning the same. Aim for a
reading duration that matches the slot.
reading duration that matches the slot. Never invent new information, names,
or dialogue that is not already in the line; do not more than double the line.
Reply with ONLY the new line. No quotes, no commentary."""
# Below this predicted rate ratio a line can never honestly fill its slot —
# any LLM "expansion" that far would be fabricated dialogue. Skip the expand
# pass entirely and keep the short line (slot-aware TTS absorbs the silence).
_MIN_EXPANDABLE_RATIO = 0.15
def adjust_for_slot(
text: str,
@@ -107,17 +117,36 @@ def adjust_for_slot(
Falls back to the input text if the LLM is off or the loop gives up.
``strict`` (Autofit mode) caps the accepted upper bound at 1.0 instead of
``TOL_HIGH`` i.e. the line must fit *within* the slot, never overrun it
so the target-language reading time can't exceed the segment and push the
video timing out. A too-short line is still accepted down to ``TOL_LOW`` (we
don't pad just to fill silence). Best-effort: after ``MAX_ATTEMPTS`` it
returns the closest candidate seen, so a stubborn line degrades gracefully.
``strict`` (Autofit mode) changes exactly one thing: the accepted upper
bound is 1.0 instead of ``TOL_HIGH`` the line must fit *within* the
slot, never overrun it so the target-language reading time can't exceed
the segment and push the video timing out. Lines under ``TOL_LOW`` still
go through the LLM expand pass in strict mode too (same as loose mode);
padding is bounded by the divergence guard below, and a line under
``_MIN_EXPANDABLE_RATIO`` is never expanded at all it could only "fill"
the slot with fabricated dialogue, so it stays short. Best-effort: after
``MAX_ATTEMPTS`` it returns the closest candidate seen, so a stubborn line
degrades gracefully.
Every LLM reply is validated with ``translator.refine_output_ok`` against
the ORIGINAL input ``text`` (not the previous candidate divergence
compounds across attempts otherwise). A reply that fails the guard is
discarded: the attempt is burned, ``current``/``best`` stay put, and if
nothing valid ever came back the input text is returned with
``error="fit-diverged"`` a hallucinating model can no longer invent the
dub line (v0.3.9 field report).
"""
tol_high = 1.0 if strict else TOL_HIGH
initial_ratio = rate_ratio(text, slot_seconds, target_lang)
if TOL_LOW <= initial_ratio <= tol_high:
return {"text": text, "rate_ratio": initial_ratio, "attempts": 0}
if initial_ratio < _MIN_EXPANDABLE_RATIO:
return {
"text": text,
"rate_ratio": initial_ratio,
"attempts": 0,
"error": "fit-skip-short",
}
from services import llm_skills
# `active=` forwards this module's (monkeypatch-able) name so the
@@ -133,6 +162,7 @@ def adjust_for_slot(
current = text
best = (current, initial_ratio)
diverged = False
for attempt in range(1, MAX_ATTEMPTS + 1):
r = rate_ratio(current, slot_seconds, target_lang)
if TOL_LOW <= r <= tol_high:
@@ -150,23 +180,43 @@ def adjust_for_slot(
user_lines.append(f"Source line (for meaning): {source_text}")
try:
next_text = llm.chat(system=system, user="\n".join(user_lines))
next_text = llm.chat(
system=system, user="\n".join(user_lines),
temperature=0.2, # pinned like the Fast path — default 1.0 drifts/invents
)
except Exception as e:
logger.warning("speech-rate attempt %d failed: %s", attempt, e)
return {"text": best[0], "rate_ratio": best[1], "attempts": attempt - 1, "error": str(e)}
if next_text and next_text.strip():
current = next_text.strip()
candidate = next_text.strip()
# Divergence guard — validate against the ORIGINAL text, not
# `current`: each accepted reply becomes the next prompt's input,
# so per-step checks would let drift compound across attempts.
ok, reason = refine_output_ok(text, candidate, target_lang)
if not ok:
diverged = True
logger.warning(
"speech-rate attempt %d rejected (%s) — discarding candidate",
attempt, reason,
)
continue # attempt burned; current/best untouched
current = candidate
new_r = rate_ratio(current, slot_seconds, target_lang)
# Keep the best candidate seen so far in case we exhaust retries.
if abs(new_r - 1.0) < abs(best[1] - 1.0):
best = (current, new_r)
return {
out = {
"text": best[0],
"rate_ratio": best[1],
"attempts": MAX_ATTEMPTS,
}
# Every usable reply diverged and the input text survived unchanged —
# surface it on the row (rate_error in dub_translate, like fit-budget).
if diverged and best[0] == text:
out["error"] = "fit-diverged"
return out
def adjust_many(pairs: Iterable[tuple[str, float, str, Optional[str]]]) -> list[dict]:
+35 -5
View File
@@ -92,9 +92,11 @@ REGISTRY: dict[str, dict] = {
"category": "llm",
"needs_key": True,
"notes": (
"Any OpenAI-compatible endpoint: GPT-4/5 (OpenAI), Claude (via OpenRouter), "
"Gemini (OpenAI-compat mode), DeepSeek, Qwen, Ollama, LM Studio. "
"Set TRANSLATE_BASE_URL + TRANSLATE_API_KEY + TRANSLATE_MODEL."
"Uses the LLM provider you configure in Settings → LLM Providers "
"(route it via the 'Dub translation' skill in Settings → LLM Skills): "
"GPT (OpenAI), Claude (via OpenRouter), Gemini, DeepSeek, Qwen, "
"Ollama, LM Studio. Power-user env override: TRANSLATE_BASE_URL + "
"TRANSLATE_API_KEY + TRANSLATE_MODEL."
),
},
}
@@ -137,17 +139,45 @@ def install_command(engine: "str | dict | None") -> str | None:
return f"uv pip install {pkg}" if pkg else None
def _llm_configured() -> tuple[bool, "str | None"]:
"""Whether the LLM translation engine has something to call, and via what.
Resolution mirrors the translate-time path in dub_translate.py: the
"dub_translation" LLM skill (per-skill override active provider from
Settings LLM Providers) first, then the TRANSLATE_* env override. Lets
the Engine dropdown say "ready via <provider>" / "needs setup" up front
instead of a per-segment failure after the user clicks Translate.
"""
try:
from services import llm_skills
res = llm_skills.resolve_skill("dub_translation")
if res.ready and res.provider is not None:
return True, res.provider.display_name
except Exception: # noqa: BLE001 — a probe must never break list_engines()
logger.debug("dub_translation skill probe failed", exc_info=True)
if os.environ.get("TRANSLATE_BASE_URL") or os.environ.get("TRANSLATE_API_KEY"):
return True, "env"
return False, None
def list_engines() -> list[dict]:
"""Return a UI-ready list with per-engine availability stamped in."""
out = []
for e in REGISTRY.values():
installed, reason = _probe(e)
out.append({
entry = {
**e,
"installed": installed,
"availability_reason": reason,
"install_command": install_command(e),
})
}
# LLM engines additionally need a provider/key — surface configured-ness
# so the UI can distinguish "importable" from "actually ready to call".
if e.get("category") == "llm":
configured, via = _llm_configured()
entry["configured"] = configured
entry["configured_via"] = via
out.append(entry)
return out
+127 -17
View File
@@ -59,8 +59,9 @@ _ADAPT_PROMPT = """\
You are a cinematic dubbing writer. Rewrite the literal translation using the
editor's critique so it sounds natural, in-character, and fits the speaker's
time slot. Keep meaning faithful but prefer native idiom over word-for-word
accuracy. The output MUST be written in the same target language and script
as the literal translation never switch language or transliterate.
accuracy. Never introduce facts, names, or dialogue that are not present in
the source line. The output MUST be written in the same target language and
script as the literal translation never switch language or transliterate.
Reply ONLY with the adapted translation no quotes, no headers, no code
fences, no commentary."""
@@ -93,6 +94,107 @@ def _looks_like_target_script(text: str, code: str, threshold: float = 0.5) -> b
return (inside / len(letters)) >= threshold
# ── Divergence guard (shared with speech_rate's Autofit fit pass) ────────────
# For every Latin-script target `_looks_like_target_script` passes ANY text
# unconditionally (no `_SCRIPT_RANGES` entry), so it was the only — and for
# es/de/fr/… a no-op — gate on the ADAPT/fit LLM output. These checks close
# that gap for the whole class: runaway length (hallucinated dialogue,
# refusals, commentary) and the REFLECT critique echoed back as the "line".
_SHORT_REF_CHARS = 20 # below this, a length *ratio* is meaningless
_SHORT_REF_ABS_SLACK = 120 # …use an absolute cap instead: ref + this many chars
def _refine_ratio_bounds() -> tuple[float, float]:
"""Accepted ``len(candidate)/len(reference)`` window for LLM refine output.
Anything outside is treated as divergence and the caller degrades to its
input text. Defaults [0.4, 2.5]; env-tunable like the cinematic budget."""
try:
lo = float(os.environ.get("OMNIVOICE_REFINE_RATIO_MIN", "0.4"))
except ValueError:
lo = 0.4
try:
hi = float(os.environ.get("OMNIVOICE_REFINE_RATIO_MAX", "2.5"))
except ValueError:
hi = 2.5
return lo, hi
def _norm_overlap_text(s: str) -> str:
return " ".join(s.lower().split())
def _echoes_critique(candidate: str, critique: str) -> bool:
"""True when the "adaptation" is really the REFLECT critique leaking through.
Deterministic on purpose (no fuzzy matching): exact match after
case/whitespace normalization; containment the full critique inside the
candidate always counts, the candidate inside the critique only when it
covers most of it (critiques legitimately quote short phrases from the
line); or >0.8 token-set overlap.
"""
c = _norm_overlap_text(candidate)
k = _norm_overlap_text(critique)
if not c or not k:
return False
if c == k:
return True
if k in c: # critique embedded in the output
return True
if c in k and len(c) >= 0.6 * len(k): # output ≈ a big chunk of the critique
return True
ct, kt = set(c.split()), set(k.split())
union = ct | kt
return bool(union) and len(ct & kt) / len(union) > 0.8
def refine_output_ok(
reference: str,
candidate: str,
target_lang: str,
*,
critique: str | None = None,
max_ratio: float | None = None,
) -> tuple[bool, str | None]:
"""Sanity-check one LLM refine output against the text it was rewriting.
Shared by the Cinematic ADAPT step here and by ``speech_rate``'s Autofit
fit pass (speech_rate imports this; translator never imports speech_rate,
so there is no cycle). Returns ``(ok, reason)`` ``reason`` is ``None``
when ok, otherwise a short machine-readable tag for logs/error mapping.
Checks, in order:
script candidate must look like the target language's script
(``_looks_like_target_script``; Latin-script targets pass, as before);
length ``len(candidate)/len(reference)`` must sit inside
[``OMNIVOICE_REFINE_RATIO_MIN``, ``OMNIVOICE_REFINE_RATIO_MAX``]
(default 0.42.5; ``max_ratio`` overrides the upper bound). References
shorter than ~20 chars use an absolute cap (reference + 120 chars)
instead a two-word line legitimately doubles or halves;
critique echo the candidate must not be the critique itself.
"""
cand = (candidate or "").strip()
ref = (reference or "").strip()
if not cand:
return False, "empty"
if not _looks_like_target_script(cand, target_lang):
return False, f"wrong-script:{target_lang}"
lo, hi = _refine_ratio_bounds()
if max_ratio is not None:
hi = max_ratio
if ref:
if len(ref) < _SHORT_REF_CHARS:
if len(cand) > len(ref) + _SHORT_REF_ABS_SLACK:
return False, f"length-abs:{len(cand)}>{len(ref)}+{_SHORT_REF_ABS_SLACK}"
else:
ratio = len(cand) / len(ref)
if not (lo <= ratio <= hi):
return False, f"length-ratio:{ratio:.2f}"
if critique and _echoes_critique(cand, critique):
return False, "critique-echo"
return True, None
# The LLM Skills registry entry this pipeline resolves through — lets the
# user disable Cinematic/Autofit's LLM use or route it to a specific provider
# (Settings → LLM Skills) independently of the other LLM features.
@@ -173,6 +275,7 @@ def _chat(client, *, system: str, user: str) -> str:
res = client.chat.completions.create(
model=_llm_model(),
timeout=_llm_timeout(),
temperature=0.2, # pinned like the Fast path — default 1.0 drifts/invents
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
@@ -274,21 +377,28 @@ def cinematic_refine_sync(
}
final = (adapted or "").strip() or literal_text
# Refuse adaptations that drifted off the target script (e.g. local LLM
# rewrote a Devanagari line in Latin/German). Caller still gets the
# critique so the UI can show what happened, but the live text falls
# back to the literal translation rather than corrupting the dub.
if final is not literal_text and not _looks_like_target_script(final, target_lang):
logger.warning(
"cinematic adapt produced wrong-script output for %s — falling back to literal",
target_lang,
)
return {
"text": literal_text,
"literal": literal_text,
"critique": critique,
"error": f"adapt-wrong-script:{target_lang}",
}
# Refuse adaptations that diverged from the line they were rewriting:
# wrong script (e.g. a local LLM rewrote a Devanagari line in
# Latin/German), runaway length (hallucinated dialogue, refusals,
# commentary — the script check alone passes ANY text for Latin-script
# targets), or the critique echoed back as the "adaptation". Caller still
# gets the critique so the UI can show what happened, but the live text
# falls back to the literal translation rather than corrupting the dub.
if final is not literal_text:
ok, reason = refine_output_ok(literal_text, final, target_lang, critique=critique)
if not ok:
logger.warning(
"cinematic adapt diverged for %s (%s) — falling back to literal",
target_lang, reason,
)
wrong_script = (reason or "").startswith("wrong-script")
return {
"text": literal_text,
"literal": literal_text,
"critique": critique,
"error": (f"adapt-wrong-script:{target_lang}" if wrong_script
else "adapt-diverged"),
}
return {
"text": final,
"literal": literal_text,
+7 -1
View File
@@ -1280,7 +1280,13 @@ class _LazyRegistry(dict):
# effect on every list_backends() call — we keep iteration light
# and let the caller's __getitem__ trigger the import.
seen: set[str] = set()
for k in dict.__iter__(self):
# Snapshot the live keys before yielding. A concurrent thread's lazy
# __getitem__ inserts into self (self[key] = cls), and list_backends()
# runs in a FastAPI threadpool — so holding a *live* dict iterator open
# across the per-engine is_available() probes would raise
# "dictionary changed size during iteration". list() consumes the
# iterator atomically under the GIL, closing that window.
for k in list(dict.__iter__(self)):
seen.add(k)
yield k
for k in _LAZY_REGISTRY:
+1 -1
View File
@@ -16,7 +16,7 @@
},
"frontend": {
"name": "omnivoice-studio",
"version": "0.3.9",
"version": "0.3.11",
"dependencies": {
"@fontsource-variable/inter": "^5.2.8",
"@fontsource-variable/source-serif-4": "^5.2.9",
+85 -25
View File
@@ -5,13 +5,28 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
## Prerequisites
### Using the AppImage or .deb
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
a Tauri / WebKitGTK app.
- **~10 GB free disk** for the app, its Python environment, and model weights.
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
by the app itself on first launch. No toolchain needed.
### Building from source
Everything above, plus the toolchain:
- **git**`sudo apt install git` (Debian/Ubuntu), `sudo dnf install git` (Fedora), or `sudo pacman -S git` (Arch).
- **curl** — usually preinstalled; used by the Bun and rustup install one-liners below.
- **Python 3.11+** — typically `sudo apt install python3.11` on Debian/Ubuntu,
`sudo dnf install python3.11` on Fedora, or already installed on Arch.
- **Bun**`curl -fsSL https://bun.sh/install | bash`.
- **FFmpeg**`sudo apt install ffmpeg` (Debian/Ubuntu), `sudo dnf install ffmpeg-free` (Fedora), or `sudo pacman -S ffmpeg` (Arch).
- **Rust / Cargo** (required for building from source only)`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
- **Rust / Cargo**`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
- **GTK/WebKit deps** for the Tauri shell:
@@ -74,24 +89,49 @@ APP_ID="com.debpalash.omnivoice-studio"
APP_NAME="OmniVoice Studio"
```
## AppImage white-screen on Fedora 44 / Ubuntu 24.04
## AppImage white screen / EGL errors (Fedora 44, Ubuntu 24.04+, 26.04)
<a id="appimage-white-screen-on-fedora-44--ubuntu-2404"></a>
Newer distros ship WebKitGTK 2.44 / 2.46, which has a compositing-mode
regression that lands the Tauri window as a fully-white frame with no UI.
Two separate WebKitGTK rendering issues land the Tauri window as a
fully-white frame with no UI. Which one you have depends on your WebKitGTK
version (`pkg-config --modversion webkit2gtk-4.1` prints it).
**Workaround:** set `WEBKIT_DISABLE_COMPOSITING_MODE=1` before launching:
**Modern WebKitGTK (2.48+ — Ubuntu 24.04 and newer, incl. 26.04): try this
first.** WebKit's DMA-BUF renderer fails against some GPU drivers; the
terminal typically shows:
```
Could not create default EGL display: EGL_BAD_PARAMETER
```
Disable the DMA-BUF renderer before launching:
```bash
WEBKIT_DISABLE_DMABUF_RENDERER=1 ./OmniVoice.Studio_*.AppImage
```
**WebKitGTK 2.44 / 2.46 (Fedora 44, Ubuntu 24.04 at release):** a
compositing-mode regression blanks the surface on first paint. Disable
compositing mode instead:
```bash
WEBKIT_DISABLE_COMPOSITING_MODE=1 ./OmniVoice.Studio_*.AppImage
```
OmniVoice's AppRun launcher autodetects the broken WebKitGTK range and sets
this for you (shipped in v0.3+). The manual env-var path remains the documented
fallback when running from a checked-out source tree.
OmniVoice's AppRun launcher autodetects the broken 2.44/2.46 range and sets
this second variable for you (shipped in v0.3+). The manual env-var path
remains the documented fallback when running from a checked-out source tree.
Tracking issue: [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62).
**Last resort** — if neither variable alone helps, force software rendering
(slower, but always paints):
```bash
WEBKIT_DISABLE_DMABUF_RENDERER=1 LIBGL_ALWAYS_SOFTWARE=1 ./OmniVoice.Studio_*.AppImage
```
Tracking issues: [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62),
[#961](https://github.com/debpalash/OmniVoice-Studio/issues/961).
## .deb ffprobe conflict
@@ -134,17 +174,35 @@ that picks these defaults automatically; for v0.3 set them by hand.
<a id="amd-gpu-rocm"></a>
OmniVoice **auto-detects AMD GPUs**`get_best_device()` returns the GPU when a
ROCm build of PyTorch is installed (ROCm-built PyTorch reports through
`torch.cuda.is_available()`), and OmniVoice auto-sets `HSA_OVERRIDE_GFX_VERSION`
for consumer cards whose GFX ID isn't in the official ROCm support matrix. No
code changes or flags are needed.
ROCm support is **Linux-only and opt-in**. The **default install ships the
CUDA build** of PyTorch (the `pytorch-cuda` index in `pyproject.toml`), so on
an AMD-only machine `torch.cuda.is_available()` is `False` and OmniVoice runs
on CPU until you opt into the ROCm variant. (On Windows there is no ROCm path
at all — PyTorch publishes no Windows ROCm wheels; see
[windows.md](windows.md#gpu-support).)
The catch: the **default install ships the CUDA build** of PyTorch (the
`pytorch-cuda` index in `pyproject.toml`), so on an AMD-only machine
`torch.cuda.is_available()` is `False` and OmniVoice falls back to CPU. To use
your AMD GPU, replace torch with the ROCm wheel **after** the first-run install
populates the venv:
Three ways to opt in, in order of preference:
**1. First-run setup screen (recommended).** On Linux the setup screen's
**Compute** card offers **"AMD GPU (ROCm, Linux)"** next to the default
**Auto**. When OmniVoice detects an AMD GPU *and* the ROCm userspace
(`/opt/rocm` present, or `rocminfo` on PATH), the ROCm option is pre-selected;
with an AMD GPU but no ROCm runtime it stays offered-but-unselected — install
ROCm first (or continue on CPU). Choosing ROCm makes the bootstrap reinstall
`torch`/`torchaudio` from the ROCm wheel index
(`https://download.pytorch.org/whl/rocm6.2` by default) right after the
dependency sync.
**2. Environment variable (existing installs / headless).** Set
`OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
the same ROCm reinstall. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
index when you need a different ROCm version
([pytorch.org](https://pytorch.org/get-started/locally/) lists available
wheels). If the reinstall fails (network, unsupported card), OmniVoice keeps
the default torch build and warns instead of breaking the install.
**3. Manual wheel swap (fallback).** Replace torch with the ROCm wheel
**after** the first-run install populates the venv:
```bash
# From the project directory (source install), into OmniVoice's uv venv.
@@ -154,16 +212,20 @@ uv pip install --reinstall torch torchaudio \
--index-url https://download.pytorch.org/whl/rocm6.2
```
Then relaunch — the Settings → System panel should now report the GPU device
instead of `cpu`. Verify the wheel sees your card:
Once a ROCm build of PyTorch is in the venv, detection is automatic —
`get_best_device()` returns the GPU (ROCm-built PyTorch reports through
`torch.cuda.is_available()`), and OmniVoice auto-sets
`HSA_OVERRIDE_GFX_VERSION` for consumer cards whose GFX ID isn't in the
official ROCm support matrix. Relaunch and the Settings → System panel should
report the GPU device instead of `cpu`. Verify the wheel sees your card:
```bash
uv run python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
```
Notes:
- ROCm is **Linux-only** and **opt-in** — the default cross-platform behavior
(CUDA on NVIDIA, MPS on Apple, CPU elsewhere) is unchanged.
- ROCm is exercised far less than the default CUDA/MPS/CPU paths — it works,
but expect rough edges on consumer cards and report what you hit.
- Unsupported GFX (e.g. some consumer RDNA cards): if it still won't run, set
`HSA_OVERRIDE_GFX_VERSION` yourself (e.g. `export HSA_OVERRIDE_GFX_VERSION=11.0.0`)
to the nearest supported architecture before launching.
@@ -171,8 +233,6 @@ Notes:
native ROCm wheel.
Tracking issue: [#124](https://github.com/debpalash/OmniVoice-Studio/issues/124).
An installer-integrated, env-var-driven ROCm wheel selection is a planned
follow-up; until then this manual step is the supported path.
## Hugging Face token (optional but recommended)
+15 -2
View File
@@ -17,13 +17,26 @@ working OmniVoice Studio install on macOS (Apple Silicon).
## Prerequisites
### Using the DMG
- **macOS 12 (Monterey) or newer** — Apple Silicon (Intel: UI only, see the
note above).
- **~10 GB free disk** for the app, its Python environment, and model weights.
That's it — GPU acceleration (Apple MPS) is automatic on Apple Silicon, and
Python, FFmpeg, and the model weights are bundled or bootstrapped by the app
itself on first launch. No toolchain needed.
### Building from source
Everything above, plus the toolchain:
- **Xcode Command Line Tools**`xcode-select --install` (includes **git**
and the C toolchain; `curl` ships with macOS).
- **Python 3.11+**`brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
- **Bun**`curl -fsSL https://bun.sh/install | bash`.
- **Xcode Command Line Tools**`xcode-select --install`.
- **FFmpeg** (used by the dubbing + capture pipelines) — `brew install ffmpeg`.
- **Rust / Cargo** (required for building from source only)`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
- **Rust / Cargo**`curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
Optional but recommended:
+23 -4
View File
@@ -115,13 +115,22 @@ quarantines every download.
**Fix:** see [macos.md#gatekeeper-quarantine](macos.md#gatekeeper-quarantine).
## 4. AppImage white screen on Fedora 44 / Ubuntu 24.04
## 4. AppImage white screen / EGL errors (Fedora 44, Ubuntu 24.04+, 26.04)
**Symptom:** the AppImage window opens fully white. No UI ever appears.
**Symptom:** the AppImage window opens fully white. No UI ever appears. On
newer distros (Ubuntu 24.04 and later, incl. 26.04) the terminal often shows
`Could not create default EGL display: EGL_BAD_PARAMETER`.
**Cause:** WebKitGTK 2.44 / 2.46 compositing-mode regression.
**Cause:** WebKitGTK rendering regressions — the DMA-BUF renderer on modern
WebKitGTK (2.48+), or the 2.44 / 2.46 compositing mode.
**Fix:** see [linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404](linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404).
**Fix:** try `WEBKIT_DISABLE_DMABUF_RENDERER=1` first (modern WebKitGTK / the
EGL error), then `WEBKIT_DISABLE_COMPOSITING_MODE=1` — full walkthrough incl.
the software-rendering last resort:
[linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404](linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404).
**Linked issues:** [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62),
[#961](https://github.com/debpalash/OmniVoice-Studio/issues/961)
## 5. Windows Triton / torch.compile OOM
@@ -382,6 +391,16 @@ overhead. It reuses your existing faster-whisper install (nothing extra to
download). OmniVoice never switches engines automatically — this stays your
call.
> **Seeing "The backend crashed (exit code …)" instead?** That's the other
> failure mode: the backend **process died** (native CUDA abort, out-of-memory
> kill, DLL crash) rather than hanging. Newer desktop builds detect the death,
> restart the backend automatically (giving up after 3 crashes in 10 minutes),
> and show a crash notice with a **View crash details** button (exit code +
> the last error output). Use **Report this bug** from that notice — the crash
> evidence is attached to the prefilled GitHub issue automatically, with home
> paths scrubbed. The raw markers live next to the backend logs in
> `backend_crash_markers.json`.
## 15. Stuck at "preparing" forever after a crash / BSOD (Windows)
**Symptom:** after an unclean shutdown (Windows BSOD, forced power-off), every
+33 -5
View File
@@ -5,7 +5,24 @@ working OmniVoice Studio install on Windows 10 / 11 (x64).
## Prerequisites
### Using the MSI installer
- **Windows 10 (21H2 or newer) or Windows 11**, x64.
- **~10 GB free disk** for the app, its Python environment, and model weights.
- Optional: an **NVIDIA GPU + driver** for CUDA acceleration — see
[GPU support on Windows](#gpu-support). AMD GPUs run CPU-only on Windows.
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
by the app itself on first launch. No toolchain needed.
### Building from source
Everything above, plus the toolchain:
- **Git for Windows**`winget install --id Git.Git -e`. Needed for
`git clone`, and it includes **Git Bash**, which `bun run desktop-prod`
uses to run its build-and-launch script. Without it, `desktop-prod` stops
with an error telling you to install it.
- **Python 3.11+**`winget install Python.Python.3.11` (or download from
[python.org](https://www.python.org/downloads/windows/)).
- **Microsoft C++ Build Tools** — required by some PyPI source distributions
@@ -14,13 +31,24 @@ working OmniVoice Studio install on Windows 10 / 11 (x64).
with the **"Desktop development with C++"** workload checked.
- **Bun**`powershell -c "irm bun.sh/install.ps1 | iex"`.
- **FFmpeg**`winget install Gyan.FFmpeg`.
- **Git for Windows** (from-source installs only) — `winget install --id Git.Git -e`.
You need it for `git clone` anyway, and it includes **Git Bash**, which
`bun run desktop-prod` uses to run its build-and-launch script. Without it,
`desktop-prod` stops with an error telling you to install it.
- **Rust / Cargo** (required for building from source only) — `winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
- **Rust / Cargo**`winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
After installing Rustup, close and reopen PowerShell before running `bun run desktop-prod`.
## GPU support on Windows
<a id="gpu-support"></a>
**GPU acceleration on Windows is NVIDIA/CUDA-only.** The Windows install
ships the CUDA build of PyTorch; with an NVIDIA GPU and a regular NVIDIA
driver it's picked up automatically (no CUDA Toolkit install needed).
**AMD GPUs — including Ryzen / Ryzen AI integrated Radeon graphics — run
CPU-only on Windows.** ROCm is not supported on Windows: PyTorch publishes no
Windows ROCm wheels, and OmniVoice's ROCm option is Linux-only. (The Ryzen AI
NPU is likewise not used.) Everything still works on CPU, just slower. If you
have an AMD GPU and want GPU acceleration, run OmniVoice on Linux instead —
see [linux.md — AMD GPU (ROCm)](linux.md#amd-gpu-rocm).
## Install (from source)
Run from a regular (non-admin) PowerShell:
Binary file not shown.

Before

Width:  |  Height:  |  Size: 187 KiB

After

Width:  |  Height:  |  Size: 331 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 134 KiB

After

Width:  |  Height:  |  Size: 420 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 342 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 106 KiB

After

Width:  |  Height:  |  Size: 420 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 358 KiB

After

Width:  |  Height:  |  Size: 426 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 278 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 295 KiB

After

Width:  |  Height:  |  Size: 347 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 183 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 666 KiB

+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omnivoice-studio",
"version": "0.3.9",
"version": "0.3.11",
"private": true,
"license": "AGPL-3.0-only",
"type": "module",
+1 -1
View File
@@ -2941,7 +2941,7 @@ dependencies = [
[[package]]
name = "omnivoice-studio"
version = "0.3.9"
version = "0.3.11"
dependencies = [
"arboard",
"dirs-next",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "omnivoice-studio"
version = "0.3.9"
version = "0.3.11"
description = "OmniVoice Studio AI voice cloning & dubbing desktop app"
authors = ["Debpalash"]
license = "AGPL-3.0-only"
+75 -1
View File
@@ -34,11 +34,55 @@ pub fn port_in_use(port: u16) -> bool {
pub fn backend_healthy(port: u16) -> bool {
let url = format!("http://127.0.0.1:{}/system/info", port);
match ureq_get_with_timeout(&url, Duration::from_millis(500)) {
Ok(body) => body.contains("\"model_checkpoint\"") || body.contains("\"data_dir\""),
Ok(body) => is_omnivoice_body(&body),
Err(_) => false,
}
}
fn is_omnivoice_body(body: &str) -> bool {
body.contains("\"model_checkpoint\"") || body.contains("\"data_dir\"")
}
/// The `app_version` reported by the OmniVoice backend at :port.
/// `None` when nothing OmniVoice answers there (port free, or a foreign
/// process). `Some("")` when it IS our backend but predates the
/// `app_version` field — callers treat that as stale.
pub fn running_backend_version(port: u16) -> Option<String> {
let url = format!("http://127.0.0.1:{}/system/info", port);
let body = ureq_get_with_timeout(&url, Duration::from_millis(500)).ok()?;
if !is_omnivoice_body(&body) {
return None;
}
Some(parse_app_version(&body).unwrap_or_default())
}
/// Extract `"app_version": "X"` from a /system/info body. String-sniff on one
/// field (consistent with `backend_healthy`) — no JSON dependency needed.
fn parse_app_version(body: &str) -> Option<String> {
let key = "\"app_version\"";
let rest = &body[body.find(key)? + key.len()..];
let rest = rest[rest.find(':')? + 1..].trim_start();
let rest = rest.strip_prefix('"')?;
Some(rest[..rest.find('"')?].to_string())
}
/// Whether a running backend's version matches THIS app build, comparing
/// **base** versions (any `-N` pre-release suffix stripped from both sides) so
/// a preview build `0.3.10-4` still attaches to its `0.3.10` backend.
///
/// Why this exists (the "bound port blocked the newer version" report): an
/// orphaned backend from a *previous* version keeps answering health checks
/// after an update, so "healthy" alone made the new UI silently attach to old
/// backend code — every fix in the update appeared to change nothing. A
/// version-mismatched (or unversioned) OmniVoice responder is stale by
/// definition; callers kill it and spawn the bundled backend instead.
pub fn same_app_version(running: &str) -> bool {
fn base(v: &str) -> &str {
v.split('-').next().unwrap_or(v).trim()
}
!running.is_empty() && base(running) == base(env!("CARGO_PKG_VERSION"))
}
fn ureq_get_with_timeout(url: &str, timeout: Duration) -> Result<String, String> {
let url = url.strip_prefix("http://").ok_or("only http:// supported")?;
let (host_port, path) = match url.find('/') {
@@ -365,4 +409,34 @@ mod tests {
assert!(diag.contains("Interpreter present on disk: false"));
assert!(diag.contains("Clean & Retry"), "must give an actionable hint");
}
// ── stale-backend detection (the "bound port blocked the newer version"
// report: a healthy orphan from a previous version must NOT be
// attached to) ─────────────────────────────────────────────────────
#[test]
fn parse_app_version_reads_system_info_shape() {
let body = r#"{"app_version":"0.3.9","data_dir":"/x","model_checkpoint":"k2"}"#;
assert_eq!(parse_app_version(body).as_deref(), Some("0.3.9"));
// whitespace after the colon is fine
assert_eq!(
parse_app_version(r#"{ "app_version" : "1.2.3" }"#).as_deref(),
Some("1.2.3")
);
// pre-app_version backends and foreign bodies yield None
assert_eq!(parse_app_version(r#"{"data_dir":"/x"}"#), None);
assert_eq!(parse_app_version("<html>not json</html>"), None);
}
#[test]
fn same_app_version_matches_current_build_and_rejects_stale() {
let ours = env!("CARGO_PKG_VERSION");
assert!(same_app_version(ours), "own version must attach");
// preview stamp of the same base still attaches
assert!(same_app_version(&format!("{}-7", ours)));
// a different (older) release is stale
assert!(!same_app_version("0.0.1"));
// unversioned (pre-app_version backend) is stale by definition
assert!(!same_app_version(""));
}
}
+148 -37
View File
@@ -12,6 +12,7 @@ use serde::Serialize;
use tauri::{Emitter, Manager};
use crate::config::get_effective_region;
use crate::crash::BackendExit;
use crate::tools::resolve_uv;
use crate::{AppFlags, BackendState, backend_port};
@@ -145,13 +146,34 @@ pub fn retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_, BootstrapS
set_stage(&stage_handle, BootstrapStage::Ready);
return;
}
if crate::backend::backend_healthy(backend_port()) {
log::info!("Port {} already serving OmniVoice backend — attaching", backend_port());
set_stage(&stage_handle, BootstrapStage::Ready);
return;
match crate::backend::running_backend_version(backend_port()) {
Some(v) if crate::backend::same_app_version(&v) => {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
}
Some(v) => {
// A healthy-but-stale backend from a previous version (the
// classic post-update orphan). Attaching would silently run
// OLD backend code under the new UI — replace it instead.
log::warn!(
"Port {} serves a stale OmniVoice backend (v{} != app v{}) — replacing it",
backend_port(),
if v.is_empty() { "<unknown>" } else { v.as_str() },
env!("CARGO_PKG_VERSION"),
);
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
crate::backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
None => {}
}
if crate::backend::port_in_use(backend_port()) {
log::warn!("Port {} in use — taking ownership", backend_port());
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
crate::backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
@@ -173,9 +195,7 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
let mut venv_heal_attempted = false;
'bootstrap: loop {
let child = crate::backend::spawn_backend(app, Some(stage_handle));
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
*guard = child;
}
track_backend_child(app, child);
let start = std::time::Instant::now();
while start.elapsed() < Duration::from_secs(300) {
if crate::backend::backend_healthy(backend_port()) {
@@ -194,20 +214,41 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
}
return;
}
let process_dead = if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
match guard.as_mut() {
Some(child) => match child.try_wait() {
Ok(Some(status)) => Some(status.to_string()),
Ok(None) => None,
Err(_) => Some("unknown".to_string()),
},
None => Some("never started".to_string()),
}
} else {
None
};
if let Some(exit_info) = process_dead {
let process_dead: Option<(String, Option<BackendExit>)> =
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
match guard.as_mut() {
Some(child) => match child.try_wait() {
Ok(Some(status)) => {
let exit = BackendExit::from_status(status);
Some((exit.description.clone(), Some(exit)))
}
Ok(None) => None,
// try_wait errored — the death is real but its
// shape is unknown; no exit code for the marker.
Err(_) => Some(("unknown".to_string(), None)),
},
// Spawn itself failed — no process ever ran, so this
// is a spawn failure (spawn_failure_diagnostic owns
// it), NOT a crash: no marker.
None => Some(("never started".to_string(), None)),
}
} else {
None
};
if let Some((exit_info, real_exit)) = process_dead {
let err_tail = crate::backend::read_error_log_tail(30);
// #941: persist the forensics for every true process death —
// startup crashes included — unless the app is shutting down
// or a retry flow deliberately killed the child.
if let Some(ref exit) = real_exit {
if !app_is_quitting(app) && !backend_kill_intended() {
crate::crash::record_crash(crate::crash::marker_now(
exit,
backend_uptime_s(app),
crate::backend::read_error_log_tail(CRASH_STDERR_TAIL_LINES),
));
}
}
// #314: a backend that dies because the venv itself is broken
// can only be healed by rebuilding the venv — do that once
// instead of failing into an unwinnable retry loop.
@@ -289,12 +330,34 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
/// first to reach Ready claims this and the rest fall through.
static SUPERVISOR_ACTIVE: AtomicBool = AtomicBool::new(false);
/// Give up (surface Failed) if the backend dies this many times within
/// `RESTART_WINDOW` — a deterministic startup crash must not become a
/// fork-bomb. The #314 broken-venv self-heal stays the venv-failure path; the
/// #941: set while a retry/clean-retry flow deliberately kills the backend to
/// replace it, so the death watchers (startup poll + supervisor) never write a
/// crash marker for — or respawn against — an *intentional* kill. Cleared the
/// moment a fresh child is spawned and tracked (`track_backend_child`).
static BACKEND_KILL_INTENDED: AtomicBool = AtomicBool::new(false);
pub fn set_backend_kill_intended(value: bool) {
BACKEND_KILL_INTENDED.store(value, Ordering::SeqCst);
}
fn backend_kill_intended() -> bool {
BACKEND_KILL_INTENDED.load(Ordering::SeqCst)
}
/// How much of backend_err.log rides inside a crash marker (#941). ~40 lines
/// is enough for a Python traceback or a native abort banner without bloating
/// the marker file or the bug-report URL (the frontend truncates further).
const CRASH_STDERR_TAIL_LINES: usize = 40;
/// Crash-loop escalation guard (#941, supersedes the #567 5-in-60s budget):
/// give up (surface Failed with the crash details) once the backend has died
/// `MAX_RESTARTS` times inside `RESTART_WINDOW`. The longer 10-minute window
/// catches *slow* crash loops (e.g. an engine that OOMs a couple of minutes
/// into every generation) that the old 60-second window let spin silently
/// forever. The #314 broken-venv self-heal stays the venv-failure path; the
/// supervisor only handles post-Ready deaths.
const MAX_RESTARTS: usize = 5;
const RESTART_WINDOW: Duration = Duration::from_secs(60);
const MAX_RESTARTS: usize = 3;
const RESTART_WINDOW: Duration = Duration::from_secs(600);
fn app_is_quitting(app: &tauri::AppHandle) -> bool {
app.try_state::<AppFlags>()
@@ -302,17 +365,39 @@ fn app_is_quitting(app: &tauri::AppHandle) -> bool {
.unwrap_or(false)
}
/// Returns `Some(exit description)` if the tracked backend child has exited,
/// Store the freshly spawned backend child (and its spawn time, for the crash
/// marker's `uptime_s`), and re-arm the death watchers: any deliberate-kill
/// window ends the moment a new child is tracked.
fn track_backend_child(app: &tauri::AppHandle, child: Option<std::process::Child>) {
let state = app.state::<BackendState>();
if let Ok(mut guard) = state.process.lock() {
*guard = child;
}
if let Ok(mut spawned) = state.spawned_at.lock() {
*spawned = Some(Instant::now());
}
set_backend_kill_intended(false);
}
/// Seconds since the tracked backend child was spawned (0 when unknown).
fn backend_uptime_s(app: &tauri::AppHandle) -> u64 {
app.try_state::<BackendState>()
.and_then(|s| s.spawned_at.lock().ok().and_then(|g| *g))
.map(|t| t.elapsed().as_secs())
.unwrap_or(0)
}
/// Returns `Some(BackendExit)` if the tracked backend child has exited,
/// `None` if it is still running (or none is tracked — which we never treat as
/// a death to respawn, to avoid fighting a deliberate teardown).
fn backend_child_exit(app: &tauri::AppHandle) -> Option<String> {
fn backend_child_exit(app: &tauri::AppHandle) -> Option<BackendExit> {
let state = app.try_state::<BackendState>()?;
let mut guard = state.process.lock().ok()?;
match guard.as_mut() {
Some(child) => match child.try_wait() {
Ok(Some(status)) => Some(status.to_string()),
Ok(Some(status)) => Some(BackendExit::from_status(status)),
Ok(None) => None,
Err(e) => Some(format!("try_wait error: {e}")),
Err(e) => Some(BackendExit::unknown(&format!("try_wait error: {e}"))),
},
None => None,
}
@@ -339,21 +424,39 @@ fn supervise_backend(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<BootstrapS
if app_is_quitting(app) {
return;
}
let exit_info = match backend_child_exit(app) {
Some(info) => info,
let exit = match backend_child_exit(app) {
Some(exit) => exit,
None => continue, // still running
};
// The exit may have raced with a shutdown that killed the child.
if app_is_quitting(app) {
return;
}
// A retry/clean-retry flow killed the child on purpose and owns the
// respawn — no crash marker, and step aside so the retry's own
// spawn_backend_and_wait claims the supervisor slot at Ready (#941).
if backend_kill_intended() {
log::info!("Backend exit was a deliberate replace — supervisor yielding to the retry flow");
return;
}
let exit_info = exit.description.clone();
// #941: make the death self-documenting BEFORE any restart attempt —
// the marker (exit code/signal + stderr tail + uptime) is what turns
// the next "Can't reach the backend" report into a diagnosable one.
let uptime_s = backend_uptime_s(app);
crate::crash::record_crash(crate::crash::marker_now(
&exit,
uptime_s,
crate::backend::read_error_log_tail(CRASH_STDERR_TAIL_LINES),
));
if restart_budget_exhausted(&mut restart_times, Instant::now()) {
let tail = crate::backend::read_error_log_tail(30);
let msg = format!(
"The backend kept crashing ({} times in {}s) and couldn't be kept running. \
Use Clean & Retry, or check Settings Logs Backend.{}",
"The backend kept crashing ({} times in {} min; last death: {}) and couldn't \
be kept running. Use Clean & Retry, or check Settings Logs Backend.{}",
MAX_RESTARTS,
RESTART_WINDOW.as_secs(),
RESTART_WINDOW.as_secs() / 60,
exit.label(),
if tail.is_empty() { String::new() } else { format!("\n\nLast output:\n{tail}") },
);
log::error!("Backend supervisor giving up: {msg}");
@@ -374,9 +477,7 @@ fn supervise_backend(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<BootstrapS
std::thread::sleep(Duration::from_millis(300));
}
let child = crate::backend::spawn_backend(app, Some(stage_handle));
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
*guard = child;
}
track_backend_child(app, child);
// Wait (bounded) for the respawn to become healthy. If it dies again
// immediately, bail early so the next loop counts it toward the cap.
let start = Instant::now();
@@ -411,6 +512,7 @@ pub fn clean_and_retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_,
// project dir, otherwise bootstrap will "attach" to the stale process.
if crate::backend::port_in_use(backend_port()) {
log::warn!("Clean retry: killing stale backend on port {}", backend_port());
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
crate::backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
@@ -1649,6 +1751,15 @@ mod tests {
assert_eq!(envs.get("UV_HTTP_RETRIES").map(String::as_str), Some("5"));
}
#[test]
fn crash_loop_policy_is_three_deaths_in_ten_minutes() {
// #941 escalation guard: ≥3 crashes inside 10 min must stop the
// respawn loop and land on the Failed screen with the crash details —
// the old 5-in-60s budget let slow crash loops spin silently forever.
assert_eq!(MAX_RESTARTS, 3);
assert_eq!(RESTART_WINDOW, Duration::from_secs(600));
}
#[test]
fn restart_budget_caps_respawns_and_prunes_old_ones() {
// Supervisor backoff policy (#567): fewer than MAX_RESTARTS deaths
+318
View File
@@ -0,0 +1,318 @@
//! Backend crash forensics (#941).
//!
//! When the backend PROCESS dies (native CUDA abort, OOM kill, DLL crash),
//! the user used to see only "Can't reach the local OmniVoice backend" — and
//! the evidence (exit code, stderr tail) evaporated with the process. Every
//! such report was undiagnosable without asking for logs nobody sends.
//!
//! This module makes every backend death self-documenting: the death watchers
//! in `bootstrap.rs` (the startup health poll and the post-Ready supervisor)
//! call [`record_crash`] with the exit status and captured stderr tail, which
//! persists a small JSON **crash marker** next to the backend logs. The
//! frontend reads the newest marker via the `get_last_backend_crash` command
//! to replace the vague unreachable-toast with the honest story ("the backend
//! crashed (exit code X)…"), and the bug-report prefill attaches it so the
//! next #941-class GitHub issue arrives WITH the evidence.
//!
//! Only the last [`MAX_MARKERS`] crashes are kept. Acknowledgment is a
//! persisted timestamp (not deletion!) so viewing the crash details doesn't
//! destroy the evidence a subsequent bug report needs.
use std::fs;
use std::path::{Path, PathBuf};
use std::process::ExitStatus;
use serde::{Deserialize, Serialize};
/// How many crash markers to retain (newest first).
pub const MAX_MARKERS: usize = 3;
// ── Exit-status decomposition ──────────────────────────────────────────────
/// Structured view of how the backend child ended: the numeric exit code (or
/// Unix signal) for the marker, plus the human-readable `ExitStatus` display
/// for logs and bootstrap messages.
#[derive(Clone, Debug, PartialEq)]
pub struct BackendExit {
pub code: Option<i32>,
pub signal: Option<i32>,
pub description: String,
}
impl BackendExit {
pub fn from_status(status: ExitStatus) -> Self {
#[cfg(unix)]
let signal = {
use std::os::unix::process::ExitStatusExt;
status.signal()
};
#[cfg(not(unix))]
let signal = None;
BackendExit { code: status.code(), signal, description: status.to_string() }
}
/// For deaths we can't decompose (`try_wait` errored).
pub fn unknown(description: &str) -> Self {
BackendExit { code: None, signal: None, description: description.to_string() }
}
/// Short human label — "exit code 3221226505" / "signal 6" — for messages.
pub fn label(&self) -> String {
match (self.code, self.signal) {
(Some(c), _) => format!("exit code {}", c),
(None, Some(s)) => format!("signal {}", s),
(None, None) => self.description.clone(),
}
}
}
// ── Marker model ───────────────────────────────────────────────────────────
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct CrashMarker {
/// Unix seconds when the death was detected.
pub ts: u64,
/// Process exit code, when the OS reported one.
pub exit_code: Option<i32>,
/// Unix signal that killed the process (None on Windows / normal exits).
pub signal: Option<i32>,
/// Human-readable `ExitStatus` display ("exit status: 134", …).
pub exit_desc: String,
/// App/backend version (lockstep per the versioning rule).
pub backend_version: String,
/// Seconds the backend had been running when it died.
pub uptime_s: u64,
/// Tail of backend_err.log captured at death time.
pub last_stderr: String,
}
/// The single on-disk store: newest-first markers plus the acknowledgment
/// watermark. One file keeps rotation + ack updates atomic-ish and avoids
/// filename collisions for same-second crashes.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct CrashStore {
/// `ts` of the newest marker the user has acknowledged (seen). Markers
/// with `ts <= acked_ts` are "old news" for UI purposes but are retained
/// for bug-report attachment.
#[serde(default)]
pub acked_ts: u64,
/// Newest first, capped at [`MAX_MARKERS`].
#[serde(default)]
pub markers: Vec<CrashMarker>,
}
/// Prepend `marker` and keep only the newest [`MAX_MARKERS`]. Pure so the
/// rotation policy is unit-tested without touching the filesystem.
pub fn push_marker(store: &mut CrashStore, marker: CrashMarker) {
store.markers.insert(0, marker);
store.markers.truncate(MAX_MARKERS);
}
/// Newest marker + whether the user has already acknowledged it.
pub fn newest_with_ack(store: &CrashStore) -> Option<(CrashMarker, bool)> {
store.markers.first().map(|m| (m.clone(), m.ts <= store.acked_ts))
}
// ── Persistence ────────────────────────────────────────────────────────────
/// The marker store lives next to the backend logs (same rationale: it's
/// forensic output of the backend process, discoverable alongside
/// backend.log / backend_err.log).
pub fn markers_path() -> PathBuf {
crate::backend::backend_log_path().with_file_name("backend_crash_markers.json")
}
pub fn load_store_from(path: &Path) -> CrashStore {
fs::read_to_string(path)
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save_store_to(path: &Path, store: &CrashStore) {
match serde_json::to_string_pretty(store) {
Ok(json) => {
if let Err(e) = fs::write(path, json) {
log::warn!("Could not persist crash marker to {}: {}", path.display(), e);
}
}
Err(e) => log::warn!("Could not serialize crash marker: {}", e),
}
}
fn now_unix_s() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
/// Build a marker for a death detected right now.
pub fn marker_now(exit: &BackendExit, uptime_s: u64, last_stderr: String) -> CrashMarker {
CrashMarker {
ts: now_unix_s(),
exit_code: exit.code,
signal: exit.signal,
exit_desc: exit.description.clone(),
backend_version: env!("CARGO_PKG_VERSION").to_string(),
uptime_s,
last_stderr,
}
}
/// Persist an unexpected backend death. Called by the death watchers in
/// `bootstrap.rs` AFTER they have ruled out intentional shutdowns (app quit,
/// deliberate retry/clean-retry kills).
pub fn record_crash(marker: CrashMarker) {
log::error!(
"Backend process died unexpectedly ({}, uptime {} s). Crash marker written. Stderr tail:\n{}",
marker.exit_desc,
marker.uptime_s,
if marker.last_stderr.is_empty() { "<none captured>" } else { &marker.last_stderr },
);
let path = markers_path();
let mut store = load_store_from(&path);
push_marker(&mut store, marker);
save_store_to(&path, &store);
}
// ── Tauri commands ─────────────────────────────────────────────────────────
/// Newest crash marker + its acknowledgment state, as returned to the
/// frontend (`get_last_backend_crash`).
#[derive(Clone, Debug, Serialize)]
pub struct CrashNotice {
#[serde(flatten)]
pub marker: CrashMarker,
pub acknowledged: bool,
}
/// Newest backend crash marker, or null when the backend has never crashed.
/// `acknowledged` tells the UI whether the user already viewed/dismissed it.
#[tauri::command]
pub fn get_last_backend_crash() -> Option<CrashNotice> {
let store = load_store_from(&markers_path());
newest_with_ack(&store).map(|(marker, acknowledged)| CrashNotice { marker, acknowledged })
}
/// Mark the newest crash as seen. Deliberately does NOT delete the marker —
/// the bug-report prefill still needs the evidence after the user viewed it.
#[tauri::command]
pub fn acknowledge_backend_crash() {
let path = markers_path();
let mut store = load_store_from(&path);
if let Some(newest_ts) = store.markers.first().map(|m| m.ts) {
if store.acked_ts < newest_ts {
store.acked_ts = newest_ts;
save_store_to(&path, &store);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn marker(ts: u64) -> CrashMarker {
CrashMarker {
ts,
exit_code: Some(1),
signal: None,
exit_desc: format!("exit status: 1 (#{ts})"),
backend_version: "0.0.0-test".into(),
uptime_s: 42,
last_stderr: "Traceback…".into(),
}
}
#[test]
fn rotation_keeps_only_the_last_three_newest_first() {
// #941: write 4 markers → only the newest MAX_MARKERS survive.
let mut store = CrashStore::default();
for ts in [1, 2, 3, 4] {
push_marker(&mut store, marker(ts));
}
assert_eq!(store.markers.len(), MAX_MARKERS);
let kept: Vec<u64> = store.markers.iter().map(|m| m.ts).collect();
assert_eq!(kept, vec![4, 3, 2], "newest first, oldest dropped");
}
#[test]
fn ack_semantics_survive_newer_crashes() {
let mut store = CrashStore::default();
push_marker(&mut store, marker(100));
// Fresh crash → unacknowledged.
let (m, acked) = newest_with_ack(&store).expect("has a marker");
assert_eq!(m.ts, 100);
assert!(!acked, "a fresh crash must be unacknowledged");
// Viewing acks the newest…
store.acked_ts = 100;
assert!(newest_with_ack(&store).unwrap().1, "viewed crash is acknowledged");
// …but a NEWER crash re-arms the notice, and the marker itself is
// retained (evidence survives the ack — bug reports still attach it).
push_marker(&mut store, marker(200));
let (m2, acked2) = newest_with_ack(&store).unwrap();
assert_eq!(m2.ts, 200);
assert!(!acked2, "a newer crash must surface again");
assert_eq!(store.markers.len(), 2, "ack never deletes markers");
}
#[test]
fn store_roundtrips_through_json_and_defaults_when_missing() {
let dir = std::env::temp_dir().join(format!("omnivoice-test-941-{}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
fs::create_dir_all(&dir).unwrap();
let path = dir.join("backend_crash_markers.json");
// Missing file → default store, never an error (first run).
assert_eq!(load_store_from(&path), CrashStore::default());
// Corrupt file → default store (a truncated write must not wedge the
// whole forensics path).
fs::write(&path, "{not json").unwrap();
assert_eq!(load_store_from(&path), CrashStore::default());
let mut store = CrashStore::default();
push_marker(
&mut store,
CrashMarker {
ts: 1,
exit_code: None,
signal: Some(6), // SIGABRT — the native-CUDA-abort shape
exit_desc: "signal: 6 (SIGABRT)".into(),
backend_version: "0.3.10".into(),
uptime_s: 7,
last_stderr: "CUDA error: an illegal memory access".into(),
},
);
store.acked_ts = 0;
save_store_to(&path, &store);
let loaded = load_store_from(&path);
assert_eq!(loaded, store, "Option fields (code=None, signal=Some) must roundtrip");
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn backend_exit_labels_code_signal_and_unknown() {
let coded = BackendExit { code: Some(-1073740791), signal: None, description: "x".into() };
assert_eq!(coded.label(), "exit code -1073740791");
let signaled = BackendExit { code: None, signal: Some(9), description: "x".into() };
assert_eq!(signaled.label(), "signal 9");
let unknown = BackendExit::unknown("try_wait error: gone");
assert_eq!(unknown.label(), "try_wait error: gone");
}
#[cfg(unix)]
#[test]
fn backend_exit_decomposes_real_exit_statuses() {
use std::os::unix::process::ExitStatusExt;
// Normal exit with code 3.
let e = BackendExit::from_status(ExitStatus::from_raw(3 << 8));
assert_eq!(e.code, Some(3));
assert_eq!(e.signal, None);
// Killed by SIGABRT (6) — code is None, signal carries the story.
let k = BackendExit::from_status(ExitStatus::from_raw(6));
assert_eq!(k.code, None);
assert_eq!(k.signal, Some(6));
assert_eq!(k.label(), "signal 6");
}
}
+39 -7
View File
@@ -13,6 +13,7 @@ pub mod bootstrap;
pub mod tools;
pub mod backend;
pub mod commands;
pub mod crash;
pub mod updater_channel;
use std::process::Child;
@@ -41,6 +42,9 @@ pub fn backend_port() -> u16 {
pub struct BackendState {
pub process: Mutex<Option<Child>>,
/// When the tracked child was spawned — feeds the crash marker's
/// `uptime_s` (#941). Set alongside `process` in bootstrap.rs.
pub spawned_at: Mutex<Option<std::time::Instant>>,
}
pub struct AppFlags {
@@ -271,6 +275,8 @@ pub fn run() {
commands::get_launch_as_widget,
commands::set_launch_as_widget,
commands::clear_webview_cache_and_relaunch,
crash::get_last_backend_crash,
crash::acknowledge_backend_crash,
])
.setup(move |app| {
app.handle().plugin(tauri_plugin_dialog::init())?;
@@ -639,6 +645,7 @@ pub fn run() {
app.manage(bootstrap_state);
app.manage(BackendState {
process: Mutex::new(None),
spawned_at: Mutex::new(None),
});
let app_handle = app.handle().clone();
@@ -657,13 +664,30 @@ pub fn run() {
set_stage(&stage_handle, BootstrapStage::AwaitingSetup);
return;
}
if backend::backend_healthy(backend_port()) {
log::info!(
"Port {} already serving OmniVoice backend — attaching",
backend_port()
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
match backend::running_backend_version(backend_port()) {
Some(v) if backend::same_app_version(&v) => {
log::info!(
"Port {} already serving OmniVoice backend v{} — attaching",
backend_port(), v
);
set_stage(&stage_handle, BootstrapStage::Ready);
return;
}
Some(v) => {
// Healthy-but-stale backend from a previous version —
// the post-update orphan that made new installs run
// old backend code. Replace it (see backend.rs
// same_app_version for the full story).
log::warn!(
"Port {} serves a stale OmniVoice backend (v{} != app v{}) — replacing it",
backend_port(),
if v.is_empty() { "<unknown>" } else { v.as_str() },
env!("CARGO_PKG_VERSION"),
);
backend::kill_orphan_on_port(backend_port());
std::thread::sleep(Duration::from_millis(500));
}
None => {}
}
if backend::port_in_use(backend_port()) {
log::warn!(
@@ -720,6 +744,14 @@ pub fn run() {
app.run(|app_handle, event| {
if let tauri::RunEvent::ExitRequested { .. } = event {
// Raise the quitting flag FIRST: exits that don't pass through the
// tray Quit item (macOS ⌘Q, OS session end) would otherwise let a
// death watcher observe our own SIGTERM below and record a false
// "backend crashed" marker (#941).
app_handle
.state::<AppFlags>()
.quitting
.store(true, Ordering::SeqCst);
if let Ok(mut lock) = app_handle.state::<BackendState>().process.lock() {
if let Some(ref mut child) = *lock {
let pid = child.id();
+63 -6
View File
@@ -42,6 +42,7 @@ import WorkspaceVoices from './components/WorkspaceVoices';
import WorkspaceProjects from './components/WorkspaceProjects';
import ErrorBoundary from './components/ErrorBoundary';
import FloatingPill from './components/FloatingPill';
import BackendCrashNotice from './components/BackendCrashNotice';
// RemoteAuthGate is mounted at the true outermost provider in main-app.jsx so
// it covers all app states (setup check / wizard / bootstrap), not just the
// main studio return below. Do not re-wrap here double-gating renders two
@@ -72,6 +73,7 @@ import {
CLONE_MAX_SECONDS,
} from './utils/constants';
import { LANG_CODES } from './utils/languages';
import { restoreProjectExtras } from './utils/projectState';
import { API, apiFetch } from './api/client';
import { flushMemory as apiFlushMemory } from './api/system';
import {
@@ -416,9 +418,15 @@ function App() {
const defaultTrack = useAppStore((s) => s.defaultTrack);
const setDefaultTrack = useAppStore((s) => s.setDefaultTrack);
const exportTracks = useAppStore((s) => s.exportTracks);
const setExportTracks = useAppStore((s) => s.setExportTracks);
const previewSegIds = useAppStore((s) => s.previewSegIds);
const speakerClones = useAppStore((s) => s.speakerClones);
const setSpeakerClones = useAppStore((s) => s.setSpeakerClones);
// Multi-language batch picks (P1.4) saved with the project payload.
const multiLangMode = useAppStore((s) => s.multiLangMode);
const setMultiLangMode = useAppStore((s) => s.setMultiLangMode);
const multiLangs = useAppStore((s) => s.multiLangs);
const setMultiLangs = useAppStore((s) => s.setMultiLangs);
const setGlossaryTerms = useAppStore((s) => s.setGlossaryTerms);
const dualSubs = useAppStore((s) => s.dualSubs);
@@ -450,6 +458,8 @@ function App() {
closeDirection,
saveDirection,
setLastGenFingerprints,
fingerprintsByLang,
setFingerprintsByLang,
incrementalPlan,
recomputeIncremental,
} = useSegmentEditing();
@@ -952,6 +962,17 @@ function App() {
preserveBg,
defaultTrack,
speakerClones,
// P1.4 multi-language batch setup + export-track prefs travel with
// the project. Additive: loaders default them when absent (see
// utils/projectState.js).
multiLangMode,
multiLangs,
exportTracks,
// P1.3 per-language segment fingerprints, so reopening a project
// keeps every track's "Regen N changed" plan. Additive: legacy
// loaders ignore the key; segments' `translations` maps ride along
// inside dubSegments above.
segHashesByLang: fingerprintsByLang,
},
};
try {
@@ -993,9 +1014,24 @@ function App() {
setDubStep(s.dubStep === 'done' ? 'done' : s.dubSegments?.length ? 'editing' : 'idle');
// Phase 4.5 rehydrate per-segment fingerprints. The incremental plan
// immediately shows "N segments changed" for any segments edited after
// the last generate.
setLastGenFingerprints(s.segHashes || {});
// the last generate. P1.3: prefer the per-language map; a legacy flat
// `segHashes` can only describe the project's saved target language.
if (
s.segHashesByLang &&
typeof s.segHashesByLang === 'object' &&
!Array.isArray(s.segHashesByLang)
) {
setFingerprintsByLang(s.segHashesByLang);
} else {
setLastGenFingerprints(s.segHashes || {}, s.dubLangCode || 'en');
}
setSpeakerClones(s.speakerClones || {});
// P1.4 restore multi-lang picks; legacy payloads default to off/empty
// and leave the in-session exportTracks untouched (null sentinel).
const extras = restoreProjectExtras(s);
setMultiLangMode(extras.multiLangMode);
setMultiLangs(extras.multiLangs);
if (extras.exportTracks) setExportTracks(extras.exportTracks);
toast.success(i18n.t('app.toast_opened', { name: data.name }));
} catch (err) {
toast.error(err.message);
@@ -1045,14 +1081,31 @@ function App() {
})),
);
setDubTranscript(job.full_transcript || '');
setDubLang(item.language || 'Auto');
setDubLangCode(item.language_code || 'und');
// Older DBs froze the language/language_code COLUMNS at the ingest-time
// "" (the UPSERT didn't update them until #P0 fixed it), but the job_data
// JSON always carried the value generation set. Falling back to job_data
// restores existing rows correctly without a migration.
setDubLang(item.language || job.language || 'Auto');
setDubLangCode(item.language_code || job.language_code || 'und');
setDubTracks(Object.keys(job.dubbed_tracks || {}));
setDubStep(Object.keys(job.dubbed_tracks || {}).length > 0 ? 'done' : 'editing');
// Phase 4.5 seg_hashes are written per successful segment by
// dub_generate.py. Reloading a half-generated dub lets the "Regen N
// changed" button resume right where the crash happened.
setLastGenFingerprints(job.seg_hashes || {});
// changed" button resume right where the crash happened. P1.3: prefer
// the per-language map (multi-track jobs); a legacy flat map belongs to
// the job's last-generated language the code restored just above.
if (
job.seg_hashes_by_lang &&
typeof job.seg_hashes_by_lang === 'object' &&
!Array.isArray(job.seg_hashes_by_lang)
) {
setFingerprintsByLang(job.seg_hashes_by_lang);
} else {
setLastGenFingerprints(
job.seg_hashes || {},
item.language_code || job.language_code || 'und',
);
}
// Rehydrate the auto-extracted speaker clones so the CAST dropdown's
// "🎤 From video" option reappears after a reload. Projects that
// predate the speaker-clone feature have an empty map; the Extract
@@ -1217,6 +1270,10 @@ function App() {
<FloatingPill />
{/* #941: honest surfacing of backend process crashes (exit code +
stderr tail from the shell's crash marker), with ack-on-view. */}
<BackendCrashNotice />
<Header
mode={mode}
setMode={setMode}
+32
View File
@@ -8,6 +8,16 @@
// and the API, so a remote device on http://<host>:<share-port> must hit
// that same origin — NOT a hardcoded :3900, which is cross-origin (CORS)
// and loopback-only/unreachable from another machine.
// Explicit .ts extension: tests/frontend/apiClient.test.mjs loads this module
// under `node --experimental-strip-types`, whose ESM resolver requires real
// file extensions (tsconfig has allowImportingTsExtensions for tsc).
import {
getUnacknowledgedBackendCrash,
describeCrashExit,
crashAge,
type BackendCrashMarker,
} from '../utils/backendCrash.ts';
const viteEnv = import.meta.env ?? {};
// Remote-backend settings (Wave 2.3): user-configured in Settings → Sharing.
// localStorage so the choice survives restarts; read once at module load —
@@ -143,6 +153,28 @@ export async function apiFetch(path: string, opts: RequestInit = {}): Promise<Re
await new Promise((r) => setTimeout(r, TRANSPORT_RETRY_BACKOFF_MS[attempt]));
continue;
}
// #941: if the desktop shell recorded an unacknowledged backend crash,
// tell the honest story instead of the vague "can't reach" — and let
// BackendCrashNotice raise its "View crash details" affordance.
let crash: BackendCrashMarker | null = null;
try {
crash = await getUnacknowledgedBackendCrash();
} catch {
/* forensics unavailable — fall through to the generic message */
}
if (crash) {
try {
window.dispatchEvent(new CustomEvent('ov:backend-crashed', { detail: crash }));
} catch {
/* no window (tests) — the ApiError below still tells the story */
}
throw new ApiError(
`The local OmniVoice backend crashed (${describeCrashExit(crash)}) ${crashAge(crash)} ago ` +
'and is being restarted — this request could not reach it. ' +
'Open the crash notice for the error output, or check Settings → Logs → Backend.',
{ status: 0, detail: lastDetail },
);
}
throw new ApiError(
"Can't reach the local OmniVoice backend — it may still be starting up, or it stopped. " +
'Wait a few seconds and try again; if it persists, restart the app (or check Settings → Logs → Backend).',
+17
View File
@@ -111,6 +111,23 @@ export async function clearDubHistory(): Promise<Response> {
return apiFetch('/dub/history', { method: 'DELETE' });
}
export interface DubTrackInfo {
path?: string;
language?: string;
language_code?: string;
duration?: number;
timing_strategy?: string;
}
/** Per-track metadata (duration, timing strategy, ) keyed by language code.
* Backs the track-pill tooltips; the store only carries the track codes. */
export async function dubListTracks(jobId: string): Promise<Record<string, DubTrackInfo>> {
const res = await apiJson<{ tracks?: Record<string, DubTrackInfo> }>(
`/dub/tracks/${encodeURIComponent(jobId)}`,
);
return res?.tracks || {};
}
export interface DubQCResponse {
engine: string;
total: number;
@@ -0,0 +1,153 @@
import { useCallback, useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { AlertTriangle, X } from 'lucide-react';
import { Button, Dialog } from '../ui';
import {
acknowledgeBackendCrash,
crashAge,
describeCrashExit,
getUnacknowledgedBackendCrash,
} from '../utils/backendCrash';
import { openExternal } from '../api/external';
import { buildBugReportUrl } from '../utils/bugReport';
/**
* BackendCrashNotice the honest half of #941.
*
* When the backend PROCESS dies, the desktop shell records a crash marker
* (src-tauri/src/crash.rs). This component surfaces it: a banner naming the
* exit code and when it happened, with a "View crash details" affordance that
* shows the captured stderr tail and a report path. Sources:
* - `ov:backend-crashed` window events, dispatched by api/client.ts when a
* request fails against a freshly crashed backend, and
* - a mount-time check, so a crash that happened with no request in flight
* (or a crash-loop that forced an app restart) still gets told.
*
* Viewing or dismissing acknowledges the marker (it is retained on disk so
* bug reports can still attach the evidence). Outside the Tauri shell the
* marker getters resolve null and this renders nothing.
*/
export default function BackendCrashNotice() {
const { t } = useTranslation();
const [marker, setMarker] = useState(null);
const [showDetails, setShowDetails] = useState(false);
useEffect(() => {
let cancelled = false;
getUnacknowledgedBackendCrash()
.then((m) => {
if (!cancelled && m) setMarker(m);
})
.catch(() => {});
const onCrash = (e) => {
if (e?.detail) setMarker(e.detail);
};
window.addEventListener('ov:backend-crashed', onCrash);
return () => {
cancelled = true;
window.removeEventListener('ov:backend-crashed', onCrash);
};
}, []);
const view = useCallback(() => {
setShowDetails(true);
// Ack on view the user has seen the honest story; the marker itself
// stays on disk for bug-report attachment.
acknowledgeBackendCrash().catch(() => {});
}, []);
const dismiss = useCallback(() => {
acknowledgeBackendCrash().catch(() => {});
setShowDetails(false);
setMarker(null);
}, []);
if (!marker) return null;
const exit = describeCrashExit(marker);
const ago = crashAge(marker);
return (
<>
<div
role="alert"
className="fixed left-1/2 top-[var(--space-4)] z-[70] flex w-[min(600px,92vw)] -translate-x-1/2 items-center gap-[var(--space-3)] rounded-lg border border-border bg-bg-elev-1 px-[var(--space-4)] py-[var(--space-3)] shadow-lg backdrop-blur-md"
>
<AlertTriangle size={16} className="shrink-0 text-danger" aria-hidden />
<span className="flex-1 text-[length:var(--text-sm)] text-fg">
{t('crash.notice', { exit, ago })}
</span>
<Button variant="subtle" size="sm" onClick={view}>
{t('crash.view')}
</Button>
<Button
variant="ghost"
size="sm"
iconSize="sm"
onClick={dismiss}
title={t('crash.dismiss')}
>
<X size={12} />
</Button>
</div>
<Dialog
open={showDetails}
onClose={() => {
setShowDetails(false);
setMarker(null);
}}
title={t('crash.details_title')}
size="lg"
footer={
<>
<Button
variant="subtle"
onClick={async () => {
try {
// buildBugReportUrl attaches the crash marker (exit code +
// scrubbed stderr tail) automatically the report arrives
// WITH the evidence.
await openExternal(
await buildBugReportUrl({ title: `[Crash] Backend died (${exit})` }),
);
} catch (e) {
console.warn('[BackendCrashNotice] report action failed', e);
}
}}
>
{t('errors.report')}
</Button>
<Button variant="primary" onClick={dismiss}>
{t('common.close')}
</Button>
</>
}
>
<div className="flex flex-col gap-[var(--space-4)]">
<p className="m-0 text-[length:var(--text-sm)] text-fg-muted">
{t('crash.details_intro', { exit, ago })}
</p>
<dl className="m-0 grid grid-cols-[max-content_1fr] gap-x-[var(--space-5)] gap-y-[var(--space-2)] text-[length:var(--text-sm)]">
<dt className="text-fg-subtle">{t('crash.field_exit')}</dt>
<dd className="m-0 font-mono text-fg">{exit}</dd>
<dt className="text-fg-subtle">{t('crash.field_when')}</dt>
<dd className="m-0 text-fg">{new Date(marker.ts * 1000).toLocaleString()}</dd>
<dt className="text-fg-subtle">{t('crash.field_uptime')}</dt>
<dd className="m-0 text-fg">{t('crash.uptime_value', { count: marker.uptime_s })}</dd>
<dt className="text-fg-subtle">{t('crash.field_version')}</dt>
<dd className="m-0 text-fg">{marker.backend_version}</dd>
</dl>
<div>
<div className="mb-[var(--space-2)] text-[length:var(--text-sm)] text-fg-subtle">
{t('crash.stderr_title')}
</div>
<pre className="m-0 max-h-[40vh] overflow-auto rounded-md border border-border bg-bg-elev-2 p-[var(--space-3)] font-mono text-[length:var(--text-xs)] leading-relaxed text-fg whitespace-pre-wrap">
{marker.last_stderr || t('crash.no_stderr')}
</pre>
</div>
</div>
</Dialog>
</>
);
}
@@ -0,0 +1,82 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import BackendCrashNotice from './BackendCrashNotice';
import { acknowledgeBackendCrash, getUnacknowledgedBackendCrash } from '../utils/backendCrash';
// #941: the crash-notice branch a recorded backend death must surface the
// honest message (exit code + age) with a "View crash details" affordance,
// and viewing/dismissing must acknowledge the marker.
vi.mock('../utils/backendCrash', async (importOriginal) => {
const actual = await importOriginal();
return {
...actual,
getUnacknowledgedBackendCrash: vi.fn().mockResolvedValue(null),
acknowledgeBackendCrash: vi.fn().mockResolvedValue(undefined),
};
});
vi.mock('../utils/bugReport', () => ({
buildBugReportUrl: vi.fn().mockResolvedValue('https://example.test/issues/new'),
}));
vi.mock('../api/external', () => ({
openExternal: vi.fn().mockResolvedValue(undefined),
}));
const MARKER = {
ts: Math.floor(Date.now() / 1000) - 12,
exit_code: 134,
signal: null,
exit_desc: 'exit status: 134',
backend_version: '0.3.10',
uptime_s: 87,
last_stderr: 'CUDA error: an illegal memory access was encountered',
acknowledged: false,
};
describe('BackendCrashNotice', () => {
beforeEach(() => {
vi.clearAllMocks();
getUnacknowledgedBackendCrash.mockResolvedValue(null);
});
it('renders nothing when the shell reports no crash', async () => {
const { container } = render(<BackendCrashNotice />);
await waitFor(() => expect(getUnacknowledgedBackendCrash).toHaveBeenCalled());
expect(container).toBeEmptyDOMElement();
});
it('shows the honest message and the details affordance for a fresh marker', async () => {
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
render(<BackendCrashNotice />);
const alert = await screen.findByRole('alert');
// Honest: names the exit code instead of a vague "can't reach".
expect(alert.textContent).toContain('crashed');
expect(alert.textContent).toContain('exit code 134');
expect(screen.getByRole('button', { name: /view crash details/i })).toBeInTheDocument();
});
it('surfaces a crash pushed via the ov:backend-crashed event', async () => {
render(<BackendCrashNotice />);
await waitFor(() => expect(getUnacknowledgedBackendCrash).toHaveBeenCalled());
window.dispatchEvent(new CustomEvent('ov:backend-crashed', { detail: MARKER }));
const alert = await screen.findByRole('alert');
expect(alert.textContent).toContain('exit code 134');
});
it('acks on view and shows the stderr tail in the details dialog', async () => {
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
render(<BackendCrashNotice />);
fireEvent.click(await screen.findByRole('button', { name: /view crash details/i }));
expect(acknowledgeBackendCrash).toHaveBeenCalledTimes(1);
expect(await screen.findByText(/illegal memory access/)).toBeInTheDocument();
expect(screen.getByText('Backend crash details')).toBeInTheDocument();
});
it('ack + clear on dismiss', async () => {
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
render(<BackendCrashNotice />);
await screen.findByRole('alert');
fireEvent.click(screen.getByRole('button', { name: /dismiss/i }));
expect(acknowledgeBackendCrash).toHaveBeenCalledTimes(1);
await waitFor(() => expect(screen.queryByRole('alert')).not.toBeInTheDocument());
});
});
+5 -1
View File
@@ -259,7 +259,11 @@ export default function ExportModal({
onClose?.();
};
const runClips = () => {
handleAudioExport?.(`${API}/dub/export-segments/${jobId}`, 'segments.zip');
// Ask for the ACTIVE track's per-segment clips (P1.3 the cache is
// language-keyed now); omitted lang falls back to the last-generated
// track server-side, which is all a legacy single-track job has.
const langQ = dubLangCode ? `?lang=${encodeURIComponent(dubLangCode)}` : '';
handleAudioExport?.(`${API}/dub/export-segments/${jobId}${langQ}`, 'segments.zip');
onClose?.();
};
+38 -15
View File
@@ -1,8 +1,17 @@
import React, { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react';
import React, {
useCallback,
useEffect,
useLayoutEffect,
useMemo,
useRef,
useState,
useSyncExternalStore,
} from 'react';
import { useTranslation } from 'react-i18next';
import { Play, Headphones } from 'lucide-react';
import {
REGION_COLORS,
getRegionColors,
subscribeRegionColors,
SNAP_PX,
visibleSegmentRange,
snapTime,
@@ -27,11 +36,15 @@ const fmt = (t) => {
* SegmentTrack custom DOM segment editor lane for the dub timeline (#280).
*
* Replaces the WaveSurfer Regions plugin in the editing path. Renders one
* absolutely-positioned box per segment inside a lane whose horizontal
* position is derived from a single {pxPerSec, scrollLeft} source (read off
* WaveSurfer's wrapper by the parent), so boxes stay pixel-aligned with the
* waveform across zoom/scroll/resize. Virtualized by TIME only the boxes
* inside the visible window (+ buffer) are mounted.
* absolutely-positioned box per segment, each placed in PURE LAYOUT from a
* single {pxPerSec, scrollLeft} source (read off WaveSurfer's wrapper by the
* parent), so boxes stay pixel-aligned with the waveform across
* zoom/scroll/resize. The lane itself must NEVER be transform-animated: a
* lane translateX'd on every playback tick gets promoted to a compositor
* layer by Chromium, and composited semi-transparent paints flash
* invisible/visible on some Windows GPU/WebView2 drivers (#373; #381 only
* dampened it). Virtualized by TIME only the boxes inside the visible
* window (+ buffer) are mounted.
*
* Props:
* segments sorted-by-start segment array (store shape)
@@ -131,13 +144,14 @@ export default function SegmentTrack({
[effSegments, viewStart, viewEnd],
);
// Palette snapshot re-blends against the new --chrome-bg on theme change
// (#963) new array identity per re-blend, so the memo below recolors.
const regionColors = useSyncExternalStore(subscribeRegionColors, getRegionColors);
const speakerColor = useMemo(() => {
const speakers = [...new Set(segments.map((s) => s.speaker_id).filter(Boolean))];
const bySpeaker = new Map(
speakers.map((sp, i) => [sp, REGION_COLORS[i % REGION_COLORS.length]]),
);
return (seg, idx) => bySpeaker.get(seg.speaker_id) || REGION_COLORS[idx % REGION_COLORS.length];
}, [segments]);
const bySpeaker = new Map(speakers.map((sp, i) => [sp, regionColors[i % regionColors.length]]));
return (seg, idx) => bySpeaker.get(seg.speaker_id) || regionColors[idx % regionColors.length];
}, [segments, regionColors]);
// Onset tick strip (one viewport-sized canvas, non-interactive)
useEffect(() => {
@@ -465,6 +479,13 @@ export default function SegmentTrack({
const innerWidth = Math.max(viewWidth, Math.ceil(duration * pxPerSec));
const playheadX = currentTime * pxPerSec - effScroll;
const windowed = effSegments.slice(lo, hi);
// Scroll offset baked into each box's `left` (viewport coordinates) instead
// of a `translateX` on the lane an animated lane transform is composited
// by Chromium and flashes on some Windows GPU/WebView2 drivers (#373). In
// the selfScroll fallback the viewport is a real scroll container, so boxes
// stay in lane coordinates (offset 0). The virtualization window above
// derives from the same effScroll, so both stay consistent by construction.
const laneShift = selfScroll ? 0 : effScroll;
return (
<div
@@ -489,14 +510,16 @@ export default function SegmentTrack({
aria-orientation="horizontal"
title={t('timeline.keyboard_hint')}
style={{
width: innerWidth,
transform: selfScroll ? undefined : `translateX(${-effScroll}px)`,
// selfScroll needs the full content width for the native
// scrollbar range; in synced mode the lane is a static
// viewport-sized strip (NO transform see laneShift above).
width: selfScroll ? innerWidth : '100%',
}}
>
{windowed.map((s) => {
const sid = String(s.id);
const idx = indexById.get(sid) ?? 0;
const left = s.start * pxPerSec;
const left = s.start * pxPerSec - laneShift;
const width = Math.max(2, (s.end - s.start) * pxPerSec);
const isSel = selectedId != null && String(selectedId) === sid;
const isFocus = focusId === sid;
+82 -2
View File
@@ -1,6 +1,6 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, fireEvent } from '@testing-library/react';
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { render, screen, fireEvent, act } from '@testing-library/react';
import SegmentTrack from './SegmentTrack';
// Mocked transport: fixed pxPerSec/scrollLeft, no WaveSurfer. jsdom has no
@@ -171,6 +171,86 @@ describe('SegmentTrack — keyboard', () => {
});
});
describe('SegmentTrack — compositor-safe positioning (#373)', () => {
const baseProps = {
segments: SEGS,
pxPerSec: 100,
duration: 10,
currentTime: 0,
onsets: [],
};
it('the lane never carries a transform — at rest and after a scroll update', () => {
// Regression guard for #373: an animated `translateX` on the lane gets
// composited by Chromium during playback and its boxes flash on some
// Windows GPU/WebView2 drivers. Any reintroduction must fail here.
const { rerender } = render(<SegmentTrack {...baseProps} scrollLeft={0} />);
expect(screen.getByRole('listbox').style.transform).toBe('');
// Simulate a WaveSurfer autoscroll tick during playback.
rerender(<SegmentTrack {...baseProps} scrollLeft={250} />);
expect(screen.getByRole('listbox').style.transform).toBe('');
});
it('boxes are laid out in viewport coordinates: left = start·pxPerSec scrollLeft', () => {
setup({ scrollLeft: 250 });
expect(box(0).style.left).toBe('-250px'); // start 0
expect(box(1).style.left).toBe('50px'); // start 3 300 250
expect(box(2).style.left).toBe('250px'); // start 5 500 250
});
it('selfScroll fallback keeps lane coordinates — viewport scroll must not double-shift boxes', () => {
setup({ selfScroll: true });
const lane = screen.getByRole('listbox');
const viewport = lane.parentElement;
// jsdom has no layout: stub the scroll position, then fire the event the
// component listens to.
Object.defineProperty(viewport, 'scrollLeft', { value: 120, configurable: true });
fireEvent.scroll(viewport);
expect(lane.style.transform).toBe('');
expect(box(1).style.left).toBe('300px'); // lane coords: 3s · 100px/s
});
});
describe('SegmentTrack — engine-independent box paint (#963)', () => {
const root = document.documentElement;
afterEach(async () => {
// Restore the default theme and let the palette observer settle so the
// module-level cache can't leak into other tests in this file.
await act(async () => {
root.style.removeProperty('--chrome-bg');
root.removeAttribute('data-theme');
await new Promise((resolve) => setTimeout(resolve, 0));
});
});
it('inline background is a literal opaque rgb() — no color-mix/var() the CSSOM could reject', () => {
// WebView2/Chromium < 111 rejects a color-mix() inline-style assignment
// wholesale, and .seg-track__box declares no fallback background the
// boxes rendered fully transparent (#963). The inline value must be
// plain rgb() so every engine parses it.
setup();
for (const el of screen.getAllByRole('option')) {
expect(el.style.background).toMatch(/^rgb\(\d{1,3}, \d{1,3}, \d{1,3}\)$/);
}
// Default theme, first palette slot: 0.45·rgb(211,134,155) over #0f1011.
expect(box(0).style.background).toBe('rgb(103, 69, 79)');
});
it('boxes re-blend live when the theme changes ([data-theme] on <html>)', async () => {
setup();
expect(box(0).style.background).toBe('rgb(103, 69, 79)');
await act(async () => {
// Same seam App.jsx uses: swap --chrome-bg and flag the theme.
root.style.setProperty('--chrome-bg', '#1e293b');
root.setAttribute('data-theme', 'slate');
await new Promise((resolve) => setTimeout(resolve, 0)); // flush MutationObserver
});
// round(0.45·[211,134,155] + 0.55·[30,41,59])
expect(box(0).style.background).toBe('rgb(111, 83, 102)');
});
});
describe('SegmentTrack — pointer + selection', () => {
it('pointerdown selects the segment (table sync)', () => {
const { onSelectSeg } = setup();
+4 -20
View File
@@ -30,6 +30,7 @@ import WaveformPlayer from './WaveformPlayer';
import { useAppStore } from '../store';
import { useTranslation } from 'react-i18next';
import { askConfirm } from '../utils/dialog';
import { absoluteTime, timeAgo } from '../utils/relativeTime';
const SIDEBAR_TABS = [
{ id: 'projects', icon: FolderOpen, accent: '#b8bb26' },
@@ -71,20 +72,6 @@ function LazyWaveformPlayer({ height = 36, className = '', ...rest }) {
return <div ref={holderRef} className={className} style={{ height }} aria-hidden="true" />;
}
function timeAgo(ms) {
const diff = Date.now() - ms;
if (!isFinite(diff) || diff < 0) return '';
const s = Math.floor(diff / 1000);
if (s < 60) return `${s}s ago`;
const m = Math.floor(s / 60);
if (m < 60) return `${m}m ago`;
const h = Math.floor(m / 60);
if (h < 24) return `${h}h ago`;
const d = Math.floor(h / 24);
if (d < 7) return `${d}d ago`;
return new Date(ms).toLocaleDateString([], { month: 'short', day: 'numeric' });
}
export default function Sidebar(props) {
const {
availableTabs = ['projects', 'history', 'downloads'],
@@ -311,11 +298,8 @@ export default function Sidebar(props) {
<span className="history-kind history-kind--audio">
<Film size={9} /> {t('sidebar.dub_label')}
</span>
<span
className="history-meta"
title={new Date(proj.updated_at * 1000).toLocaleString()}
>
{timeAgo(proj.updated_at * 1000)}
<span className="history-meta" title={absoluteTime(proj.updated_at)}>
{timeAgo(proj.updated_at)}
</span>
</div>
<div className="history-title">{proj.name}</div>
@@ -772,7 +756,7 @@ export default function Sidebar(props) {
>
<KindIcon size={9} /> {item.mode}
</span>
<span className="history-meta">{timeAgo(item.created_at * 1000)}</span>
<span className="history-meta">{timeAgo(item.created_at)}</span>
</div>
<div className="history-title">{item.filename}</div>
<div className="history-subtitle">
@@ -2,6 +2,7 @@ import React, { useEffect, useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Mic, Search, Clock, Languages } from 'lucide-react';
import { Dialog, Input } from '../ui';
import { toMillis } from '../utils/relativeTime';
import { loadTranscriptions, TRANSCRIPTION_EVENT } from '../utils/transcriptionsStore';
/**
@@ -27,10 +28,12 @@ export default function TranscriptionPicker({ open, onClose, onPick }) {
}, [open]);
// Relative time, host-locale absolute fallback; null on unparseable timestamp.
// toMillis keeps this unit-safe (ISO strings today; seconds/ms tolerated).
const formatTime = (iso) => {
const d = new Date(iso);
if (Number.isNaN(d.getTime())) return null;
const diff = Date.now() - d.getTime();
const ms = toMillis(iso);
if (ms == null) return null;
const d = new Date(ms);
const diff = Date.now() - ms;
if (diff < 60000) return t('transcriptions.just_now');
if (diff < 3600000) return t('transcriptions.m_ago', { count: Math.floor(diff / 60000) });
if (diff < 86400000) return t('transcriptions.h_ago', { count: Math.floor(diff / 3600000) });
+3 -20
View File
@@ -10,21 +10,7 @@ import React, { useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { Search, Film, FolderOpen, Trash2, Save, Pencil, Check, X } from 'lucide-react';
import { Button } from '../ui';
// Local copy of the sidebar's relative-time formatter (small, self-contained).
function timeAgo(ms) {
const diff = Date.now() - ms;
if (!isFinite(diff) || diff < 0) return '';
const s = Math.floor(diff / 1000);
if (s < 60) return `${s}s ago`;
const m = Math.floor(s / 60);
if (m < 60) return `${m}m ago`;
const h = Math.floor(m / 60);
if (h < 24) return `${h}h ago`;
const d = Math.floor(h / 24);
if (d < 7) return `${d}d ago`;
return new Date(ms).toLocaleDateString([], { month: 'short', day: 'numeric' });
}
import { absoluteTime, timeAgo } from '../utils/relativeTime';
export default function WorkspaceProjects({
projects = [],
@@ -105,11 +91,8 @@ export default function WorkspaceProjects({
<span className="history-kind history-kind--audio">
<Film size={9} /> {t('sidebar.dub_label')}
</span>
<span
className="history-meta"
title={new Date(proj.updated_at * 1000).toLocaleString()}
>
{timeAgo(proj.updated_at * 1000)}
<span className="history-meta" title={absoluteTime(proj.updated_at)}>
{timeAgo(proj.updated_at)}
</span>
</div>
{editingId === proj.id ? (
+109 -101
View File
@@ -25,6 +25,7 @@ export default function DubHeader({
handleDubStop,
dubProgress,
onGenerateClick,
isTranslating,
multiLangMode,
multiLangs,
incrementalPlan,
@@ -34,121 +35,128 @@ export default function DubHeader({
setExportOpen,
}) {
return (
<div className="flex flex-wrap justify-between items-center gap-x-[var(--space-2)] gap-y-[4px] min-w-0 px-[10px] py-[4px] shrink-0 bg-[var(--color-bg-elev-1)] rounded-md mb-[2px]">
{/* Pipeline spine, inlined onto the header row (Upload → … → Export). */}
<DubPipelineStepper dubStep={dubStep} inline />
<div className="label-row dub-head__title !gap-[6px]">
<FileText className="label-icon" size={11} />
<span className="font-medium text-[0.78rem] min-w-0 overflow-hidden text-ellipsis whitespace-nowrap text-fg normal-case">
{dubFilename}
</span>
<span className="text-fg-muted font-normal whitespace-nowrap text-[0.68rem] normal-case shrink-0">
· {formatTime(dubDuration)} · {dubSegments.length} {t('dub.segs')}
</span>
{activeProjectName && activeProjectName !== dubFilename && (
<span className="text-[#b8bb26] ml-[var(--space-2)] whitespace-nowrap text-[0.68rem] normal-case overflow-hidden text-ellipsis min-w-0">
{activeProjectName}
<div className="flex flex-col gap-[2px] min-w-0 px-[10px] py-[4px] shrink-0 bg-[var(--color-bg-elev-1)] rounded-md mb-[2px]">
{/* Row 1: project title (left) + actions (right). Row 2: the pipeline
spine (Upload Export) sits directly under the title with a
tight 2px gap title-first, owner-requested order. */}
<div className="flex flex-wrap justify-between items-center gap-x-[var(--space-2)] gap-y-[4px] min-w-0">
<div className="label-row dub-head__title !gap-[6px]">
<FileText className="label-icon" size={11} />
<span className="font-medium text-[0.78rem] min-w-0 overflow-hidden text-ellipsis whitespace-nowrap text-fg normal-case">
{dubFilename}
</span>
)}
</div>
<div className="flex gap-[6px] items-center shrink-0">
{/* Icon-only secondary actions (tooltips carry the labels);
<span className="text-fg-muted font-normal whitespace-nowrap text-[0.68rem] normal-case shrink-0">
· {formatTime(dubDuration)} · {dubSegments.length} {t('dub.segs')}
</span>
{activeProjectName && activeProjectName !== dubFilename && (
<span className="text-[#b8bb26] ml-[var(--space-2)] whitespace-nowrap text-[0.68rem] normal-case overflow-hidden text-ellipsis min-w-0">
{activeProjectName}
</span>
)}
</div>
<div className="flex gap-[6px] items-center shrink-0">
{/* Icon-only secondary actions (tooltips carry the labels);
Generate Dub keeps its label as the primary verb. */}
<Button
variant="subtle"
size="sm"
onClick={saveProject}
title={t('dub.save')}
aria-label={t('dub.save')}
>
<Save size={12} />
</Button>
<Button
variant="danger"
size="sm"
onClick={resetDub}
title={t('dub.reset')}
aria-label={t('dub.reset')}
>
<RotateCcw size={12} />
</Button>
{/* Primary actions live on the header bar (compact) — moved up from the footer. */}
<div className="flex gap-[6px] items-center pl-[var(--space-2)] ml-[2px]">
{dubStep === 'stopping' ? (
<FooterBtn
sm
tone="stopping"
disabled
icon={<Loader className="spinner" size={9} />}
label={t('dub.stopping')}
/>
) : dubStep === 'generating' ? (
<FooterBtn
sm
tone="danger"
onClick={handleDubStop}
icon={<Square size={9} />}
label={t('dub.stop_progress', {
current: dubProgress.current,
total: dubProgress.total,
})}
/>
) : (
<>
<Button
variant="subtle"
size="sm"
onClick={saveProject}
title={t('dub.save')}
aria-label={t('dub.save')}
>
<Save size={12} />
</Button>
<Button
variant="danger"
size="sm"
onClick={resetDub}
title={t('dub.reset')}
aria-label={t('dub.reset')}
>
<RotateCcw size={12} />
</Button>
{/* Primary actions live on the header bar (compact) — moved up from the footer. */}
<div className="flex gap-[6px] items-center pl-[var(--space-2)] ml-[2px]">
{dubStep === 'stopping' ? (
<FooterBtn
sm
tone={dubSegments.length ? 'pink' : 'idle'}
onClick={onGenerateClick}
disabled={!dubSegments.length}
icon={<Play size={11} />}
label={
multiLangMode && multiLangs.length > 1
? t('dub.generate_dub_multi', {
count: multiLangs.length,
defaultValue: 'Generate {{count}} dubs',
})
: t('dub.generate_dub')
}
tone="stopping"
disabled
icon={<Loader className="spinner" size={9} />}
label={t('dub.stopping')}
/>
{dubStep === 'done' && incrementalPlan && incrementalPlan.stale?.length > 0 && (
) : dubStep === 'generating' ? (
<FooterBtn
sm
tone="danger"
onClick={handleDubStop}
icon={<Square size={9} />}
label={t('dub.stop_progress', {
current: dubProgress.current,
total: dubProgress.total,
})}
/>
) : (
<>
<FooterBtn
sm
tone="pink"
onClick={() =>
handleDubGenerate({ regenOnly: incrementalPlan.stale, preview: true })
}
tone={dubSegments.length && !isTranslating ? 'pink' : 'idle'}
onClick={onGenerateClick}
// The multi-language batch translates between generates while
// dubStep briefly sits back at 'editing' keep the CTA inert
// during that phase so a re-click can't start a second batch.
disabled={!dubSegments.length || isTranslating}
icon={<Play size={11} />}
label={t('dub.regen_changed', { count: incrementalPlan.stale.length })}
label={
multiLangMode && multiLangs.length > 1
? t('dub.generate_dub_multi', {
count: multiLangs.length,
defaultValue: 'Generate {{count}} dubs',
})
: t('dub.generate_dub')
}
/>
)}
</>
)}
{dubStep === 'done' && (
{dubStep === 'done' && incrementalPlan && incrementalPlan.stale?.length > 0 && (
<FooterBtn
sm
tone="pink"
onClick={() =>
handleDubGenerate({ regenOnly: incrementalPlan.stale, preview: true })
}
icon={<Play size={11} />}
label={t('dub.regen_changed', { count: incrementalPlan.stale.length })}
/>
)}
</>
)}
{dubStep === 'done' && (
<FooterBtn
sm
tone="idle"
disabled={qcRunning || !dubSegments.length}
onClick={handleDubQc}
icon={
qcRunning ? <Loader className="spinner" size={11} /> : <ShieldCheck size={11} />
}
title={t('dub.qc_btn', { defaultValue: 'Verify dub timing (second-pass check)' })}
aria-label={t('dub.qc_btn', {
defaultValue: 'Verify dub timing (second-pass check)',
})}
/>
)}
<FooterBtn
sm
tone="idle"
disabled={qcRunning || !dubSegments.length}
onClick={handleDubQc}
icon={
qcRunning ? <Loader className="spinner" size={11} /> : <ShieldCheck size={11} />
}
title={t('dub.qc_btn', { defaultValue: 'Verify dub timing (second-pass check)' })}
aria-label={t('dub.qc_btn', {
defaultValue: 'Verify dub timing (second-pass check)',
})}
tone={dubStep === 'done' ? 'green' : 'idle'}
disabled={dubStep !== 'done' && !dubSegments.length}
onClick={() => setExportOpen(true)}
icon={<Download size={12} />}
title={t('dub.export_btn')}
aria-label={t('dub.export_btn')}
/>
)}
<FooterBtn
sm
tone={dubStep === 'done' ? 'green' : 'idle'}
disabled={dubStep !== 'done' && !dubSegments.length}
onClick={() => setExportOpen(true)}
icon={<Download size={12} />}
title={t('dub.export_btn')}
aria-label={t('dub.export_btn')}
/>
</div>
</div>
</div>
<DubPipelineStepper dubStep={dubStep} inline />
</div>
);
}
@@ -18,6 +18,7 @@ import { useAppStore } from '../../store';
import WaveformTimeline from '../WaveformTimeline';
import MultiLangPicker from '../MultiLangPicker';
import { API } from '../../api/client';
import { dubListTracks } from '../../api/dub';
import { LANG_CODES } from '../../utils/languages';
import ALL_LANGUAGES from '../../languages.json';
import { POPULAR_LANGS, PRESETS } from '../../utils/constants';
@@ -143,6 +144,49 @@ export default function DubLeftColumn({
else toast.error(t('dub.copy_failed'));
};
// Per-track metadata (duration + timing strategy) for the pill tooltips.
// The store only carries the track codes, so hydrate lazily from the
// existing GET /dub/tracks/{job_id} once the editor shows tracks (re-runs
// when a new language finishes and dubTracks changes). Failure-silent:
// the pills render fine without tooltips.
const [trackInfo, setTrackInfo] = useState({});
useEffect(() => {
if (!hasDubbedTrack || !dubJobId) return undefined;
let cancelled = false;
dubListTracks(dubJobId)
.then((tracks) => {
if (!cancelled) setTrackInfo(tracks || {});
})
.catch(() => {
/* tooltip enrichment only — never block or toast */
});
return () => {
cancelled = true;
};
}, [hasDubbedTrack, dubJobId, dubTracks]);
const trackTooltip = (code) => {
const info = trackInfo[code];
if (!info) return undefined;
const parts = [];
if (Number.isFinite(info.duration) && info.duration > 0) {
parts.push(
t('dub.track_tip_duration', {
duration: fmtDur(Math.round(info.duration)),
defaultValue: 'Duration {{duration}}',
}),
);
}
if (info.timing_strategy) {
// Reuse the timing-strategy display names where they exist
// (dub.timing_<id>); unknown/future strategies fall back to the raw id.
const strategy = t(`dub.timing_${info.timing_strategy}`, {
defaultValue: info.timing_strategy,
});
parts.push(t('dub.track_tip_timing', { strategy, defaultValue: 'Timing {{strategy}}' }));
}
return parts.length ? parts.join(' · ') : undefined;
};
return (
<div className="studio-panel dub-panel-col">
{hasDubbedTrack && (
@@ -170,6 +214,7 @@ export default function DubLeftColumn({
aria-checked={previewMode === code}
className={`dub-lang-pill ${previewMode === code ? 'is-active' : ''}`}
onClick={() => setPreviewMode(code)}
title={trackTooltip(code)}
>
{label}
</button>
@@ -40,6 +40,10 @@ export default function LLMProvidersPanel() {
const [modelsTruncated, setModelsTruncated] = useState(false);
const [loadingModels, setLoadingModels] = useState(false);
const [error, setError] = useState(null);
// True after a save/Test whose provider is still NOT the active one the
// save persisted fine but translation keeps using another provider, so be
// honest about it instead of letting a green Test read as "done" (#963).
const [savedInactive, setSavedInactive] = useState(false);
const current = useMemo(
() => providers.find((p) => p.id === editing) || null,
@@ -76,6 +80,7 @@ export default function LLMProvidersPanel() {
setTest(null);
setModels(null);
setModelsTruncated(false);
setSavedInactive(false);
}, []);
const refresh = useCallback(
@@ -127,7 +132,12 @@ export default function LLMProvidersPanel() {
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
await refresh(current.id);
const data = await refresh(current.id);
// Saved but another provider stays active say so (populate() above
// cleared the previous notice). Suppress while LLM_DEFAULT_PROVIDER
// pins the choice the env banner already explains and the suggested
// button is disabled.
setSavedInactive(Boolean(data) && data.active !== current.id && !current.active_from_env);
} catch (e) {
setError(e?.message || t('settings.llmp_save_failed'));
} finally {
@@ -411,6 +421,16 @@ export default function LLMProvidersPanel() {
</div>
}
/>
{savedInactive && (
<div
role="status"
data-testid="llm-not-active-notice"
className="text-[length:var(--text-xs)] text-[color:var(--chrome-fg-dim)] leading-[1.5] py-[var(--space-2)]"
>
{t('settings.llmp_saved_not_active')}
</div>
)}
</>
)}
</SettingsSection>
@@ -181,4 +181,35 @@ describe('LLMProvidersPanel', () => {
fireEvent.change(select, { target: { value: 'ollama' } });
await waitFor(() => expect(screen.queryByTestId('llm-provider-key')).toBeNull());
});
// #963 honesty: a green Test on a provider that is NOT the active one must
// say the provider isn't used for translation yet pre-fix the panel read
// as "done" while translation kept using another provider.
it('test-only flow on a non-active provider surfaces the not-yet-active notice', async () => {
global.fetch = mockFetchSequence(
{ body: PROVIDERS }, // mount GET (active: groq)
{ body: {} }, // save PUT (ollama, make_active:false)
{ body: PROVIDERS }, // refresh GET active is still groq
{ body: { ok: true, model: 'llama3', reply: 'ok', latency_ms: 9 } }, // test POST
);
render(<LLMProvidersPanel />);
const select = await screen.findByTestId('llm-provider-select');
fireEvent.change(select, { target: { value: 'ollama' } });
fireEvent.click(screen.getByTestId('llm-provider-test'));
await waitFor(() => expect(screen.getByTestId('llm-not-active-notice')).toBeInTheDocument());
expect(screen.getByText(/not yet used for translation/)).toBeInTheDocument();
});
it('no notice when the saved provider IS the active one', async () => {
global.fetch = mockFetchSequence(
{ body: PROVIDERS }, // mount GET (active: groq)
{ body: {} }, // save PUT (groq)
{ body: PROVIDERS }, // refresh GET groq active
{ body: { ok: true, model: 'llama-3.3-70b', reply: 'ok', latency_ms: 412 } }, // test POST
);
render(<LLMProvidersPanel />);
fireEvent.click(await screen.findByTestId('llm-provider-test'));
await waitFor(() => expect(screen.getByText(/llama-3\.3-70b · 412 ms/)).toBeInTheDocument());
expect(screen.queryByTestId('llm-not-active-notice')).toBeNull();
});
});
+143 -107
View File
@@ -687,109 +687,137 @@ export default function useDubWorkflow({
}
}, [dubJobId, dubSegments, setDubSegments]);
const handleTranslateAll = useCallback(async () => {
if (!dubSegments.length || !dubLangCode) return;
setIsTranslating(true);
// Root cause of the "sticky TRANSLATION FAILED banner": a new translate
// attempt never cleared the previous failure, so a stale 400 survived even
// a successful retry. Clear it up front — the whole class of translate/
// pipeline error banners should reset on the next relevant action.
setDubError('');
try {
const data = await dubTranslate({
segments: dubSegments.map((s) => ({
id: String(s.id),
text: s.text_original && s.text_original.trim() ? s.text_original : s.text,
target_lang: s.target_lang,
direction: s.direction || undefined,
slot_seconds: s.end != null && s.start != null ? s.end - s.start : undefined,
})),
target_lang: dubLangCode,
provider: translateProvider,
quality: translateQuality,
// #280: regional dialect — only sent when it matches the target
// language so a stale "es-AR" never rides on a French translate.
dialect: dialectMatchesLang(dubDialect, dubLangCode) ? dubDialect : undefined,
glossary: glossaryTerms.length
? glossaryTerms.map((t) => ({ source: t.source, target: t.target, note: t.note || '' }))
: undefined,
});
const translatedMap = {};
const errors = [];
(data.translated || []).forEach((t) => {
translatedMap[t.id] = t;
if (t.error) errors.push({ id: t.id, error: t.error });
});
setDubSegments(
dubSegments.map((s) => {
const hit = translatedMap[s.id];
if (!hit) return s;
return {
...s,
text: hit.text && hit.text.trim() ? hit.text : s.text,
translate_error: hit.error || undefined,
translate_literal: hit.literal || undefined,
translate_critique: hit.critique || undefined,
// Carry over the predicted compression ratio so the per-row
// badge + job-level compression warning can light up before
// the user clicks Generate Dub.
rate_ratio: hit.rate_ratio != null ? hit.rate_ratio : s.rate_ratio,
rate_error: hit.rate_error || s.rate_error,
};
}),
);
if (data.cinematic_skipped === 'no-llm-configured') {
toast(t('dub_workflow.cinematic_no_llm'), { icon: '️', duration: 8000 });
// #372: the backend fell back to Fast — reflect that in the toggle so
// the UI doesn't claim Cinematic while delivering Fast.
useAppStore.getState().setTranslateQuality?.('fast');
}
// #280: the user picked a dialect but the chosen engine can't honor it
// (Argos/NLLB/Google in Fast mode). Tell them how to make it count.
// #372: skip when the cinematic toast above already fired — both at once
// sent users in a circle ("pick Cinematic" ↔ "Cinematic needs an LLM").
if (
data.dialect &&
data.dialect_applied === false &&
data.cinematic_skipped !== 'no-llm-configured'
) {
toast(t('dub_workflow.dialect_not_applied'), { icon: '️', duration: 8000 });
}
if (errors.length) {
const unique = [...new Set(errors.map((e) => e.error))];
toast.error(
t('dub_workflow.translate_errors', {
errorCount: errors.length,
totalCount: data.translated.length,
firstError: unique[0].slice(0, 120),
// `langOverride` (optional ISO code string) is the multi-language batch path:
// the generate loop translates INTO each pick before dubbing it. No-arg calls
// (the Translate All button, the review checkpoint) behave exactly as before
// — the guard also shields the direct `onClick={handleTranslateAll}` usages,
// where the first argument is a click event, not a language.
// Resolves `true` when a translation landed in the segments, `false` when the
// request failed or nothing got translated — the batch loop skips generating
// that language rather than rendering a wrong-language track.
const handleTranslateAll = useCallback(
async (langOverride) => {
const targetLang =
typeof langOverride === 'string' && langOverride ? langOverride : dubLangCode;
// Snapshot segments at call time: inside the multi-language loop the
// click-time closure is stale after the previous pick's translate pass.
const segs = useAppStore.getState().dubSegments;
if (!segs.length || !targetLang) return false;
setIsTranslating(true);
// Root cause of the "sticky TRANSLATION FAILED banner": a new translate
// attempt never cleared the previous failure, so a stale 400 survived even
// a successful retry. Clear it up front — the whole class of translate/
// pipeline error banners should reset on the next relevant action.
setDubError('');
let ok = false;
try {
const data = await dubTranslate({
segments: segs.map((s) => ({
id: String(s.id),
text: s.text_original && s.text_original.trim() ? s.text_original : s.text,
target_lang: s.target_lang,
direction: s.direction || undefined,
slot_seconds: s.end != null && s.start != null ? s.end - s.start : undefined,
})),
target_lang: targetLang,
provider: translateProvider,
quality: translateQuality,
// #280: regional dialect — only sent when it matches the target
// language so a stale "es-AR" never rides on a French translate.
dialect: dialectMatchesLang(dubDialect, targetLang) ? dubDialect : undefined,
glossary: glossaryTerms.length
? glossaryTerms.map((t) => ({ source: t.source, target: t.target, note: t.note || '' }))
: undefined,
});
const translatedMap = {};
const errors = [];
(data.translated || []).forEach((t) => {
translatedMap[t.id] = t;
if (t.error) errors.push({ id: t.id, error: t.error });
});
setDubSegments((prev) =>
prev.map((s) => {
const hit = translatedMap[s.id];
if (!hit) return s;
const gotText = !!(hit.text && hit.text.trim());
return {
...s,
text: gotText ? hit.text : s.text,
// P1.2 — keep every language's translation, keyed by target.
// `text` stays the currently-shown language (legacy single-slot
// contract); switching the target language swaps from this map
// instead of destroying the previous language's work.
...(gotText ? { translations: { ...s.translations, [targetLang]: hit.text } } : {}),
translate_error: hit.error || undefined,
translate_literal: hit.literal || undefined,
translate_critique: hit.critique || undefined,
// Carry over the predicted compression ratio so the per-row
// badge + job-level compression warning can light up before
// the user clicks Generate Dub.
rate_ratio: hit.rate_ratio != null ? hit.rate_ratio : s.rate_ratio,
rate_error: hit.rate_error || s.rate_error,
};
}),
{ duration: 6000 },
);
} else {
const qLabel =
data.quality_used === 'cinematic' ? t('dub_workflow.translated_cinematic_suffix') : '';
toast.success(
t('dub_workflow.translated_segments', {
count: data.translated.length,
lang: data.target_lang,
}) + qLabel,
);
// "Translated" for the batch loop means at least one segment actually
// got new text — an empty result or an all-errors result would make
// the follow-up generate render the source language verbatim.
const total = (data.translated || []).length;
ok = total > 0 && errors.length < total;
if (data.cinematic_skipped === 'no-llm-configured') {
toast(t('dub_workflow.cinematic_no_llm'), { icon: '️', duration: 8000 });
// #372: the backend fell back to Fast — reflect that in the toggle so
// the UI doesn't claim Cinematic while delivering Fast.
useAppStore.getState().setTranslateQuality?.('fast');
}
// #280: the user picked a dialect but the chosen engine can't honor it
// (Argos/NLLB/Google in Fast mode). Tell them how to make it count.
// #372: skip when the cinematic toast above already fired — both at once
// sent users in a circle ("pick Cinematic" ↔ "Cinematic needs an LLM").
if (
data.dialect &&
data.dialect_applied === false &&
data.cinematic_skipped !== 'no-llm-configured'
) {
toast(t('dub_workflow.dialect_not_applied'), { icon: '️', duration: 8000 });
}
if (errors.length) {
const unique = [...new Set(errors.map((e) => e.error))];
toast.error(
t('dub_workflow.translate_errors', {
errorCount: errors.length,
totalCount: data.translated.length,
firstError: unique[0].slice(0, 120),
}),
{ duration: 6000 },
);
} else {
const qLabel =
data.quality_used === 'cinematic' ? t('dub_workflow.translated_cinematic_suffix') : '';
toast.success(
t('dub_workflow.translated_segments', {
count: data.translated.length,
lang: data.target_lang,
}) + qLabel,
);
}
} catch (err) {
setDubError(t('dub_workflow.translation_failed', { message: err.message }));
}
} catch (err) {
setDubError(t('dub_workflow.translation_failed', { message: err.message }));
}
setIsTranslating(false);
}, [
dubSegments,
dubLangCode,
dubDialect,
translateProvider,
translateQuality,
glossaryTerms,
setIsTranslating,
setDubSegments,
setDubError,
]);
setIsTranslating(false);
return ok;
},
[
dubLangCode,
dubDialect,
translateProvider,
translateQuality,
glossaryTerms,
setIsTranslating,
setDubSegments,
setDubError,
],
);
const handleDubGenerate = useCallback(
async (opts = {}) => {
@@ -801,8 +829,12 @@ export default function useDubWorkflow({
// here, overriding the store's single selection (which is stale inside the
// loop). Each run appends its track to the job's dubbed_tracks.
const langOv = opts.langOverride || null;
// Snapshot segments at call time, not click time: the multi-language
// loop awaits a translate pass right before each generate, and the
// click-time closure would still hold the pre-translation text.
const segs = useAppStore.getState().dubSegments;
setDubStep('generating');
setDubProgress({ current: 0, total: dubSegments.length, text: '' });
setDubProgress({ current: 0, total: segs.length, text: '' });
setDubError('');
const genLabel = regenOnly
? t('dub_workflow.regenerating', { count: regenOnly.length })
@@ -812,12 +844,12 @@ export default function useDubWorkflow({
.showPill('generating', genLabel, { cancellable: true, homeMode: 'dub' });
try {
const body = {
segment_ids: dubSegments.map((s) => String(s.id)),
segment_ids: segs.map((s) => String(s.id)),
regen_only: regenOnly,
// Generation inputs come from the shared helper so the stored
// fingerprints (seg_hashes) match what /tools/incremental recomputes
// later — see utils/segments.js (#281).
segments: dubSegments.map((s) => ({
segments: segs.map((s) => ({
start: s.start,
end: s.end,
gain: s.gain !== undefined && s.gain !== 1.0 ? s.gain : undefined,
@@ -890,17 +922,22 @@ export default function useDubWorkflow({
.map(([id]) => id);
setPreviewSegIds(previewIds);
}
// P1.3 — hashes belong to the track that just generated
// (the event carries its language), not to whatever the
// store's selection is when the stream drains.
const genLang = evt.language_code || body.language_code;
if (evt.seg_hashes && Object.keys(evt.seg_hashes).length > 0) {
setLastGenFingerprints(evt.seg_hashes);
setLastGenFingerprints(evt.seg_hashes, genLang);
} else {
try {
const plan = await apiPost('/tools/incremental', {
segments: dubSegments.map((s) => ({
segments: segs.map((s) => ({
id: String(s.id),
...segmentGenInputs(s),
})),
lang: genLang,
});
setLastGenFingerprints(plan.fingerprints || {});
setLastGenFingerprints(plan.fingerprints || {}, genLang);
} catch (err) {
console.warn('Incremental plan fallback failed:', err);
}
@@ -941,7 +978,6 @@ export default function useDubWorkflow({
},
[
dubJobId,
dubSegments,
dubLang,
dubLangCode,
dubInstruct,
+78 -8
View File
@@ -11,6 +11,10 @@ import { apiPost } from '../api/client';
import { segmentGenInputs } from '../utils/segments';
import { commitMoveResize } from '../utils/timeline';
// Stable empty map so `lastGenFingerprints` keeps a constant identity for a
// language with no stored hashes (avoids effect/callback churn).
const EMPTY_FINGERPRINTS = {};
export default function useSegmentEditing() {
const dubSegments = useAppStore((s) => s.dubSegments);
const setDubSegments = useAppStore((s) => s.setDubSegments);
@@ -50,7 +54,20 @@ export default function useSegmentEditing() {
const segmentEditField = useCallback(
(id, field, value) => {
pushUndo(dubSegments);
setDubSegments((prev) => prev.map((s) => (s.id === id ? { ...s, [field]: value } : s)));
// P1.2 — a manual text edit is a translation edit for the CURRENT
// target language: keep `translations[lang]` in lock-step with `text`
// so switching languages and back never loses the edit.
const lang = useAppStore.getState().dubLangCode;
setDubSegments((prev) =>
prev.map((s) => {
if (s.id !== id) return s;
const next = { ...s, [field]: value };
if (field === 'text' && lang) {
next.translations = { ...s.translations, [lang]: value };
}
return next;
}),
);
},
[dubSegments],
);
@@ -87,10 +104,22 @@ export default function useSegmentEditing() {
const segmentRestoreOriginal = useCallback(
(id) => {
pushUndo(dubSegments);
// "Use the original text for this row" is a per-language decision like
// any other text edit — record it under the current language so a
// round-trip through another language doesn't resurrect the discarded
// translation (P1.2).
const lang = useAppStore.getState().dubLangCode;
setDubSegments((prev) =>
prev.map((s) =>
s.id === id ? { ...s, text: s.text_original || s.text, translate_error: undefined } : s,
),
prev.map((s) => {
if (s.id !== id) return s;
const restored = s.text_original || s.text;
return {
...s,
text: restored,
...(lang ? { translations: { ...s.translations, [lang]: restored } } : {}),
translate_error: undefined,
};
}),
);
},
[dubSegments],
@@ -164,12 +193,16 @@ export default function useSegmentEditing() {
const pos = Math.max(1, Math.min(cursorPos, text.length - 1));
const ratio = text.length > 0 ? pos / text.length : 0.5;
const midT = seg.start + (seg.end - seg.start) * ratio;
// Other languages' saved texts (P1.2) can't be split at a sensible
// position for the halves — drop them; the halves are new segment ids
// that need fresh TTS per language anyway.
const left = {
...seg,
id: `${seg.id}_a`,
text: text.slice(0, pos).trim(),
end: midT,
text_original: text.slice(0, pos).trim(),
translations: undefined,
};
const right = {
...seg,
@@ -177,6 +210,7 @@ export default function useSegmentEditing() {
text: text.slice(pos).trim(),
start: midT,
text_original: text.slice(pos).trim(),
translations: undefined,
};
return [...prev.slice(0, idx), left, right, ...prev.slice(idx + 1)];
});
@@ -193,12 +227,24 @@ export default function useSegmentEditing() {
if (idx < 0 || idx >= prev.length - 1) return prev;
const a = prev[idx];
const b = prev[idx + 1];
// Merge per-language texts (P1.2) only where BOTH sides carry the
// language — a half-known language would otherwise mix two languages
// in one entry. Missing entries just mean "translate again".
const ta = a.translations || {};
const tb = b.translations || {};
const mergedTranslations = {};
for (const lang of Object.keys(ta)) {
if (typeof ta[lang] === 'string' && typeof tb[lang] === 'string') {
mergedTranslations[lang] = `${ta[lang]} ${tb[lang]}`.trim();
}
}
const merged = {
...a,
text: `${a.text || ''} ${b.text || ''}`.trim(),
text_original:
`${a.text_original || a.text || ''} ${b.text_original || b.text || ''}`.trim(),
end: b.end,
translations: Object.keys(mergedTranslations).length ? mergedTranslations : undefined,
};
return [...prev.slice(0, idx), merged, ...prev.slice(idx + 2)];
});
@@ -221,8 +267,25 @@ export default function useSegmentEditing() {
[directionSegId, dubSegments],
);
// Incremental plan — tracks which segments changed since last generate
const [lastGenFingerprints, setLastGenFingerprints] = useState({});
// Incremental plan — tracks which segments changed since last generate.
// P1.3: fingerprints are stored PER LANGUAGE ({ lang: { segId: hash } }),
// and `lastGenFingerprints` is the ACTIVE language's map — so "Regen N
// changed" is judged against the track you're looking at, never against
// whichever language happened to generate last. Switching to a language
// that was never generated yields an empty map → no plan (no false
// "all fresh" / "all stale" claims).
const dubLangCode = useAppStore((s) => s.dubLangCode);
const [fingerprintsByLang, setFingerprintsByLang] = useState({});
const lastGenFingerprints = fingerprintsByLang[dubLangCode] || EMPTY_FINGERPRINTS;
// Same call signature as before for existing single-track callers; the
// optional `lang` pins the map to the track that produced the hashes
// (e.g. each pick of the multi-language batch loop) instead of whatever
// the store's selection is by the time the response lands.
const setLastGenFingerprints = useCallback((map, lang) => {
const key = lang || useAppStore.getState().dubLangCode;
if (!key) return;
setFingerprintsByLang((prev) => ({ ...prev, [key]: map || {} }));
}, []);
const [incrementalPlan, setIncrementalPlan] = useState(null);
const recomputeIncremental = useCallback(async () => {
@@ -232,16 +295,19 @@ export default function useSegmentEditing() {
}
try {
// Same payload shape as the generate request (utils/segments.js) so
// stored fingerprints actually match unchanged segments (#281).
// stored fingerprints actually match unchanged segments (#281). `lang`
// must match the language the generate run hashed with — it's part of
// the fingerprint now (P1.3).
const res = await apiPost('/tools/incremental', {
segments: dubSegments.map((s) => ({ id: String(s.id), ...segmentGenInputs(s) })),
stored_hashes: lastGenFingerprints,
lang: dubLangCode,
});
setIncrementalPlan({ stale: res.stale, fresh: res.fresh });
} catch (e) {
console.warn('incremental plan failed', e);
}
}, [dubSegments, lastGenFingerprints]);
}, [dubSegments, lastGenFingerprints, dubLangCode]);
return {
// Undo/Redo
@@ -275,6 +341,10 @@ export default function useSegmentEditing() {
// Incremental plan
lastGenFingerprints,
setLastGenFingerprints,
// Per-language fingerprint store (P1.3) — for project save/load and dub
// history restore, which persist/rehydrate ALL tracks' hashes at once.
fingerprintsByLang,
setFingerprintsByLang,
incrementalPlan,
setIncrementalPlan,
recomputeIncremental,
+24
View File
@@ -348,6 +348,7 @@
"llmp_save": "Save",
"llmp_save_active": "Save & use for translation",
"llmp_save_keep": "Save & keep active",
"llmp_saved_not_active": "Saved — not yet used for translation. Click “Save & use for translation” to switch.",
"llmp_test": "Test",
"llmp_active_badge": "active",
"llmp_test_ok": "ok — {{model}} · {{ms}} ms",
@@ -370,6 +371,8 @@
"llmskills_open_providers": "Configure providers",
"llmskills_load_failed": "Failed to load LLM skills",
"llmskills_save_failed": "Failed to save",
"llmskills_dub_translation_name": "Dub translation",
"llmskills_dub_translation_desc": "Translates dub lines directly with your configured LLM provider (the Dub tab's 'LLM' engine). Off: the LLM engine is unavailable — pick another engine.",
"llmskills_cinematic_translation_name": "Cinematic & Autofit translation",
"llmskills_cinematic_translation_desc": "Rewrites each translated line for natural, in-character delivery (and Autofit's time budget). Off: dubs use the Fast translation result.",
"llmskills_slot_fitting_name": "Speech-rate slot fitting",
@@ -890,6 +893,11 @@
"burn_subs_title": "Render subtitles directly into the MP4 video stream (hardsubs). Uses the dual-subtitle format when Dual subtitles is on.",
"timing_smart_fit": "Smart Fit",
"timing_smart_fit_title": "Splits the difference: slightly speeds up the audio (pitch preserved, up to 1.5×) and slightly slows down that segment of the video (up to 2×) so natural-rate speech fits. Anything beyond the caps is trimmed and flagged. Export re-encodes the video.",
"timing_concise": "Concise",
"timing_stretch_video": "Stretch Video",
"timing_strict_slot": "Strict slot",
"track_tip_duration": "Duration {{duration}}",
"track_tip_timing": "Timing {{strategy}}",
"default_track": "Default Track:",
"original_track": "Original",
"selected_dub": "{{code}} (Selected Dub)",
@@ -920,6 +928,8 @@
"hq_needs_llm_hint": "High-quality translation fits each line to its segment time using a local or cloud LLM. Set one up to enable it.",
"set_up_llm": "Set up",
"generate_dub_multi": "Generate {{count}} dubs",
"multi_translating": "Translating → {{lang}} ({{current}}/{{total}})…",
"multi_lang_skipped": "Translation failed for {{langs}} — those dubs were skipped so you never get a wrong-language track.",
"pipeline": "Dubbing pipeline",
"preview_language": "Preview language",
"target_language": "Dub into",
@@ -1589,6 +1599,20 @@
"searchIssues": "Search similar issues",
"unexpected": "Unexpected error: {{message}}"
},
"crash": {
"notice": "The voice backend crashed ({{exit}}) {{ago}} ago and is being restarted automatically.",
"view": "View crash details",
"dismiss": "Dismiss",
"details_title": "Backend crash details",
"details_intro": "The backend process died unexpectedly ({{exit}}) {{ago}} ago. The error output it left behind is below — reporting it helps us fix the crash.",
"field_exit": "Exit",
"field_when": "When",
"field_uptime": "Uptime before crash",
"field_version": "Backend version",
"uptime_value": "{{count}} s",
"stderr_title": "Last error output (stderr)",
"no_stderr": "No error output was captured."
},
"common": {
"open": "Open",
"cancel": "Cancel",
+3 -13
View File
@@ -26,6 +26,7 @@ import BatchAddDialog from '../components/BatchAddDialog';
import toast from 'react-hot-toast';
import { toastErrorWithReport } from '../utils/errorToast';
import { recordValueMoment } from '../utils/donationMoments';
import { absoluteTime, timeAgo } from '../utils/relativeTime';
/**
* BatchQueue UI for the /batch/* dubbing pipeline.
@@ -261,7 +262,7 @@ function JobCard({ job, onCancel, onDelete, t }) {
const st = STATUS_TONE[job.status] || STATUS_TONE.queued;
const StIcon = st.icon;
const ageLabel = formatAge((Date.now() / 1000 - (job.created_at || 0)) * 1000);
const ageLabel = timeAgo(job.created_at);
const duration =
job.finished_at && job.started_at ? Math.max(0, job.finished_at - job.started_at) : null;
@@ -287,7 +288,7 @@ function JobCard({ job, onCancel, onDelete, t }) {
<span className="batch-queue__card-spacer flex-1" />
<span
className="batch-queue__card-age text-[var(--text-xs)] text-fg-subtle [font-variant-numeric:tabular-nums]"
title={new Date((job.created_at || 0) * 1000).toLocaleString()}
title={absoluteTime(job.created_at)}
>
{ageLabel}
</span>
@@ -394,17 +395,6 @@ function JobCard({ job, onCancel, onDelete, t }) {
);
}
function formatAge(ms) {
if (!isFinite(ms) || ms < 0) return '—';
const s = Math.floor(ms / 1000);
if (s < 60) return `${s}s ago`;
const m = Math.floor(s / 60);
if (m < 60) return `${m}m ago`;
const h = Math.floor(m / 60);
if (h < 24) return `${h}h ago`;
return new Date(Date.now() - ms).toLocaleDateString();
}
function formatDuration(secs) {
if (secs < 60) return `${secs.toFixed(1)}s`;
const m = Math.floor(secs / 60);
+93 -16
View File
@@ -84,7 +84,12 @@ export default function DubTab(props) {
const dubLang = useAppStore((s) => s.dubLang);
const setDubLang = useAppStore((s) => s.setDubLang);
const dubLangCode = useAppStore((s) => s.dubLangCode);
const setDubLangCode = useAppStore((s) => s.setDubLangCode);
// User-driven language switches go through switchDubLangCode (P1.2): it
// swaps segment text through the per-language `translations` map instead
// of leaving the previous language's text on screen (and previously,
// letting the next translate destroy it). Non-user rehydration paths
// (project load, history restore) keep the plain setter.
const switchDubLangCode = useAppStore((s) => s.switchDubLangCode);
const dubNumSpeakers = useAppStore((s) => s.dubNumSpeakers);
const setDubNumSpeakers = useAppStore((s) => s.setDubNumSpeakers);
const dubDialect = useAppStore((s) => s.dubDialect);
@@ -188,31 +193,96 @@ export default function DubTab(props) {
const [exportOpen, setExportOpen] = useState(false);
const [qcRunning, setQcRunning] = useState(false);
// Multi-language mode
const [multiLangMode, setMultiLangMode] = useState(false);
const [multiLangs, setMultiLangs] = useState([]);
// Multi-language mode store-backed (P1.4) so the picks survive tab
// switches and ride the project save/load payload.
const multiLangMode = useAppStore((s) => s.multiLangMode);
const setMultiLangMode = useAppStore((s) => s.setMultiLangMode);
const multiLangs = useAppStore((s) => s.multiLangs);
const setMultiLangs = useAppStore((s) => s.setMultiLangs);
// Landing "Advanced" disclosure (pre-upload options).
const [landingAdvOpen, setLandingAdvOpen] = useState(false);
// Generate CTA when multi-language mode has picks, dub each language
// sequentially; every run appends its track to dubbed_tracks, so the
// preview switcher pills fill up one by one.
//
// P1.1: each language is TRANSLATED first (`handleTranslateAll(code)`), then
// generated the backend synthesizes segment text verbatim, so without the
// translate pass every "multi-language" track rendered the same words.
// A pick whose translate fails is skipped (never render a wrong-language
// track); the batch continues and the skips are reported at the end.
const multiBatchRunningRef = useRef(false);
const onGenerateClick = useCallback(async () => {
if (multiLangMode && multiLangs.length > 0) {
if (multiBatchRunningRef.current) return; // ignore re-clicks mid-batch
multiBatchRunningRef.current = true;
const skipped = [];
// Skip the redundant translate ONLY for the first pick, and only when
// it targets the language the editor text is already in (every segment
// carries a translation differing from its original i.e. the user
// just ran Translate All into this exact language). After the first
// pick the editor text is the previous pick's language, so every later
// pick always translates. Correctness beats cleverness.
const editorAlreadyTranslated =
dubSegments.length > 0 &&
dubSegments.every((s) => s.text_original && s.text !== s.text_original);
try {
for (const l of multiLangs) {
for (let i = 0; i < multiLangs.length; i++) {
const l = multiLangs[i];
setDubLang(l.lang);
setDubLangCode(l.code); // keep UI/exports in sync
// Keep UI/exports in sync AND snapshot the previous pick's
// translations before this pick's translate pass overwrites the
// visible text (P1.2).
switchDubLangCode(l.code);
const skipTranslate = i === 0 && l.code === dubLangCode && editorAlreadyTranslated;
if (!skipTranslate) {
// Honest phase label: this pill slot otherwise only says
// "Generating", hiding the translate pass entirely.
useAppStore.getState().showPill(
'translating',
t('dub.multi_translating', {
lang: l.lang,
current: i + 1,
total: multiLangs.length,
}),
{ homeMode: 'dub' },
);
// eslint-disable-next-line no-await-in-loop
const ok = await handleTranslateAll(l.code);
if (!ok) {
// Error already surfaced by handleTranslateAll (banner/toast);
// drop the phase pill and move on to the next language.
useAppStore.getState().dismissPill();
skipped.push(l.lang);
continue;
}
}
// eslint-disable-next-line no-await-in-loop
await handleDubGenerate({ langOverride: { language: l.lang, language_code: l.code } });
}
} catch {
/* a failed language stops the batch; its error is already surfaced */
}
multiBatchRunningRef.current = false;
if (skipped.length) {
toast.error(t('dub.multi_lang_skipped', { langs: skipped.join(', ') }), {
duration: 8000,
});
}
} else {
handleDubGenerate();
}
}, [multiLangMode, multiLangs, handleDubGenerate, setDubLang, setDubLangCode]);
}, [
multiLangMode,
multiLangs,
dubSegments,
dubLangCode,
handleTranslateAll,
handleDubGenerate,
setDubLang,
switchDubLangCode,
t,
]);
// Live ETA while generating elapsed ticks each second; remaining is
// extrapolated from the current/total rate so it's only meaningful once
@@ -349,11 +419,12 @@ export default function DubTab(props) {
});
setIngestUrl('');
};
const hasDubbedTrack =
dubStep === 'done' &&
dubLangCode &&
dubLangCode !== 'und' &&
(dubTracks?.length > 0 || !!dubTracks);
// Track-switcher visibility is keyed to the persisted tracks ONLY not the
// language dropdown. Restored projects can carry finished tracks while
// dubLangCode reads 'und' (older dub_history rows froze language_code at
// ""), and the old `dubLangCode !== 'und'` guard hid their tabs until the
// user re-picked a language.
const hasDubbedTrack = dubStep === 'done' && dubTracks.length > 0;
// Cache-busting nonce, bumped every time a generation completes (see
// useDubWorkflow's done handler). The preview URL is otherwise identical
// across re-dubs, so the WebView could keep serving the previously
@@ -367,9 +438,14 @@ export default function DubTab(props) {
: `${API}/dub/media/${dubJobId}`;
// When a dub finishes, jump the preview to the freshly-dubbed language so the
// result plays immediately the user can tap back to Original any time.
// Membership guard: only jump to a language that actually has a track,
// otherwise fall back to the first track. Restored projects can have
// dubLangCode out of sync with the tracks (e.g. 'en'/'und' with tracks
// ['bn']) and an unguarded jump would point the player at
// /dub/preview-video?lang=en a guaranteed 404.
useEffect(() => {
if (hasDubbedTrack && previewMode === 'original' && dubLangCode && dubLangCode !== 'und') {
setPreviewMode(dubLangCode);
if (hasDubbedTrack && previewMode === 'original') {
setPreviewMode(dubTracks.includes(dubLangCode) ? dubLangCode : dubTracks[0]);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [hasDubbedTrack, dubLangCode]);
@@ -478,7 +554,7 @@ export default function DubTab(props) {
fetchYtSubs={fetchYtSubs}
setFetchYtSubs={setFetchYtSubs}
dubLangCode={dubLangCode}
setDubLangCode={setDubLangCode}
setDubLangCode={switchDubLangCode}
setDubLang={setDubLang}
landingAdvOpen={landingAdvOpen}
setLandingAdvOpen={setLandingAdvOpen}
@@ -502,6 +578,7 @@ export default function DubTab(props) {
handleDubStop={handleDubStop}
dubProgress={dubProgress}
onGenerateClick={onGenerateClick}
isTranslating={isTranslating}
multiLangMode={multiLangMode}
multiLangs={multiLangs}
incrementalPlan={incrementalPlan}
@@ -550,7 +627,7 @@ export default function DubTab(props) {
hasAnyTranslation={hasAnyTranslation}
handleCleanupSegments={handleCleanupSegments}
setDubLang={setDubLang}
setDubLangCode={setDubLangCode}
setDubLangCode={switchDubLangCode}
dubDialect={dubDialect}
setDubDialect={setDubDialect}
i18n={i18n}
+16 -21
View File
@@ -18,6 +18,7 @@ import {
BookOpen,
} from 'lucide-react';
import { apiFetch } from '../api/client';
import { timeAgo, toMillis } from '../utils/relativeTime';
import { loadTranscriptions, TRANSCRIPTION_EVENT } from '../utils/transcriptionsStore';
import { audioUrl } from '../api/generate';
import { playBlobAudio } from '../utils/media';
@@ -41,18 +42,6 @@ import { playBlobAudio } from '../utils/media';
* this page stays in sync with the Sidebar and Launchpad automatically.
*/
function fmtTime(ts) {
if (!ts) return '';
const d = typeof ts === 'number' ? ts : Date.parse(ts);
if (!Number.isFinite(d)) return '';
const diff = Date.now() - d;
const s = Math.floor(diff / 1000);
if (s < 60) return `${s}s ago`;
if (s < 3600) return `${Math.floor(s / 60)}m ago`;
if (s < 86400) return `${Math.floor(s / 3600)}h ago`;
return `${Math.floor(s / 86400)}d ago`;
}
function fmtDuration(sec) {
if (!sec) return '';
const n = Number(sec);
@@ -177,7 +166,10 @@ export default function Projects({
}, []);
// Normalise every source into a common shape so the filter + search +
// sort pipeline is identical regardless of origin.
// sort pipeline is identical regardless of origin. Timestamps arrive in
// mixed units (backend rows: Unix seconds; story projects: Date.now() ms;
// transcriptions: ISO strings) toMillis() normalizes them all to epoch
// ms (or 0 when missing) so both the sort and timeAgo() stay unit-safe.
const items = useMemo(() => {
const list = [];
for (const p of studioProjects) {
@@ -186,7 +178,7 @@ export default function Projects({
id: p.id,
title: p.name || p.video_path?.split('/').pop() || p.id,
subtitle: fmtDuration(p.duration),
ts: (p.updated_at || p.created_at || 0) * 1000,
ts: toMillis(p.updated_at || p.created_at) ?? 0,
accent: '#fe8019',
Icon: Film,
onClick: () => onOpenDub?.(p.id),
@@ -205,7 +197,7 @@ export default function Projects({
]
.filter(Boolean)
.join(' · '),
ts: sp.updatedAt || 0,
ts: toMillis(sp.updatedAt) ?? 0,
accent: '#83a598',
Icon: BookOpen,
onClick: () => onOpenStory?.(sp.id),
@@ -218,7 +210,7 @@ export default function Projects({
id: pr.id,
title: pr.name || pr.id,
subtitle: kind === 'design' ? t('projects.designed_voice') : t('projects.cloned_voice'),
ts: (pr.updated_at || pr.created_at || 0) * 1000,
ts: toMillis(pr.updated_at || pr.created_at) ?? 0,
accent: kind === 'design' ? '#8ec07c' : '#d3869b',
Icon: kind === 'design' ? Wand2 : Fingerprint,
onClick: () => onOpenProfile?.(pr.id),
@@ -230,7 +222,10 @@ export default function Projects({
id: h.filename || h.id || String(Math.random()),
title: (h.text || h.prompt || h.filename || t('projects.generated_audio')).slice(0, 80),
subtitle: h.language || h.voice || '',
ts: h.timestamp || h.created_at || 0,
// #epoch-bug regression site: generation_history rows carry created_at
// in Unix SECONDS feeding them to a ms-based diff rendered every
// history card as "20617d ago" (1970) and sorted them last.
ts: toMillis(h.timestamp || h.created_at) ?? 0,
accent: '#f3a5b6',
Icon: Music,
onClick: undefined,
@@ -242,7 +237,7 @@ export default function Projects({
id: e.path || e.id,
title: e.path?.split('/').pop() || e.filename || t('projects.export'),
subtitle: e.mode || '',
ts: (e.created_at || 0) * 1000,
ts: toMillis(e.created_at) ?? 0,
accent: '#fabd2f',
Icon: Download,
onClick: () => e.path && onRevealExport?.(e.path),
@@ -261,7 +256,7 @@ export default function Projects({
]
.filter(Boolean)
.join(' · '),
ts: (j.created_at || 0) * 1000,
ts: toMillis(j.created_at) ?? 0,
accent: '#d3869b',
Icon: BookMarked,
onClick: () => j.output && playRenderInApp(audioUrl(j.output)),
@@ -275,7 +270,7 @@ export default function Projects({
subtitle: [tr.language, tr.duration_s ? `${Math.round(tr.duration_s)}s` : '']
.filter(Boolean)
.join(' · '),
ts: tr.timestamp ? Date.parse(tr.timestamp) : 0,
ts: toMillis(tr.timestamp) ?? 0,
accent: '#83a598',
Icon: FileText,
onClick: () => {
@@ -410,7 +405,7 @@ export default function Projects({
trailing={
<span className="inline-flex items-center gap-[3px] [font-family:var(--chrome-font-mono)]">
<Clock size={10} />
{fmtTime(it.ts)}
{timeAgo(it.ts)}
</span>
}
onClick={it.onClick}
+7 -3
View File
@@ -12,6 +12,7 @@ import { useTranslation } from 'react-i18next';
import { Mic, Copy, Trash2, Search, Clock, Languages, FileText, Download } from 'lucide-react';
import { Button } from '../ui';
import { toast } from 'react-hot-toast';
import { toMillis } from '../utils/relativeTime';
import {
loadTranscriptions,
TRANSCRIPTIONS_KEY,
@@ -108,10 +109,13 @@ export default function TranscriptionsPage() {
toast.success(t('transcriptions.exported'));
}, [transcriptions]);
// toMillis keeps this unit-safe (ISO strings today; seconds/ms tolerated)
// and guards unparseable stamps, which used to render "Invalid Date".
const formatTime = (iso) => {
const d = new Date(iso);
const now = new Date();
const diff = now - d;
const ms = toMillis(iso);
if (ms == null) return null;
const d = new Date(ms);
const diff = Date.now() - ms;
if (diff < 60000) return t('transcriptions.just_now');
if (diff < 3600000) return t('transcriptions.m_ago', { count: Math.floor(diff / 60000) });
if (diff < 86400000) return t('transcriptions.h_ago', { count: Math.floor(diff / 3600000) });
+56
View File
@@ -45,6 +45,12 @@ interface DubPrepProgress {
/** Segments are a loose shape — many optional fields added over time. */
type DubSegment = Record<string, unknown> & { id: string; text: string };
/** One multi-language batch pick — display name + ISO code (MultiLangPicker). */
export interface MultiLangPick {
lang: string;
code: string;
}
type Updater<T> = T | ((prev: T) => T);
function resolve<T>(updater: Updater<T>, prev: T): T {
@@ -103,6 +109,13 @@ export interface DubSlice {
// paths (OpenAI/Ollama provider or Cinematic quality).
dubDialect: string;
// Multi-language batch mode (P1.4) — the checkbox + language picks used by
// the "Generate N dubs" loop. Lived in DubTab component state before, so a
// tab switch or project reload silently dropped the picks; now they ride
// the store and the project save/load payload.
multiLangMode: boolean;
multiLangs: MultiLangPick[];
// ── Generation options ────────────────────────────────────────────────
dubInstruct: string;
preserveBg: boolean;
@@ -156,8 +169,21 @@ export interface DubSlice {
bumpDubGenNonce: () => void;
setDubLang: (v: Updater<string>) => void;
setDubLangCode: (v: Updater<string>) => void;
/**
* User-driven target-language switch (P1.2). Unlike the plain setter it
* also remaps segment text through the per-language `translations` store:
* the outgoing language's text is snapshotted into `translations[prev]`
* (only when it's an actual translation differs from `text_original`),
* and the incoming language's saved text is swapped into `text` when one
* exists. Non-destructive: with no saved entry, `text` is left untouched
* exactly the legacy behaviour. Restore/rehydrate paths (project load, dub
* history) must keep using `setDubLangCode`, which never touches segments.
*/
switchDubLangCode: (code: string) => void;
setDubNumSpeakers: (v: Updater<number | null>) => void;
setDubDialect: (v: Updater<string>) => void;
setMultiLangMode: (v: Updater<boolean>) => void;
setMultiLangs: (v: Updater<MultiLangPick[]>) => void;
setDubInstruct: (v: Updater<string>) => void;
setPreserveBg: (v: Updater<boolean>) => void;
setDefaultTrack: (v: Updater<string>) => void;
@@ -190,8 +216,11 @@ const INITIAL: Omit<
| 'bumpDubGenNonce'
| 'setDubLang'
| 'setDubLangCode'
| 'switchDubLangCode'
| 'setDubNumSpeakers'
| 'setDubDialect'
| 'setMultiLangMode'
| 'setMultiLangs'
| 'setDubInstruct'
| 'setPreserveBg'
| 'setDefaultTrack'
@@ -223,6 +252,8 @@ const INITIAL: Omit<
dubLangCode: 'en',
dubNumSpeakers: null,
dubDialect: '',
multiLangMode: false,
multiLangs: [],
dubInstruct: '',
preserveBg: true,
defaultTrack: 'original',
@@ -255,8 +286,33 @@ export const createDubSlice: StateCreator<DubSlice, [], [], DubSlice> = (set, ge
bumpDubGenNonce: () => set(() => ({ dubGenNonce: Date.now() })),
setDubLang: (v) => set((s) => ({ dubLang: resolve(v, s.dubLang) })),
setDubLangCode: (v) => set((s) => ({ dubLangCode: resolve(v, s.dubLangCode) })),
switchDubLangCode: (code) =>
set((s) => {
const prev = s.dubLangCode;
if (!code || code === prev) return {};
const dubSegments = s.dubSegments.map((seg) => {
const translations: Record<string, string> = {
...(seg.translations as Record<string, string> | undefined),
};
// Snapshot the outgoing language's text — but only real translations
// (differs from the source), so a never-translated row can't stamp
// source-language text as the previous language's translation.
// Legacy projects (no `translations` yet) get theirs seeded here.
const text = typeof seg.text === 'string' ? seg.text : '';
if (prev && text.trim() && text !== seg.text_original) translations[prev] = text;
const incoming = translations[code];
return {
...seg,
translations,
...(typeof incoming === 'string' && incoming.trim() ? { text: incoming } : {}),
};
});
return { dubLangCode: code, dubSegments };
}),
setDubNumSpeakers: (v) => set((s) => ({ dubNumSpeakers: resolve(v, s.dubNumSpeakers) })),
setDubDialect: (v) => set((s) => ({ dubDialect: resolve(v, s.dubDialect) })),
setMultiLangMode: (v) => set((s) => ({ multiLangMode: resolve(v, s.multiLangMode) })),
setMultiLangs: (v) => set((s) => ({ multiLangs: resolve(v, s.multiLangs) })),
setDubInstruct: (v) => set((s) => ({ dubInstruct: resolve(v, s.dubInstruct) })),
setPreserveBg: (v) => set((s) => ({ preserveBg: resolve(v, s.preserveBg) })),
setDefaultTrack: (v) => set((s) => ({ defaultTrack: resolve(v, s.defaultTrack) })),
@@ -0,0 +1,126 @@
import React, { createRef } from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/react';
import i18n from '../i18n';
// P0.2 track-bar polish: the Original/<lang> pills enrich with per-track
// metadata (duration + timing strategy) hydrated lazily from the existing
// GET /dub/tracks/{job_id}. The fetch must be failure-silent: the pills are
// the P0 visibility fix and can never depend on the enrichment call.
vi.mock('../components/WaveformTimeline', () => ({ default: () => <div data-testid="wf" /> }));
vi.mock('../components/MultiLangPicker', () => ({ default: () => <div data-testid="mlp" /> }));
vi.mock('react-hot-toast', () => ({
default: { error: vi.fn(), success: vi.fn(), loading: vi.fn() },
}));
const dubListTracks = vi.hoisted(() => vi.fn());
vi.mock('../api/dub', () => ({ dubListTracks: (...a) => dubListTracks(...a) }));
import DubLeftColumn from '../components/dub/DubLeftColumn';
const t = i18n.t.bind(i18n);
function makeProps(over = {}) {
return {
hasDubbedTrack: true,
t,
i18n,
previewMode: 'bn',
setPreviewMode: vi.fn(),
dubTracks: ['bn'],
videoSrc: '',
waveformRef: createRef(),
dubJobId: 'job1',
dubSegments: [{ id: '1', text: 'hi' }],
timelineOnsets: [],
timelineSelSegId: null,
setTimelineSelSegId: vi.fn(),
incrementalPlan: null,
segmentMoveResize: vi.fn(),
segmentDelete: vi.fn(),
onTimelinePreviewSegment: vi.fn(),
dubStep: 'done',
dubProgress: { current: 0, total: 0, text: '' },
fmtDur: (s) => `${s}s`,
genElapsed: 0,
genRemaining: null,
speakerClones: {},
setDubSegments: vi.fn(),
profiles: [],
settingsOpen: false,
setSettingsOpen: vi.fn(),
dubLang: 'Bengali',
dubLangCode: 'bn',
translateQuality: 'fast',
activeEngineUnavailable: false,
translateProvider: 'google',
dubInstruct: '',
setDubInstruct: vi.fn(),
handleTranslateAll: vi.fn(),
isTranslating: false,
hasAnyTranslation: false,
handleCleanupSegments: vi.fn(),
setDubLang: vi.fn(),
setDubLangCode: vi.fn(),
dubDialect: '',
setDubDialect: vi.fn(),
enginesSandboxed: false,
handleInstallEngine: vi.fn(),
engineInstalling: null,
activeEngineEntry: undefined,
engines: [],
setTranslateProvider: vi.fn(),
setTranslateQuality: vi.fn(),
llmEndpoint: { available: true },
multiLangMode: false,
setMultiLangMode: vi.fn(),
multiLangs: [],
setMultiLangs: vi.fn(),
editSegments: vi.fn(),
...over,
};
}
describe('DubLeftColumn — track pill tooltips (P0.2)', () => {
beforeEach(() => {
dubListTracks.mockReset();
});
it('hydrates duration + timing strategy from /dub/tracks and reflects the previewed track', async () => {
dubListTracks.mockResolvedValue({
bn: { duration: 72.4, timing_strategy: 'smart_fit', language: 'Bengali' },
});
render(<DubLeftColumn {...makeProps()} />);
const pill = screen.getByRole('radio', { name: 'Bengali' });
// Selection indicator must track previewMode (accurate post-restore).
expect(pill).toHaveAttribute('aria-checked', 'true');
expect(screen.getByRole('radio', { name: t('dub.original_audio') })).toHaveAttribute(
'aria-checked',
'false',
);
expect(dubListTracks).toHaveBeenCalledWith('job1');
await waitFor(() => expect(pill).toHaveAttribute('title', 'Duration 72s · Timing Smart Fit'));
});
it('is failure-silent: a failed metadata fetch leaves the pills fully usable', async () => {
dubListTracks.mockRejectedValue(new Error('boom'));
render(<DubLeftColumn {...makeProps()} />);
expect(dubListTracks).toHaveBeenCalledWith('job1');
const pill = await screen.findByRole('radio', { name: 'Bengali' });
await waitFor(() => expect(dubListTracks).toHaveBeenCalled());
expect(pill).not.toHaveAttribute('title');
});
it('does not call the endpoint when there are no dubbed tracks', () => {
render(
<DubLeftColumn
{...makeProps({ hasDubbedTrack: false, dubTracks: [], dubStep: 'editing' })}
/>,
);
expect(screen.queryByRole('radiogroup')).not.toBeInTheDocument();
expect(dubListTracks).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,149 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render } from '@testing-library/react';
import { useAppStore } from '../store';
// Regression guard for the "completed dub tracks' tabs hidden until the
// language is re-selected" P0:
// - `hasDubbedTrack` must key off the persisted tracks ONLY. The old
// expression required `dubLangCode !== 'und'` and ended in a tautology
// (`dubTracks?.length > 0 || !!dubTracks`), so a restored project with
// finished tracks but a frozen language_code ('und') hid the track
// switcher and a job with NO tracks showed it.
// - The done-state auto-jump must be membership-guarded: jumping the preview
// to a dubLangCode that has no track (restores fall back to 'en' with
// tracks ['bn']) pointed the player at /dub/preview-video?lang=en 404.
// Heavy children are stubbed; DubLeftColumn is the probe DubTab owns both
// `hasDubbedTrack` and the previewMode auto-jump, and hands them down as props.
const captured = vi.hoisted(() => ({ left: [] }));
vi.mock('../components/dub/DubLeftColumn', () => ({
default: (props) => {
captured.left.push(props);
return <div data-testid="left-col" />;
},
}));
vi.mock('../components/dub/DubHeader', () => ({ default: () => null }));
vi.mock('../components/dub/DubRightColumn', () => ({ default: () => null }));
vi.mock('../components/dub/DubFooter', () => ({ default: () => null }));
vi.mock('../components/dub/DubPipelineStepper', () => ({ default: () => null }));
vi.mock('../components/dub/IdleSkeleton', () => ({ default: () => null }));
vi.mock('../components/ExportModal', () => ({ default: () => null }));
vi.mock('../hooks/useTimelineOnsets', () => ({ default: () => ({ onsets: [] }) }));
vi.mock('../api/dub', () => ({
dubQc: vi.fn(),
dubListTracks: vi.fn(() => new Promise(() => {})),
}));
// Never-resolving async deps keep the render synchronous (no post-test act noise).
vi.mock('../api/engines', () => ({
listTranslationEngines: vi.fn(() => new Promise(() => {})),
installTranslationEngine: vi.fn(),
}));
vi.mock('../api/client', async (importOriginal) => {
const mod = await importOriginal();
return { ...mod, apiJson: vi.fn(() => new Promise(() => {})) };
});
import DubTab from '../pages/DubTab';
const noop = () => {};
function makeProps() {
return {
dubVideoFile: null,
dubLocalBlobUrl: null,
transcribeElapsed: 0,
translateProvider: 'google',
setTranslateProvider: noop,
showTranscript: false,
setShowTranscript: noop,
onGlossaryChange: noop,
profiles: [],
segmentPreviewLoading: null,
selectedSegIds: new Set(),
setDubVideoFile: noop,
setDubLocalBlobUrl: noop,
handleDubAbort: noop,
handleDubUpload: noop,
handleDubIngestUrl: noop,
handleDubRetryTranscribe: noop,
handleDubStop: noop,
handleDubGenerate: noop,
handleDubImportSrt: noop,
handleDubDownload: noop,
handleDubAudioDownload: noop,
handleAudioExport: noop,
handleSegmentPreview: noop,
onDirectSegment: noop,
handleTranslateAll: noop,
handleCleanupSegments: noop,
incrementalPlan: null,
triggerDownload: noop,
fileToMediaUrl: noop,
editSegments: noop,
saveProject: noop,
resetDub: noop,
segmentEditField: noop,
segmentDelete: noop,
segmentRestoreOriginal: noop,
segmentSplit: noop,
segmentMerge: noop,
segmentMoveResize: noop,
timelineSelSegId: null,
setTimelineSelSegId: noop,
toggleSegSelect: noop,
selectAllSegs: noop,
clearSegSelection: noop,
bulkApplyToSelected: noop,
bulkDeleteSelected: noop,
};
}
const baseState = useAppStore.getState();
function renderDone({ tracks, langCode, lang = 'Auto' }) {
useAppStore.setState({
dubJobId: 'job1',
dubStep: 'done',
dubTracks: tracks,
dubLangCode: langCode,
dubLang: lang,
});
render(<DubTab {...makeProps()} />);
return captured.left.at(-1);
}
describe('DubTab — completed tracks always show their tabs (restore P0)', () => {
beforeEach(() => {
useAppStore.setState(baseState, true);
captured.left.length = 0;
});
it("restored project (tracks ['bn'], language_code frozen at 'und'): switcher shows and preview jumps to the track", () => {
const left = renderDone({ tracks: ['bn'], langCode: 'und' });
// Pre-fix: `dubLangCode !== 'und'` hid the finished tracks' tabs.
expect(left.hasDubbedTrack).toBe(true);
// Auto-jump falls back to the only real track never a lang without one.
expect(left.previewMode).toBe('bn');
});
it("membership guard: dubLangCode 'en' with tracks ['bn'] previews tracks[0], not the 404 lang", () => {
const left = renderDone({ tracks: ['bn'], langCode: 'en', lang: 'English' });
expect(left.hasDubbedTrack).toBe(true);
// Pre-fix the auto-jump previewed 'en' /dub/preview-video?lang=en 404.
expect(left.previewMode).toBe('bn');
});
it('dubLangCode that has a track previews that track (fresh-generate path unchanged)', () => {
const left = renderDone({ tracks: ['bn', 'es'], langCode: 'es', lang: 'Spanish' });
expect(left.hasDubbedTrack).toBe(true);
expect(left.previewMode).toBe('es');
});
it('done with NO persisted tracks hides the switcher and stays on Original (tautology guard)', () => {
const left = renderDone({ tracks: [], langCode: 'es', lang: 'Spanish' });
// Pre-fix `(dubTracks?.length > 0 || !!dubTracks)` was always true, so the
// switcher appeared trackless and the auto-jump 404'd the preview.
expect(left.hasDubbedTrack).toBe(false);
expect(left.previewMode).toBe('original');
});
});
@@ -0,0 +1,64 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { render, screen } from '@testing-library/react';
// Regression guard for the "20617d ago" epoch bug: /history rows carry
// created_at as Unix SECONDS (backend time.time()). Projects/OmniDrive fed
// them to a ms-based diff, so every generation-history card rendered as
// ~1970 ("20617d ago") and sorted to the bottom. The fix funnels every
// timestamp through utils/relativeTime.toMillis().
vi.mock('../utils/media', () => ({ playBlobAudio: vi.fn() }));
vi.mock('../api/generate', () => ({ audioUrl: (f) => `http://test.local/audio/${f}` }));
vi.mock('../api/client', () => ({
apiFetch: vi.fn(async () => ({ json: async () => ({ jobs: [] }) })),
}));
import Projects from '../pages/Projects';
describe('Projects — relative timestamps (seconds-vs-ms epoch class)', () => {
beforeEach(() => {
localStorage.clear();
});
afterEach(() => {
vi.restoreAllMocks();
});
it('renders a seconds created_at from today as hours ago, not "20617d ago"', async () => {
const history = [
{
id: 'h1',
filename: 'gen1.wav',
text: 'Hello from today',
created_at: Date.now() / 1000 - 7200, // 2h ago, in Unix SECONDS
},
];
render(<Projects history={history} />);
expect(await screen.findByText('2h ago')).toBeInTheDocument();
// The literal pre-fix rendering: tens of thousands of days.
expect(screen.queryByText(/\d{3,}d ago/)).toBeNull();
});
it('renders a dash for records with a missing timestamp', async () => {
const history = [{ id: 'h2', filename: 'gen2.wav', text: 'No stamp', created_at: null }];
render(<Projects history={history} />);
expect(await screen.findByText('No stamp')).toBeInTheDocument();
expect(screen.getByText('—')).toBeInTheDocument();
});
it('sorts seconds-stamped records among ms-stamped ones by real recency', async () => {
const nowS = Date.now() / 1000;
render(
<Projects
history={[{ id: 'h3', filename: 'g3.wav', text: 'Newest gen', created_at: nowS - 60 }]}
storyProjects={[{ id: 's1', name: 'Old story', updatedAt: Date.now() - 3 * 86400e3 }]}
/>,
);
const titles = (await screen.findAllByText(/Newest gen|Old story/)).map((el) => el.textContent);
// Pre-fix, the seconds stamp sorted as ~0 and sank below the story.
expect(titles).toEqual(['Newest gen', 'Old story']);
});
});
+88
View File
@@ -0,0 +1,88 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
import { apiFetch } from '../api/client';
import { getUnacknowledgedBackendCrash } from '../utils/backendCrash';
// #941: when the transport failure coincides with a recorded backend crash,
// the vague "Can't reach the local OmniVoice backend" must become the honest
// story — exit code + how long ago — and the crash-notice event must fire so
// the UI can offer "View crash details".
vi.mock('../utils/backendCrash', async (importOriginal) => {
const actual = await importOriginal<typeof import('../utils/backendCrash')>();
return {
...actual,
getUnacknowledgedBackendCrash: vi.fn().mockResolvedValue(null),
};
});
const crashMock = vi.mocked(getUnacknowledgedBackendCrash);
function markerSecondsAgo(s: number) {
return {
ts: Math.floor(Date.now() / 1000) - s,
exit_code: 3221226505,
signal: null,
exit_desc: 'exit code: 3221226505',
backend_version: '0.3.10',
uptime_s: 42,
last_stderr: 'OSError: [WinError 1455] The paging file is too small',
acknowledged: false,
};
}
describe('apiFetch — crash-marker honesty (#941)', () => {
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
vi.unstubAllGlobals();
crashMock.mockClear();
crashMock.mockResolvedValue(null);
});
it('replaces the vague unreachable error with the honest crash story', async () => {
vi.useFakeTimers();
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new TypeError('Failed to fetch')));
crashMock.mockResolvedValue(markerSecondsAgo(15));
const events: unknown[] = [];
const onCrash = (e: Event) => events.push((e as CustomEvent).detail);
window.addEventListener('ov:backend-crashed', onCrash);
const p = apiFetch('/generate');
const assertion = expect(p).rejects.toMatchObject({
status: 0,
// fake timers advance Date.now() during the retry backoff, so assert
// the shape (exit code + a seconds-scale age), not an exact second.
message: expect.stringMatching(/crashed \(exit code 3221226505\) \d+ s ago/),
});
await vi.advanceTimersByTimeAsync(400 + 900 + 1600 + 100);
await assertion;
// The crash-notice affordance is driven by this event.
expect(events).toHaveLength(1);
expect(events[0]).toMatchObject({ exit_code: 3221226505 });
window.removeEventListener('ov:backend-crashed', onCrash);
});
it('keeps the generic message when no unacknowledged crash exists', async () => {
vi.useFakeTimers();
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new TypeError('Failed to fetch')));
crashMock.mockResolvedValue(null);
const p = apiFetch('/generate');
const assertion = expect(p).rejects.toMatchObject({
status: 0,
message: expect.stringContaining("Can't reach the local OmniVoice backend"),
});
await vi.advanceTimersByTimeAsync(400 + 900 + 1600 + 100);
await assertion;
});
it('never turns an HTTP error into a crash story (backend responded)', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(new Response('nope', { status: 500, statusText: 'Server Error' })),
);
crashMock.mockResolvedValue(markerSecondsAgo(5));
await expect(apiFetch('/x')).rejects.toMatchObject({ status: 500 });
expect(crashMock).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,205 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { render, act } from '@testing-library/react';
import toast from 'react-hot-toast';
import { useAppStore } from '../store';
// P1.1 the multi-language generate loop must TRANSLATE each pick before it
// generates it. Pre-fix the loop only called handleDubGenerate per language,
// so "Generate 3 dubs" synthesized the same (untranslated) text three times
// at most one track was actually in its language.
//
// Contract under test (call order, per pick):
// translate(code) generate({ langOverride: { language, language_code } })
// and on a failed translate: skip that pick's generate, keep going, report
// the skipped languages in a final toast.
const captured = vi.hoisted(() => ({ header: [] }));
vi.mock('../components/dub/DubHeader', () => ({
default: (props) => {
captured.header.push(props);
return null;
},
}));
vi.mock('../components/dub/DubLeftColumn', () => ({ default: () => null }));
vi.mock('../components/dub/DubRightColumn', () => ({ default: () => null }));
vi.mock('../components/dub/DubFooter', () => ({ default: () => null }));
vi.mock('../components/dub/DubPipelineStepper', () => ({ default: () => null }));
vi.mock('../components/dub/IdleSkeleton', () => ({ default: () => null }));
vi.mock('../components/ExportModal', () => ({ default: () => null }));
vi.mock('../hooks/useTimelineOnsets', () => ({ default: () => ({ onsets: [] }) }));
vi.mock('../api/dub', () => ({ dubQc: vi.fn() }));
// Never-resolving async deps keep the render synchronous (no post-test act noise).
vi.mock('../api/engines', () => ({
listTranslationEngines: vi.fn(() => new Promise(() => {})),
installTranslationEngine: vi.fn(),
}));
vi.mock('../api/client', async (importOriginal) => {
const mod = await importOriginal();
return { ...mod, apiJson: vi.fn(() => new Promise(() => {})) };
});
import DubTab from '../pages/DubTab';
const noop = () => {};
function makeProps(over = {}) {
return {
dubVideoFile: null,
dubLocalBlobUrl: null,
transcribeElapsed: 0,
translateProvider: 'google',
setTranslateProvider: noop,
showTranscript: false,
setShowTranscript: noop,
onGlossaryChange: noop,
profiles: [],
segmentPreviewLoading: null,
selectedSegIds: new Set(),
setDubVideoFile: noop,
setDubLocalBlobUrl: noop,
handleDubAbort: noop,
handleDubUpload: noop,
handleDubIngestUrl: noop,
handleDubRetryTranscribe: noop,
handleDubStop: noop,
handleDubGenerate: noop,
handleDubImportSrt: noop,
handleDubDownload: noop,
handleDubAudioDownload: noop,
handleAudioExport: noop,
handleSegmentPreview: noop,
onDirectSegment: noop,
handleTranslateAll: noop,
handleCleanupSegments: noop,
incrementalPlan: null,
triggerDownload: noop,
fileToMediaUrl: noop,
editSegments: noop,
saveProject: noop,
resetDub: noop,
segmentEditField: noop,
segmentDelete: noop,
segmentRestoreOriginal: noop,
segmentSplit: noop,
segmentMerge: noop,
segmentMoveResize: noop,
timelineSelSegId: null,
setTimelineSelSegId: noop,
toggleSegSelect: noop,
selectAllSegs: noop,
clearSegSelection: noop,
bulkApplyToSelected: noop,
bulkDeleteSelected: noop,
...over,
};
}
const baseState = useAppStore.getState();
const PICKS = [
{ lang: 'Bengali', code: 'bn' },
{ lang: 'Spanish', code: 'es' },
];
/** Render DubTab in multi-lang mode and return { onGenerateClick, calls, mocks }. */
function setup({ translateOk = () => true, langCode = 'en', segments } = {}) {
const calls = [];
const handleTranslateAll = vi.fn(async (code) => {
calls.push(`translate:${code}`);
return translateOk(code);
});
const handleDubGenerate = vi.fn(async (opts) => {
calls.push(`generate:${opts?.langOverride?.language_code ?? 'default'}`);
});
useAppStore.setState({
dubJobId: 'job1',
dubStep: 'editing',
dubLangCode: langCode,
dubLang: 'English',
multiLangMode: true,
multiLangs: PICKS,
dubSegments: segments ?? [{ id: '1', text: 'hello', text_original: 'hello' }],
});
render(<DubTab {...makeProps({ handleTranslateAll, handleDubGenerate })} />);
return {
onGenerateClick: captured.header.at(-1).onGenerateClick,
calls,
handleTranslateAll,
handleDubGenerate,
};
}
describe('DubTab — multi-language generate translates each language first (P1.1)', () => {
beforeEach(() => {
useAppStore.setState(baseState, true);
captured.header.length = 0;
});
afterEach(() => {
vi.restoreAllMocks();
});
it("picks ['bn','es']: each language's translate runs BEFORE its generate, in order", async () => {
const { onGenerateClick, calls, handleDubGenerate } = setup();
await act(async () => {
await onGenerateClick();
});
// Pre-fix this was ['generate:bn', 'generate:es'] translate never ran.
expect(calls).toEqual(['translate:bn', 'generate:bn', 'translate:es', 'generate:es']);
// langOverride keeps the existing handleDubGenerate call shape.
expect(handleDubGenerate).toHaveBeenNthCalledWith(1, {
langOverride: { language: 'Bengali', language_code: 'bn' },
});
expect(handleDubGenerate).toHaveBeenNthCalledWith(2, {
langOverride: { language: 'Spanish', language_code: 'es' },
});
});
it('a failed translate skips ONLY that languages generate, continues, and reports it', async () => {
const errorSpy = vi.spyOn(toast, 'error');
const { onGenerateClick, calls } = setup({ translateOk: (code) => code !== 'bn' });
await act(async () => {
await onGenerateClick();
});
expect(calls).toEqual(['translate:bn', 'translate:es', 'generate:es']);
expect(errorSpy).toHaveBeenCalledTimes(1);
expect(errorSpy.mock.calls[0][0]).toContain('Bengali');
});
it('skips the redundant translate only when the FIRST pick already matches freshly-translated editor text', async () => {
const { onGenerateClick, calls } = setup({
langCode: 'bn',
// text differs from text_original on every segment = a translation into
// dubLangCode ('bn') is already applied pick 1 can go straight to generate.
segments: [{ id: '1', text: 'ওহে', text_original: 'hello' }],
});
await act(async () => {
await onGenerateClick();
});
expect(calls).toEqual(['generate:bn', 'translate:es', 'generate:es']);
});
it('untranslated editor text is ALWAYS translated, even when the first pick matches dubLangCode', async () => {
const { onGenerateClick, calls } = setup({
langCode: 'bn',
segments: [{ id: '1', text: 'hello', text_original: 'hello' }],
});
await act(async () => {
await onGenerateClick();
});
expect(calls).toEqual(['translate:bn', 'generate:bn', 'translate:es', 'generate:es']);
});
it('single-language mode is untouched: generate only, no translate, no override', async () => {
const { onGenerateClick, calls, handleDubGenerate, handleTranslateAll } = setup();
act(() => {
useAppStore.setState({ multiLangMode: false });
});
void onGenerateClick; // stale capture re-read after the mode flip
const fresh = captured.header.at(-1).onGenerateClick;
await act(async () => {
await fresh();
});
expect(handleTranslateAll).not.toHaveBeenCalled();
expect(handleDubGenerate).toHaveBeenCalledWith();
expect(calls).toEqual(['generate:default']);
});
});
@@ -0,0 +1,117 @@
import { describe, it, expect, beforeEach } from 'vitest';
import { useAppStore } from '../store';
import { restoreProjectExtras } from '../utils/projectState';
import appSrc from '../App.jsx?raw';
// P1.4 — multi-language picks live in the dub store slice (not DubTab-local
// state) and ride the project save/load payload, so "Generate 3 dubs" setups
// survive tab switches and project reopens. Legacy payloads (saved before
// these fields existed) must default cleanly: multi-lang off/empty, and the
// in-session exportTracks left untouched.
const baseState = useAppStore.getState();
describe('dub slice — multiLangMode / multiLangs', () => {
beforeEach(() => {
useAppStore.setState(baseState, true);
});
it('defaults to off/empty', () => {
expect(useAppStore.getState().multiLangMode).toBe(false);
expect(useAppStore.getState().multiLangs).toEqual([]);
});
it('setters accept values and functional updaters (slice pattern)', () => {
const s = useAppStore.getState();
s.setMultiLangMode(true);
s.setMultiLangs([{ lang: 'Bengali', code: 'bn' }]);
expect(useAppStore.getState().multiLangMode).toBe(true);
expect(useAppStore.getState().multiLangs).toEqual([{ lang: 'Bengali', code: 'bn' }]);
s.setMultiLangs((prev) => [...prev, { lang: 'Spanish', code: 'es' }]);
expect(useAppStore.getState().multiLangs).toHaveLength(2);
s.setMultiLangMode((prev) => !prev);
expect(useAppStore.getState().multiLangMode).toBe(false);
});
it('resetDubState clears the picks with the rest of the pipeline state', () => {
const s = useAppStore.getState();
s.setMultiLangMode(true);
s.setMultiLangs([{ lang: 'Bengali', code: 'bn' }]);
s.resetDubState();
expect(useAppStore.getState().multiLangMode).toBe(false);
expect(useAppStore.getState().multiLangs).toEqual([]);
});
});
describe('project payload — save/load round-trip (restoreProjectExtras)', () => {
beforeEach(() => {
useAppStore.setState(baseState, true);
});
it('round-trips multiLangMode, multiLangs and exportTracks through the payload', () => {
const s = useAppStore.getState();
s.setMultiLangMode(true);
s.setMultiLangs([
{ lang: 'Bengali', code: 'bn' },
{ lang: 'Spanish', code: 'es' },
]);
s.setExportTracks({ original: true, bn: true, es: false });
// Mirror App.jsx's saveProject: the store values land in state as-is.
const cur = useAppStore.getState();
const payload = {
multiLangMode: cur.multiLangMode,
multiLangs: cur.multiLangs,
exportTracks: cur.exportTracks,
};
const restored = restoreProjectExtras(JSON.parse(JSON.stringify(payload)));
expect(restored.multiLangMode).toBe(true);
expect(restored.multiLangs).toEqual([
{ lang: 'Bengali', code: 'bn' },
{ lang: 'Spanish', code: 'es' },
]);
expect(restored.exportTracks).toEqual({ original: true, bn: true, es: false });
});
it('legacy payload (fields absent) defaults to off/empty and leaves exportTracks alone', () => {
const restored = restoreProjectExtras({ dubJobId: 'old', dubSegments: [] });
expect(restored.multiLangMode).toBe(false);
expect(restored.multiLangs).toEqual([]);
expect(restored.exportTracks).toBeNull(); // null = don't touch the current value
});
it('is shape-safe: malformed picks are dropped, junk exportTracks is ignored', () => {
const restored = restoreProjectExtras({
multiLangMode: 'yes', // not boolean true → off
multiLangs: [{ lang: 'Bengali', code: 'bn' }, { code: 'es' }, 'fr', null],
exportTracks: ['original'],
});
expect(restored.multiLangMode).toBe(false);
expect(restored.multiLangs).toEqual([{ lang: 'Bengali', code: 'bn' }]);
expect(restored.exportTracks).toBeNull();
expect(restoreProjectExtras(undefined)).toEqual({
multiLangMode: false,
multiLangs: [],
exportTracks: null,
});
});
});
describe('App.jsx wiring guard (raw source — keeps the util honest)', () => {
it('saveProject persists the three fields in statePayload.state', () => {
const start = appSrc.indexOf('const statePayload');
expect(start).toBeGreaterThan(-1);
const block = appSrc.slice(start, appSrc.indexOf('apiSaveProject', start));
for (const key of ['multiLangMode', 'multiLangs', 'exportTracks']) {
expect(block, `statePayload.state must include ${key}`).toContain(key);
}
});
it('loadProject restores through restoreProjectExtras', () => {
const start = appSrc.indexOf('const loadProject');
expect(start).toBeGreaterThan(-1);
const block = appSrc.slice(start, start + 3000);
expect(block).toContain('restoreProjectExtras');
expect(block).toContain('setMultiLangMode');
expect(block).toContain('setMultiLangs');
});
});
@@ -0,0 +1,235 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { renderHook, act } from '@testing-library/react';
import { useAppStore } from '../store';
// P1.2 / P1.3 per-language translation storage + per-track fingerprints.
//
// `dubSegments[].text` is single-slot (the currently-shown language); before
// this fix, switching the dub target language left the previous language's
// text on screen and the next Translate All DESTROYED it. Now every
// translation is kept in `s.translations[langCode]`, `switchDubLangCode`
// swaps `text` through that map non-destructively, manual edits update the
// current language's entry, and the incremental-plan fingerprints are stored
// per language so "Regen N changed" judges the ACTIVE track.
const dubApi = vi.hoisted(() => ({
dubUpload: vi.fn(),
dubIngestUrl: vi.fn(),
dubAbort: vi.fn(),
dubCleanupSegments: vi.fn(),
dubTranslate: vi.fn(),
dubGenerate: vi.fn(),
tasksStreamUrl: vi.fn(() => ''),
tasksCancel: vi.fn(),
transcribeStreamUrl: vi.fn(() => ''),
dubImportSrt: vi.fn(),
}));
vi.mock('../api/dub', () => dubApi);
const clientApi = vi.hoisted(() => ({
apiPost: vi.fn(),
apiFetch: vi.fn(),
apiJson: vi.fn(),
API: '',
}));
vi.mock('../api/client', () => clientApi);
import useDubWorkflow from '../hooks/useDubWorkflow';
import useSegmentEditing from '../hooks/useSegmentEditing';
const baseState = useAppStore.getState();
function renderWorkflow() {
return renderHook(() =>
useDubWorkflow({
loadProjects: vi.fn(),
loadProfiles: vi.fn(),
loadDubHistory: vi.fn(),
setLastGenFingerprints: vi.fn(),
}),
);
}
const seg = (over = {}) => ({
id: '1',
text: 'hello there',
text_original: 'hello there',
start: 0,
end: 2,
...over,
});
beforeEach(() => {
useAppStore.setState(baseState, true);
dubApi.dubTranslate.mockReset();
clientApi.apiPost.mockReset();
useAppStore.setState({
dubJobId: 'job1',
dubStep: 'editing',
dubLangCode: 'bn',
dubSegments: [seg()],
});
});
const translateTo = async (result, lang, text) => {
dubApi.dubTranslate.mockResolvedValueOnce({
translated: [{ id: '1', text }],
target_lang: lang,
});
await act(async () => {
await result.current.handleTranslateAll(lang);
});
};
describe('per-language translations (P1.2)', () => {
it('translate bn then es retains BOTH languages in s.translations (pre-fix: bn lost)', async () => {
const { result } = renderWorkflow();
await translateTo(result, 'bn', 'ওহে');
act(() => useAppStore.getState().switchDubLangCode('es'));
await translateTo(result, 'es', 'hola');
const s = useAppStore.getState().dubSegments[0];
expect(s.text).toBe('hola'); // text stays the shown language (legacy slot)
expect(s.translations).toMatchObject({ bn: 'ওহে', es: 'hola' });
});
it('switching the target language swaps text non-destructively, both directions', async () => {
const { result } = renderWorkflow();
await translateTo(result, 'bn', 'ওহে');
act(() => useAppStore.getState().switchDubLangCode('es'));
await translateTo(result, 'es', 'hola');
act(() => useAppStore.getState().switchDubLangCode('bn'));
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
act(() => useAppStore.getState().switchDubLangCode('es'));
expect(useAppStore.getState().dubSegments[0].text).toBe('hola');
});
it('switching to a never-translated language leaves text unchanged (legacy behaviour)', () => {
useAppStore.setState({
dubSegments: [seg({ text: 'ওহে', translations: { bn: 'ওহে' } })],
});
act(() => useAppStore.getState().switchDubLangCode('es'));
// Non-destructive: no es entry keep showing what was there.
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
expect(useAppStore.getState().dubLangCode).toBe('es');
});
it('legacy segments (no translations field) survive a switch round-trip', () => {
// A pre-upgrade project where bn text was already translated in place.
useAppStore.setState({
dubSegments: [seg({ text: 'ওহে' })], // text !== text_original, no map
});
act(() => useAppStore.getState().switchDubLangCode('es'));
act(() => useAppStore.getState().switchDubLangCode('bn'));
// The switch snapshotted bn's text into the map instead of losing it.
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
expect(useAppStore.getState().dubSegments[0].translations.bn).toBe('ওহে');
});
it('never stamps untranslated (source) text as a translation on switch', () => {
// text === text_original not a translation, must not be snapshotted.
act(() => useAppStore.getState().switchDubLangCode('es'));
expect(useAppStore.getState().dubSegments[0].translations.bn).toBeUndefined();
});
it('manual segment edit updates the CURRENT language entry only', () => {
useAppStore.setState({
dubLangCode: 'es',
dubSegments: [seg({ text: 'hola', translations: { bn: 'ওহে', es: 'hola' } })],
});
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.segmentEditField('1', 'text', 'hola editada'));
const s = useAppStore.getState().dubSegments[0];
expect(s.text).toBe('hola editada');
expect(s.translations).toEqual({ bn: 'ওহে', es: 'hola editada' });
});
it('restore-original records the decision under the current language', () => {
useAppStore.setState({
dubLangCode: 'es',
dubSegments: [seg({ text: 'hola', translations: { es: 'hola' } })],
});
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.segmentRestoreOriginal('1'));
const s = useAppStore.getState().dubSegments[0];
expect(s.text).toBe('hello there');
expect(s.translations.es).toBe('hello there');
});
it('merge joins per-language texts only where both rows carry the language', () => {
useAppStore.setState({
dubLangCode: 'es',
dubSegments: [
seg({ id: 'a', end: 1, translations: { es: 'uno', bn: 'এক' } }),
seg({ id: 'b', start: 1, translations: { es: 'dos' } }),
],
});
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.segmentMerge('a'));
const merged = useAppStore.getState().dubSegments[0];
expect(merged.translations).toEqual({ es: 'uno dos' }); // bn half-known dropped
});
it('split drops the per-language map (new ids need fresh translations)', () => {
useAppStore.setState({
dubLangCode: 'es',
dubSegments: [seg({ text: 'hola mundo', translations: { es: 'hola mundo' } })],
});
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.segmentSplit('1', 5));
const segs = useAppStore.getState().dubSegments;
expect(segs).toHaveLength(2);
expect(segs[0].translations).toBeUndefined();
expect(segs[1].translations).toBeUndefined();
});
});
describe('per-track fingerprints (P1.3)', () => {
it('lastGenFingerprints follows the ACTIVE language', () => {
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.setLastGenFingerprints({ 1: 'hash-bn' }, 'bn'));
act(() => result.current.setLastGenFingerprints({ 1: 'hash-es' }, 'es'));
expect(useAppStore.getState().dubLangCode).toBe('bn');
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hash-bn' });
act(() => useAppStore.getState().switchDubLangCode('es'));
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hash-es' });
});
it('recomputeIncremental sends the active lang + that languages hashes', async () => {
clientApi.apiPost.mockResolvedValue({ stale: [], fresh: ['1'], fingerprints: {} });
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.setLastGenFingerprints({ 1: 'hash-bn' }, 'bn'));
await act(async () => {
await result.current.recomputeIncremental();
});
expect(clientApi.apiPost).toHaveBeenCalledWith(
'/tools/incremental',
expect.objectContaining({ lang: 'bn', stored_hashes: { 1: 'hash-bn' } }),
);
expect(result.current.incrementalPlan).toEqual({ stale: [], fresh: ['1'] });
});
it('a language with no stored hashes yields no plan (unknowable ≠ stale)', async () => {
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.setLastGenFingerprints({ 1: 'hash-es' }, 'es'));
// Active language is bn no hashes plan cleared, no API call.
await act(async () => {
await result.current.recomputeIncremental();
});
expect(clientApi.apiPost).not.toHaveBeenCalled();
expect(result.current.incrementalPlan).toBeNull();
});
it('setFingerprintsByLang restores every track at once (project/history load)', () => {
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.setFingerprintsByLang({ bn: { 1: 'hb' }, es: { 1: 'he' } }));
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hb' }); // active = bn
expect(result.current.fingerprintsByLang.es).toEqual({ 1: 'he' });
});
it('setLastGenFingerprints without a lang defaults to the store selection', () => {
const { result } = renderHook(() => useSegmentEditing());
act(() => result.current.setLastGenFingerprints({ 1: 'h' }));
expect(result.current.fingerprintsByLang).toEqual({ bn: { 1: 'h' } });
});
});
@@ -0,0 +1,152 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { renderHook, act } from '@testing-library/react';
import { useAppStore } from '../store';
// P1.1 `handleTranslateAll(langOverride?)`:
// - no-arg keeps the existing Translate All behavior (target = store's
// dubLangCode),
// - a string override translates INTO that language (the multi-language
// generate loop passes each pick's code),
// - a non-string first arg (the `onClick={handleTranslateAll}` click event)
// must be ignored, not treated as a language,
// - resolves true only when a translation actually landed the batch loop
// keys "skip this language's generate" off that.
const dubApi = vi.hoisted(() => ({
dubUpload: vi.fn(),
dubIngestUrl: vi.fn(),
dubAbort: vi.fn(),
dubCleanupSegments: vi.fn(),
dubTranslate: vi.fn(),
dubGenerate: vi.fn(),
tasksStreamUrl: vi.fn(() => ''),
tasksCancel: vi.fn(),
transcribeStreamUrl: vi.fn(() => ''),
dubImportSrt: vi.fn(),
}));
vi.mock('../api/dub', () => dubApi);
vi.mock('../api/client', () => ({
apiPost: vi.fn(),
apiFetch: vi.fn(),
apiJson: vi.fn(),
API: '',
}));
import useDubWorkflow from '../hooks/useDubWorkflow';
const baseState = useAppStore.getState();
function renderWorkflow() {
return renderHook(() =>
useDubWorkflow({
loadProjects: vi.fn(),
loadProfiles: vi.fn(),
loadDubHistory: vi.fn(),
setLastGenFingerprints: vi.fn(),
}),
);
}
describe('handleTranslateAll(langOverride) — multi-language target override', () => {
beforeEach(() => {
useAppStore.setState(baseState, true);
dubApi.dubTranslate.mockReset();
useAppStore.setState({
dubJobId: 'job1',
dubStep: 'editing',
dubLangCode: 'es',
dubSegments: [
{ id: '1', text: 'hello there', text_original: 'hello there', start: 0, end: 2 },
],
});
});
it('no-arg behavior unchanged: translates into the store dubLangCode and applies the text', async () => {
dubApi.dubTranslate.mockResolvedValue({
translated: [{ id: '1', text: 'hola' }],
target_lang: 'es',
});
const { result } = renderWorkflow();
let ok;
await act(async () => {
ok = await result.current.handleTranslateAll();
});
expect(dubApi.dubTranslate).toHaveBeenCalledTimes(1);
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('es');
expect(ok).toBe(true);
expect(useAppStore.getState().dubSegments[0].text).toBe('hola');
});
it('string override translates INTO the override language, not the store selection', async () => {
dubApi.dubTranslate.mockResolvedValue({
translated: [{ id: '1', text: 'ওহে' }],
target_lang: 'bn',
});
const { result } = renderWorkflow();
let ok;
await act(async () => {
ok = await result.current.handleTranslateAll('bn');
});
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('bn');
expect(ok).toBe(true);
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
});
it('a click event as first arg (onClick={handleTranslateAll}) falls back to dubLangCode', async () => {
dubApi.dubTranslate.mockResolvedValue({
translated: [{ id: '1', text: 'hola' }],
target_lang: 'es',
});
const { result } = renderWorkflow();
await act(async () => {
await result.current.handleTranslateAll({ preventDefault() {}, type: 'click' });
});
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('es');
});
it('request failure resolves false and surfaces the existing error banner', async () => {
dubApi.dubTranslate.mockRejectedValue(new Error('engine down'));
const { result } = renderWorkflow();
let ok;
await act(async () => {
ok = await result.current.handleTranslateAll('bn');
});
expect(ok).toBe(false);
expect(useAppStore.getState().dubError).toMatch(/engine down/);
expect(useAppStore.getState().isTranslating).toBe(false);
});
it('an all-errors result resolves false (nothing translated → no wrong-language dub)', async () => {
dubApi.dubTranslate.mockResolvedValue({
translated: [{ id: '1', text: '', error: 'boom' }],
target_lang: 'bn',
});
const { result } = renderWorkflow();
let ok;
await act(async () => {
ok = await result.current.handleTranslateAll('bn');
});
expect(ok).toBe(false);
});
it('reads segments from the store at call time (stale click-time closure is the loop bug class)', async () => {
dubApi.dubTranslate.mockResolvedValue({
translated: [{ id: '2', text: 'nuevo' }],
target_lang: 'es',
});
const { result } = renderWorkflow();
const stale = result.current.handleTranslateAll; // captured before the segments change
act(() => {
useAppStore
.getState()
.setDubSegments([{ id: '2', text: 'fresh', text_original: 'fresh', start: 0, end: 1 }]);
});
await act(async () => {
await stale();
});
const sent = dubApi.dubTranslate.mock.calls[0][0].segments;
expect(sent).toHaveLength(1);
expect(sent[0].id).toBe('2');
expect(sent[0].text).toBe('fresh');
});
});
+82
View File
@@ -0,0 +1,82 @@
/**
* backendCrash frontend bridge to the desktop shell's crash forensics
* (#941, src-tauri/src/crash.rs).
*
* When the backend PROCESS dies (native CUDA abort, OOM kill, DLL crash) the
* Rust death watcher persists a crash marker (exit code/signal + stderr tail).
* This module reads it so:
* - api/client.ts can replace the vague "Can't reach the local backend"
* with the honest story,
* - components/BackendCrashNotice.jsx can offer "View crash details",
* - utils/bugReport.js can attach the evidence to the GitHub-issue prefill.
*
* Outside the Tauri shell (browser dev, Docker, LAN share) every getter
* resolves to null there is no local process to forensicate.
*/
export interface BackendCrashMarker {
/** Unix seconds when the death was detected. */
ts: number;
exit_code: number | null;
signal: number | null;
/** Human-readable ExitStatus display ("exit status: 134", …). */
exit_desc: string;
backend_version: string;
/** Seconds the backend had been running when it died. */
uptime_s: number;
/** Tail of backend_err.log captured at death time (~40 lines). */
last_stderr: string;
/** Whether the user already viewed/dismissed this crash. */
acknowledged: boolean;
}
function inTauri(): boolean {
const w = window as unknown as Record<string, unknown> | undefined;
return typeof window !== 'undefined' && !!(w?.__TAURI__ || w?.__TAURI_INTERNALS__);
}
/** Newest crash marker the shell knows about, or null (also null outside Tauri). */
export async function getLastBackendCrash(): Promise<BackendCrashMarker | null> {
if (!inTauri()) return null;
try {
const { invoke } = await import('@tauri-apps/api/core');
return ((await invoke('get_last_backend_crash')) as BackendCrashMarker | null) ?? null;
} catch {
return null;
}
}
/** Newest crash marker only if the user hasn't acknowledged it yet. */
export async function getUnacknowledgedBackendCrash(): Promise<BackendCrashMarker | null> {
const marker = await getLastBackendCrash();
return marker && !marker.acknowledged ? marker : null;
}
/** Mark the newest crash as seen (the marker itself is retained for reports). */
export async function acknowledgeBackendCrash(): Promise<void> {
if (!inTauri()) return;
try {
const { invoke } = await import('@tauri-apps/api/core');
await invoke('acknowledge_backend_crash');
} catch {
/* shell unavailable — nothing to acknowledge */
}
}
/** "exit code 3221226505" / "signal 6" / the raw ExitStatus display. */
export function describeCrashExit(
marker: Pick<BackendCrashMarker, 'exit_code' | 'signal' | 'exit_desc'>,
): string {
if (marker.exit_code != null) return `exit code ${marker.exit_code}`;
if (marker.signal != null) return `signal ${marker.signal}`;
return marker.exit_desc || 'unknown exit';
}
/** Coarse "12 s" / "3 min" / "2 h" age of a marker, for the honest message. */
export function crashAge(marker: Pick<BackendCrashMarker, 'ts'>, nowMs = Date.now()): string {
const s = Math.max(0, Math.round(nowMs / 1000 - marker.ts));
if (s < 90) return `${s} s`;
const min = Math.round(s / 60);
if (min < 90) return `${min} min`;
return `${Math.round(min / 60)} h`;
}
+39
View File
@@ -15,6 +15,7 @@
/* global __APP_VERSION__ -- injected by Vite at build time (vite.config define) */
import { API } from '../api/client';
import { formatBreadcrumbs } from './breadcrumbs';
import { crashAge, describeCrashExit, getLastBackendCrash } from './backendCrash';
export const ISSUES_URL = 'https://github.com/debpalash/OmniVoice-Studio/issues/new';
@@ -67,6 +68,9 @@ export function scrubText(text) {
// under the ~8k practical ceiling so the user never loses the form.
const MAX_STACK_CHARS = 1800;
const MAX_MSG_CHARS = 1200;
// Crash-marker stderr tail budget (#941) — keep the newest end (the actual
// traceback/abort), the head is uvicorn boot noise.
const MAX_CRASH_TAIL_CHARS = 1200;
// The real ceiling is on the URL-ENCODED body, not the raw string: markdown
// encodes ~1.31.6× larger (newlines→%0A, spaces→%20, backticks/#//), so a
// 6000-char raw body can be ~9k encoded and blow past GitHub's limit. Bound
@@ -139,6 +143,39 @@ async function captureContext() {
return lines.join('\n');
}
/** "## Last backend crash" section from the desktop shell's crash marker
* (#941): exit code/signal + scrubbed stderr tail, so a "backend became
* unreachable" report arrives WITH the evidence instead of needing a
* logs-please round-trip. Empty outside Tauri or when nothing ever crashed.
* The marker's age is stated so a stale (possibly unrelated) crash can't
* masquerade as fresh evidence. */
async function captureCrashSection() {
let marker = null;
try {
marker = await getLastBackendCrash();
} catch {
/* shell forensics unavailable */
}
if (!marker) return [];
let tail = scrubText(marker.last_stderr || '').trim();
if (tail.length > MAX_CRASH_TAIL_CHARS) {
tail = `… (truncated)\n${tail.slice(-MAX_CRASH_TAIL_CHARS)}`;
}
return [
'## Last backend crash (auto-captured — may predate this bug)',
'',
`**When:** ${new Date(marker.ts * 1000).toISOString()} (${crashAge(marker)} ago)`,
`**Exit:** \`${describeCrashExit(marker)}\``,
`**Uptime before crash:** ${marker.uptime_s} s`,
`**Backend version:** \`${marker.backend_version}\``,
'',
'```',
tail || '(no stderr captured)',
'```',
'',
];
}
/**
* Build the prefilled GitHub Issues URL.
*
@@ -150,6 +187,7 @@ async function captureContext() {
*/
export async function buildBugReportUrl({ title = '[Bug] ', error } = {}) {
const ctx = await captureContext();
const crashSection = await captureCrashSection();
const errorSection = [];
if (error) {
@@ -194,6 +232,7 @@ export async function buildBugReportUrl({ title = '[Bug] ', error } = {}) {
'',
ctx,
'',
...crashSection,
...crumbSection,
'## What I was doing',
'',
+59
View File
@@ -1,6 +1,15 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { scrubText, buildBugReportUrl, ISSUES_URL, REDACTED } from './bugReport';
import { getLastBackendCrash } from './backendCrash';
// #941: keep the real crashAge/describeCrashExit helpers; only the shell
// bridge is mocked (it resolves null by default, like a non-Tauri context —
// every pre-existing test keeps its no-crash behavior).
vi.mock('./backendCrash', async (importOriginal) => {
const actual = await importOriginal();
return { ...actual, getLastBackendCrash: vi.fn().mockResolvedValue(null) };
});
describe('scrubText — frontend twin of backend/core/scrub.py', () => {
it.each([
@@ -106,6 +115,56 @@ describe('buildBugReportUrl', () => {
});
});
describe('buildBugReportUrl — crash-marker enrichment (#941)', () => {
beforeEach(() => {
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new Error('ECONNREFUSED')));
getLastBackendCrash.mockResolvedValue(null);
});
it('attaches the crash evidence (exit code + scrubbed stderr tail)', async () => {
getLastBackendCrash.mockResolvedValue({
ts: Math.floor(Date.now() / 1000) - 30,
exit_code: 3221226505,
signal: null,
exit_desc: 'exit code: 3221226505',
backend_version: '0.3.10',
uptime_s: 42,
last_stderr:
'File "/Users/alice/omnivoice/backend/main.py", line 1\nOSError: paging file too small',
acknowledged: true, // acked markers still ride along — ack ≠ delete
});
const body = decodeURIComponent(await buildBugReportUrl());
expect(body).toContain('## Last backend crash');
expect(body).toContain('exit code 3221226505');
expect(body).toContain('**Uptime before crash:** 42 s');
// Home paths in the stderr tail are scrubbed like every other section.
expect(body).toContain('~/omnivoice/backend/main.py');
expect(body).not.toContain('/Users/alice');
});
it('omits the section entirely when no crash was ever recorded', async () => {
const body = decodeURIComponent(await buildBugReportUrl());
expect(body).not.toContain('## Last backend crash');
});
it('keeps the newest end of an oversized stderr tail and stays under the URL ceiling', async () => {
getLastBackendCrash.mockResolvedValue({
ts: Math.floor(Date.now() / 1000),
exit_code: 1,
signal: null,
exit_desc: 'exit status: 1',
backend_version: '0.3.10',
uptime_s: 1,
last_stderr: `${'boot noise line\n'.repeat(400)}THE REAL TRACEBACK LINE`,
acknowledged: false,
});
const url = await buildBugReportUrl();
const body = decodeURIComponent(url);
expect(body).toContain('THE REAL TRACEBACK LINE'); // tail kept, head dropped
expect(url.length).toBeLessThan(8000);
});
});
describe('buildIssueSearchUrl', () => {
it('builds a scrubbed, noise-free search query', async () => {
const { buildIssueSearchUrl } = await import('./bugReport');
+35
View File
@@ -0,0 +1,35 @@
/**
* Project payload extras (P1.4) multi-language picks + export-track prefs
* ride the saved project state so reopening a project restores the batch
* setup the user configured.
*
* Legacy payloads (projects saved before these fields existed) simply lack
* the keys restoring them must default cleanly: multi-lang OFF/empty, and
* exportTracks left untouched (`null` sentinel) so a legacy load never
* clobbers the user's current in-session export choices.
*/
/** True for a plain `{ track: boolean }` map (rejects arrays/null). */
function isPlainObject(v) {
return !!v && typeof v === 'object' && !Array.isArray(v);
}
/**
* Normalize the multi-lang / export-track fields of a loaded project payload.
*
* @param {object} [state] - `data.state` from the projects API (may be legacy).
* @returns {{ multiLangMode: boolean, multiLangs: {lang: string, code: string}[], exportTracks: Record<string, boolean> | null }}
* `exportTracks === null` means "absent in payload — keep the current value".
*/
export function restoreProjectExtras(state = {}) {
const s = isPlainObject(state) ? state : {};
return {
multiLangMode: s.multiLangMode === true,
multiLangs: Array.isArray(s.multiLangs)
? s.multiLangs.filter(
(l) => isPlainObject(l) && typeof l.lang === 'string' && typeof l.code === 'string',
)
: [],
exportTracks: isPlainObject(s.exportTracks) ? s.exportTracks : null,
};
}
+88
View File
@@ -0,0 +1,88 @@
/**
* Shared, unit-tolerant timestamp normalization + relative-time formatting.
*
* Root cause of the "20617d ago" bug class: the backend stores timestamps as
* Unix SECONDS (`time.time()` REAL columns: generation_history.created_at,
* dub_history, exports, longform jobs, projects), while frontend-local
* records carry MILLISECONDS (`Date.now()` story projects) or ISO strings
* (transcriptions). Any formatter that assumes one unit renders the other as
* ~1970 ("20617d ago") or an epoch date ("Jan 1, 1970"). Every timestamp
* render must funnel through toMillis() so no individual view can regress.
*
* Do NOT change the backend's stored format existing user DBs hold seconds.
*/
// Numeric timestamps below this are seconds, at/above it milliseconds.
// 1e12 ms = Sep 2001; 1e12 s = year 33658 — unambiguous for real data.
const MS_THRESHOLD = 1e12;
const SHORT_DATE_OPTS = { month: 'short', day: 'numeric' };
/**
* Normalize any timestamp shape to epoch milliseconds, or null when missing/
* unparseable. Accepts: Unix seconds (float), epoch ms, numeric strings of
* either, ISO/date strings, Date instances. 0 / null / undefined / '' are
* treated as "missing" (0 is this codebase's missing-timestamp sentinel
* never a real 1970 record).
*/
export function toMillis(ts) {
if (ts == null || ts === 0 || ts === '') return null;
if (ts instanceof Date) {
const t = ts.getTime();
return Number.isNaN(t) ? null : t;
}
if (typeof ts === 'string') {
const trimmed = ts.trim();
if (!trimmed) return null;
// Numeric strings ("1751600000", "1751600000000.5") are Unix stamps,
// not date strings — Date.parse would reject or misread them.
const asNum = Number(trimmed);
if (Number.isFinite(asNum)) return toMillis(asNum);
const parsed = Date.parse(trimmed);
return Number.isNaN(parsed) ? null : parsed;
}
if (typeof ts !== 'number' || !Number.isFinite(ts) || ts <= 0) return null;
return Math.round(ts < MS_THRESHOLD ? ts * 1000 : ts);
}
// Future stamps within this window are clock skew, not data corruption.
const CLOCK_SKEW_MS = 60_000;
/**
* Relative-time label for record timestamps (any unit see toMillis).
* - missing/unparseable "—" (never "20617d ago" / epoch dates)
* - future within 1 min (clock skew) "just now"
* - <60s "Ns ago"; <60m "Nm ago"; <24h "Nh ago"; <7d "Nd ago"
* - older (or far-future, i.e. bad clock) short absolute date
*/
export function timeAgo(ts) {
const ms = toMillis(ts);
if (ms == null) return '—';
const diff = Date.now() - ms;
if (diff < -CLOCK_SKEW_MS) return shortDate(ms);
if (diff < 0) return 'just now';
const s = Math.floor(diff / 1000);
if (s < 60) return `${s}s ago`;
const m = Math.floor(s / 60);
if (m < 60) return `${m}m ago`;
const h = Math.floor(m / 60);
if (h < 24) return `${h}h ago`;
const d = Math.floor(h / 24);
if (d < 7) return `${d}d ago`;
return shortDate(ms);
}
/** Absolute host-locale datetime for tooltips; '' when missing/unparseable. */
export function absoluteTime(ts) {
const ms = toMillis(ts);
return ms == null ? '' : new Date(ms).toLocaleString();
}
function shortDate(ms) {
const d = new Date(ms);
const opts =
d.getFullYear() === new Date().getFullYear()
? SHORT_DATE_OPTS
: { ...SHORT_DATE_OPTS, year: 'numeric' };
return d.toLocaleDateString([], opts);
}
+96
View File
@@ -0,0 +1,96 @@
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import { toMillis, timeAgo, absoluteTime } from './relativeTime';
// Regression suite for the "20617d ago" bug class: backend rows store Unix
// SECONDS (time.time()); formatters that assumed epoch MILLISECONDS rendered
// every record as ~1970 ("20617d ago" ≈ 56.5 years) or an epoch date.
// Frozen "now": 2026-07-04T12:00:00Z.
const NOW_MS = Date.UTC(2026, 6, 4, 12, 0, 0);
const NOW_S = NOW_MS / 1000;
beforeEach(() => {
vi.useFakeTimers();
vi.setSystemTime(NOW_MS);
});
afterEach(() => {
vi.useRealTimers();
});
describe('toMillis', () => {
it('converts Unix seconds (backend time.time() floats) to ms', () => {
expect(toMillis(NOW_S - 7200)).toBe(NOW_MS - 7200 * 1000);
expect(toMillis(1751600000.123)).toBe(1751600000123);
});
it('passes epoch milliseconds through unchanged', () => {
expect(toMillis(NOW_MS)).toBe(NOW_MS);
expect(toMillis(1751600000123)).toBe(1751600000123);
});
it('parses ISO strings', () => {
expect(toMillis('2026-07-04T11:00:00.000Z')).toBe(NOW_MS - 3600 * 1000);
});
it('treats numeric strings as Unix stamps, not dates', () => {
expect(toMillis(String(NOW_S))).toBe(NOW_MS);
expect(toMillis(String(NOW_MS))).toBe(NOW_MS);
});
it('accepts Date instances', () => {
expect(toMillis(new Date(NOW_MS))).toBe(NOW_MS);
expect(toMillis(new Date('garbage'))).toBeNull();
});
it('returns null for missing/degenerate values', () => {
for (const v of [null, undefined, 0, '', ' ', NaN, Infinity, -5, 'not a date']) {
expect(toMillis(v)).toBeNull();
}
});
});
describe('timeAgo', () => {
it('EPOCH REGRESSION: a seconds timestamp from today never renders as thousands of days ago', () => {
// Pre-fix, 2h-ago-in-seconds fed to a ms diff → "20617d ago" (1970).
const label = timeAgo(NOW_S - 7200);
expect(label).toBe('2h ago');
expect(label).not.toMatch(/\d{3,}d ago/);
});
it('renders identical output for seconds and milliseconds inputs', () => {
expect(timeAgo(NOW_S - 90)).toBe('1m ago');
expect(timeAgo(NOW_MS - 90 * 1000)).toBe('1m ago');
expect(timeAgo(NOW_S - 3 * 86400)).toBe('3d ago');
expect(timeAgo(NOW_MS - 3 * 86400 * 1000)).toBe('3d ago');
});
it('parses ISO strings', () => {
expect(timeAgo('2026-07-04T11:59:30.000Z')).toBe('30s ago');
});
it('renders a dash for null/0/undefined — never an epoch age', () => {
expect(timeAgo(null)).toBe('—');
expect(timeAgo(0)).toBe('—');
expect(timeAgo(undefined)).toBe('—');
expect(timeAgo('')).toBe('—');
});
it('treats future stamps within clock skew (<1 min ahead) as "just now"', () => {
expect(timeAgo(NOW_S + 30)).toBe('just now');
expect(timeAgo(NOW_MS + 59 * 1000)).toBe('just now');
});
it('falls back to an absolute date beyond 7 days (and for far-future stamps)', () => {
expect(timeAgo(NOW_S - 30 * 86400)).not.toMatch(/ago/);
expect(timeAgo(NOW_S + 3600)).not.toMatch(/ago/);
});
});
describe('absoluteTime', () => {
it('formats any unit and is empty for missing values (no "Jan 1, 1970" tooltips)', () => {
expect(absoluteTime(NOW_S)).toBe(new Date(NOW_MS).toLocaleString());
expect(absoluteTime(NOW_MS)).toBe(new Date(NOW_MS).toLocaleString());
expect(absoluteTime(null)).toBe('');
expect(absoluteTime(0)).toBe('');
});
});
+108 -10
View File
@@ -1,7 +1,9 @@
/**
* timeline.js pure math/state helpers for the dub timeline segment editor
* (#280, item 3). Everything here is DOM-free and unit-tested; SegmentTrack
* only does rendering + pointer/keyboard plumbing on top of these.
* (#280, item 3). Everything here is DOM-free and unit-tested except the
* region palette below, which reads `--chrome-bg` off the document root (with
* a non-DOM fallback) so the box colors can be pre-blended in JS (#963).
* SegmentTrack only does rendering + pointer/keyboard plumbing on top.
*
* All times are seconds (float), all pixels are CSS px.
*/
@@ -19,16 +21,112 @@ const GRID_SNAP_MAX_PX_PER_SEC = 40;
// Segment box palette — was WaveformTimeline's region palette; lives here so
// both the track and any legend can share it without circular imports.
export const REGION_COLORS = [
'rgba(211,134,155,0.45)',
'rgba(131,165,152,0.45)',
'rgba(184,187,38,0.45)',
'rgba(250,189,47,0.45)',
'rgba(142,192,124,0.45)',
'rgba(254,128,25,0.45)',
'rgba(104,157,106,0.45)',
//
// FULLY OPAQUE by design (#373): these used to be `rgba(…, 0.45)` and relied
// on alpha compositing over the panel behind the track — and on some Windows
// GPU/WebView2 drivers, semi-transparent paints on the (formerly
// transform-animated) lane flashed invisible during playback. Each entry
// pre-blends the same 45% tint against the surface behind the lane
// (`--chrome-bg`, the .studio-panel background), computing the identical
// pixels (0.45·tint + 0.55·bg) with zero alpha.
//
// PRE-BLENDED IN JS by design (#963): #951 did the blend with
// `color-mix(in srgb, …)` inside the inline style — but WebView2/Chromium
// < 111 has no color-mix, the CSSOM rejects the whole `background`
// assignment, and .seg-track__box declares no background of its own, so the
// boxes rendered fully transparent on pinned/enterprise WebView2 runtimes.
// The blend now happens here in JS and the inline style receives a literal
// `rgb(r, g, b)` every engine can parse. Theme-awareness is preserved by
// re-reading `--chrome-bg` when [data-theme] changes on the document root
// (the seam App.jsx uses to switch themes). Do NOT reintroduce alpha OR any
// engine-dependent CSS function here; guarded by timeline.test.js +
// SegmentTrack.test.jsx.
const REGION_TINTS = [
[211, 134, 155],
[131, 165, 152],
[184, 187, 38],
[250, 189, 47],
[142, 192, 124],
[254, 128, 25],
[104, 157, 106],
];
// Gruvbox Dark `--chrome-bg` (#0f1011) — the :root default in index.css.
// Used when the variable is unreadable (non-DOM test runner, CSS not loaded).
const FALLBACK_CHROME_BG = [15, 16, 17];
/** Parse a CSS color literal (#rgb, #rrggbb, rgb()/rgba()) → [r,g,b] | null. */
function parseCssColor(raw) {
if (typeof raw !== 'string') return null;
const s = raw.trim();
let m = /^#([0-9a-f]{3})$/i.exec(s);
if (m) return [...m[1]].map((c) => parseInt(c + c, 16));
m = /^#([0-9a-f]{6})$/i.exec(s);
if (m) return [0, 2, 4].map((i) => parseInt(m[1].slice(i, i + 2), 16));
m = /^rgba?\(\s*(\d{1,3})[\s,]+(\d{1,3})[\s,]+(\d{1,3})\s*(?:[,/][^)]*)?\)$/i.exec(s);
if (m) return [+m[1], +m[2], +m[3]];
return null;
}
/** Blend a 45% tint over an opaque background same math as
* `color-mix(in srgb, tint 45%, bg)`, emitted as a literal rgb() string. */
export function blendRegionColor(tint, bg) {
const [r, g, b] = tint.map((c, i) => Math.round(0.45 * c + 0.55 * bg[i]));
return `rgb(${r}, ${g}, ${b})`;
}
function readChromeBg() {
try {
const raw = getComputedStyle(document.documentElement).getPropertyValue('--chrome-bg');
return parseCssColor(raw) ?? FALLBACK_CHROME_BG;
} catch {
return FALLBACK_CHROME_BG; // SSR / non-DOM test runner
}
}
function blendPalette() {
const bg = readChromeBg();
return REGION_TINTS.map((tint) => blendRegionColor(tint, bg));
}
/**
* REGION_COLORS the current palette as literal `rgb(r, g, b)` strings.
* Live ESM binding: re-assigned (never mutated in place) when the theme
* changes, so `getRegionColors()` is a stable-reference snapshot fit for
* useSyncExternalStore, while plain `REGION_COLORS[i]` reads stay correct.
*/
export let REGION_COLORS = blendPalette();
const regionColorListeners = new Set();
/** Snapshot accessor for useSyncExternalStore — new array identity per re-blend. */
export function getRegionColors() {
return REGION_COLORS;
}
/** Subscribe to palette re-blends (theme changes). Returns unsubscribe. */
export function subscribeRegionColors(cb) {
regionColorListeners.add(cb);
return () => regionColorListeners.delete(cb);
}
function refreshRegionColors() {
const next = blendPalette();
if (next.every((c, i) => c === REGION_COLORS[i])) return;
REGION_COLORS = next;
for (const cb of regionColorListeners) cb();
}
// Theme seam: App.jsx switches themes by setting/removing [data-theme] on
// <html> (index.css scopes every theme's --chrome-bg to that attribute), so
// observing it is exactly "re-read on theme change".
if (typeof document !== 'undefined' && typeof MutationObserver !== 'undefined') {
new MutationObserver(refreshRegionColors).observe(document.documentElement, {
attributes: true,
attributeFilter: ['data-theme'],
});
}
/**
* visibleSegmentRange windowing for the virtualized track.
*
+94 -1
View File
@@ -1,7 +1,11 @@
import { describe, it, expect } from 'vitest';
import { describe, it, expect, afterEach } from 'vitest';
import {
MIN_SEG_DUR,
MAX_OVERLAP,
REGION_COLORS,
blendRegionColor,
getRegionColors,
subscribeRegionColors,
visibleSegmentRange,
snapTime,
snapCandidates,
@@ -225,3 +229,92 @@ describe('nearestOnset', () => {
expect(nearestOnset(1, [])).toBeNull();
});
});
describe('REGION_COLORS — opaque JS-pre-blended paint guard (#373, #963)', () => {
// Two invariants, one per historical regression:
// #373 — semi-transparent box fills flash on some Windows GPU/WebView2
// drivers when the lane gets composited → every entry must be
// fully opaque (no alpha channel anywhere).
// #963 — engine-dependent CSS (color-mix, var()) in an inline style is
// REJECTED wholesale by the CSSOM on WebView2/Chromium < 111, and
// .seg-track__box has no background of its own → boxes invisible.
// Every entry must therefore be a literal rgb() any engine parses,
// with the 45%-tint-over---chrome-bg blend done in JS.
const root = document.documentElement;
const flushThemeObserver = () => new Promise((resolve) => setTimeout(resolve, 0));
afterEach(async () => {
root.style.removeProperty('--chrome-bg');
root.removeAttribute('data-theme');
await flushThemeObserver(); // let the palette settle back to the default
});
it('every entry is a literal fully-opaque rgb() — no engine-dependent CSS, no alpha', () => {
expect(REGION_COLORS.length).toBeGreaterThan(0);
for (const color of REGION_COLORS) {
expect(color).toMatch(/^rgb\(\d{1,3}, \d{1,3}, \d{1,3}\)$/);
// The class of the #963 bug: anything the target engines' CSSOM may
// reject as an inline-style value.
expect(color).not.toMatch(/color-mix|var\(|calc\(/i);
expect(color).not.toMatch(/rgba\(|hsla\(|transparent|\/|%/i); // #373: no alpha syntax
}
});
it('default theme: blends exactly 45% tint over Gruvbox --chrome-bg #0f1011', () => {
// Literal expected values (independently computed: round(0.45·tint + 0.55·bg)),
// pixel-identical to what `color-mix(in srgb, tint 45%, #0f1011)` painted.
expect([...REGION_COLORS]).toEqual([
'rgb(103, 69, 79)',
'rgb(67, 83, 78)',
'rgb(91, 93, 26)',
'rgb(121, 94, 31)',
'rgb(72, 95, 65)',
'rgb(123, 66, 21)',
'rgb(55, 79, 57)',
]);
});
it('re-blends against the new --chrome-bg when [data-theme] changes, and notifies', async () => {
const before = getRegionColors();
let notified = 0;
const unsubscribe = subscribeRegionColors(() => {
notified += 1;
});
try {
root.style.setProperty('--chrome-bg', '#1e293b'); // Slate theme surface
root.setAttribute('data-theme', 'slate');
await flushThemeObserver();
expect(notified).toBe(1);
expect(getRegionColors()).not.toBe(before); // fresh snapshot identity
// round(0.45·[211,134,155] + 0.55·[30,41,59])
expect(REGION_COLORS[0]).toBe('rgb(111, 83, 102)');
// Back to the default theme (attribute removed, like App.jsx does).
root.style.removeProperty('--chrome-bg');
root.removeAttribute('data-theme');
await flushThemeObserver();
expect(notified).toBe(2);
expect(REGION_COLORS[0]).toBe('rgb(103, 69, 79)');
} finally {
unsubscribe();
}
});
it('parses rgb()-form --chrome-bg too, and falls back to #0f1011 on garbage', async () => {
root.style.setProperty('--chrome-bg', 'rgb(30, 41, 59)');
root.setAttribute('data-theme', 'rgb-form');
await flushThemeObserver();
expect(REGION_COLORS[0]).toBe('rgb(111, 83, 102)'); // same blend as #1e293b
root.style.setProperty('--chrome-bg', 'oklch(0.2 0.1 250)'); // unsupported form
root.setAttribute('data-theme', 'garbage-form');
await flushThemeObserver();
expect(REGION_COLORS[0]).toBe('rgb(103, 69, 79)'); // fallback = default blend
});
it('blendRegionColor math: 0.45·tint + 0.55·bg, rounded per channel', () => {
expect(blendRegionColor([211, 134, 155], [15, 16, 17])).toBe('rgb(103, 69, 79)');
expect(blendRegionColor([0, 0, 0], [255, 255, 255])).toBe('rgb(140, 140, 140)'); // 0.55·255 = 140.25
expect(blendRegionColor([255, 255, 255], [0, 0, 0])).toBe('rgb(115, 115, 115)'); // 0.45·255 = 114.75
});
});
+1
View File
@@ -6,6 +6,7 @@
"moduleResolution": "bundler",
"jsx": "react-jsx",
"resolveJsonModule": true,
"allowImportingTsExtensions": true,
"allowJs": true,
"checkJs": true,
"allowSyntheticDefaultImports": true,
+7 -1
View File
@@ -115,6 +115,11 @@ def main(argv=None):
ap.add_argument("--target", required=True, help="Target language ISO code (e.g. de, ja, es).")
ap.add_argument("--voice", help="Voice profile ID to apply to every segment.")
ap.add_argument("--quality", choices=("fast", "cinematic"), default="fast")
ap.add_argument(
"--speakers", type=int, default=None, metavar="N",
help="Exact number of speakers in the source (1-20). Forwarded to "
"diarization so distinct voices don't blend; omit to auto-detect.",
)
ap.add_argument("--glossary", help="Path to a JSON glossary: [{source,target,note}]")
ap.add_argument("--api", default=os.environ.get("OMNIVOICE_API", "http://localhost:8000"))
args = ap.parse_args(argv)
@@ -155,7 +160,8 @@ def main(argv=None):
# 3. Transcribe (sync).
_log("→ transcribing…")
tx = _post(api, f"/dub/transcribe/{job_id}", {})
speakers_q = f"?num_speakers={args.speakers}" if args.speakers else ""
tx = _post(api, f"/dub/transcribe/{job_id}{speakers_q}", {})
segs = tx.get("segments", [])
_log(f"{len(segs)} segment(s), source={tx.get('source_lang')}")
+27 -7
View File
@@ -182,6 +182,29 @@ class OmniVoiceConfig(PretrainedConfig):
self.audio_codebook_weights = audio_codebook_weights
def _resolve_snapshot_dir(checkpoint) -> str:
"""Local snapshot directory for ``checkpoint`` (a local dir or a HF repo id).
Cache-first (#959): a COMPLETE local cache is resolved with
``snapshot_download(..., local_files_only=True)``, which never constructs
an HTTP session so no session-construction failure (e.g. httpx's
ImportError under ``ALL_PROXY``/``HTTPS_PROXY=socks5://`` without socksio,
a malformed proxy URL, a broken cert bundle) can break synthesis of an
already-installed model. Only a cache miss / incomplete cache falls
through to the original network ``snapshot_download``, whose errors
(auth, connectivity, proxy) surface exactly as before.
"""
if os.path.isdir(checkpoint):
return checkpoint
from huggingface_hub import snapshot_download
try:
return snapshot_download(checkpoint, local_files_only=True)
except Exception:
# Miss/incomplete (LocalEntryNotFoundError et al.) → network path.
return snapshot_download(checkpoint)
class OmniVoice(PreTrainedModel):
_supports_flex_attn = True
_supports_flash_attn_2 = True
@@ -264,13 +287,10 @@ class OmniVoice(PreTrainedModel):
)
if not train_mode:
# Resolve local path for audio tokenizer subdirectory
if os.path.isdir(pretrained_model_name_or_path):
resolved_path = pretrained_model_name_or_path
else:
from huggingface_hub import snapshot_download
resolved_path = snapshot_download(pretrained_model_name_or_path)
# Resolve local path for audio tokenizer subdirectory
# cache-first so a proxy-broken HTTP session can't fail an
# installed model (#959; see _resolve_snapshot_dir).
resolved_path = _resolve_snapshot_dir(pretrained_model_name_or_path)
model.text_tokenizer = AutoTokenizer.from_pretrained(
pretrained_model_name_or_path
+11 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "omnivoice"
version = "0.3.9"
version = "0.3.11"
description = "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models"
readme = "README.md"
# Free and open-source under the GNU Affero General Public License v3 (see
@@ -144,6 +144,16 @@ dependencies = [
# frozen `uv sync` installs a working engine on every platform.
"sherpa-onnx>=1.13.3",
"sherpa-onnx-core>=1.13.3",
# SOCKS proxy support for httpx (#959). huggingface_hub's get_session()
# builds an httpx.Client, which raises ImportError AT CONSTRUCTION when
# ALL_PROXY/HTTPS_PROXY is socks5:// and socksio isn't importable — every
# model load/download 500'd for SOCKS-proxy users ("Using SOCKS proxy, but
# the 'socksio' package is not installed"). Same failure shape for the
# OpenAI SDK's client. Pure-Python, MIT, zero transitive deps, ~13 KB —
# identical on macOS/Windows/Linux. Also in backend.spec hiddenimports:
# httpx imports it lazily inside try/except, so PyInstaller's tracer
# misses it and frozen installers would stay broken without the entry.
"socksio>=1.0",
]
[project.optional-dependencies]
+13 -3
View File
@@ -15,6 +15,11 @@
# ──────────────────────────────────────────────────────────────────────────
set -euo pipefail
# Always run from the repo root — every path below (frontend/, ${TAURI_DIR},
# …) is repo-root-relative, so invoking the script from any other directory
# used to mis-resolve them (#962 hardening).
cd "$(dirname "${BASH_SOURCE[0]}")/.."
APP_ID="com.debpalash.omnivoice-studio"
TAURI_DIR="frontend/src-tauri"
APP_NAME="OmniVoice Studio"
@@ -175,13 +180,18 @@ if [ "$SKIP_BUILD" = false ]; then
# The build creates the bundle successfully, but then may fail trying
# to sign the updater artifact (no TAURI_SIGNING_PRIVATE_KEY) or to
# run linuxdeploy. The binary itself is fine — tolerate known errors.
# #962: invoke the Tauri CLI via the frontend workspace's `tauri` script,
# NOT `bunx tauri`. In the bun workspace monorepo `@tauri-apps/cli` is a
# frontend/package.json dependency, and `bunx` resolves by npm package
# name — when the locally installed bin isn't exactly where bunx looks it
# falls back to fetching the unrelated `tauri` (v1) package from npm and
# dies with "could not determine executable to run for package tauri".
# `bun run --cwd frontend tauri` always resolves the workspace-local CLI.
BUILD_LOG=$(mktemp)
cd frontend
set +e
bunx tauri build --debug 2>&1 | tee "$BUILD_LOG"
bun run --cwd frontend tauri build --debug 2>&1 | tee "$BUILD_LOG"
BUILD_EXIT=$?
set -e
cd ..
if [ $BUILD_EXIT -ne 0 ]; then
# Known-harmless failures:
# - Missing TAURI_SIGNING_PRIVATE_KEY (updater signing)
+9 -3
View File
@@ -25,6 +25,10 @@
# ──────────────────────────────────────────────────────────────────────────
set -euo pipefail
# Always run from the repo root — every path below is repo-root-relative
# (#962 hardening, same as scripts/desktop-prod.sh).
cd "$(dirname "${BASH_SOURCE[0]}")/.."
APP_ID="com.debpalash.omnivoice-studio"
TAURI_DIR="frontend/src-tauri"
APP_NAME="OmniVoice Studio"
@@ -155,13 +159,15 @@ if [ "$SKIP_BUILD" = false ]; then
[ -d "$APP_BUNDLE" ] && rm -rf "$APP_BUNDLE"
fi
# #962: resolve the workspace-local Tauri CLI via the frontend package's
# `tauri` script — `bunx tauri` resolves by npm package name and can miss
# the workspace bin, then fetches the wrong npm package. Keep in sync
# with scripts/desktop-prod.sh.
BUILD_LOG=$(mktemp)
cd frontend
set +e
bunx tauri build --debug >"$BUILD_LOG" 2>&1
bun run --cwd frontend tauri build --debug >"$BUILD_LOG" 2>&1
BUILD_EXIT=$?
set -e
cd ..
if [ $BUILD_EXIT -ne 0 ]; then
if grep -qi "TAURI_SIGNING_PRIVATE_KEY\|private key\|failed to bundle" "$BUILD_LOG"; then
@@ -0,0 +1,38 @@
"""Regression: the lazy TTS/ASR registries must not raise "dictionary changed
size during iteration" when a lazy ``__getitem__`` inserts a resolved key while
another caller iterates the ``/engines`` 500 seen in production logs.
FastAPI runs ``list_backends()`` in a threadpool, so two concurrent ``/engines``
requests race: one iterates ``_REGISTRY.items()`` (which held a *live* dict
iterator open across the slow per-engine ``is_available()`` probes) while the
other materializes the lazy entry via ``__getitem__`` (``self[key] = cls``).
The insert then tripped the open iterator. ``__iter__`` now snapshots the live
keys up front (``list(dict.__iter__(self))``, atomic under the GIL), so a
concurrent insert can no longer trip the iteration.
The tests drive the exact crash site (``__iter__``) deterministically: begin
iterating, insert mid-iteration, then drain. Pre-fix this raises on the drain;
post-fix it completes.
"""
def _assert_iter_survives_concurrent_insert(reg):
it = iter(reg) # the generator items()/list_backends() drives
first = next(it) # first yield → the real-key snapshot is taken here
reg["zzz-concurrent-insert"] = object() # a concurrent lazy insert, mid-iteration
drained = [first, *it] # must NOT raise "dictionary changed size during iteration"
assert first in drained
def test_tts_lazy_registry_iter_survives_concurrent_insert():
from services.tts_backend import _LazyRegistry
reg = _LazyRegistry({"omnivoice": object(), "b": object(), "c": object()})
_assert_iter_survives_concurrent_insert(reg)
def test_asr_lazy_registry_iter_survives_concurrent_insert():
from services.asr_backend import _LazyASRRegistry
reg = _LazyASRRegistry({"whisperx": object(), "faster-whisper": object()})
_assert_iter_survives_concurrent_insert(reg)
+39
View File
@@ -152,6 +152,45 @@ def test_synthesize_empty_spans_is_silent():
assert dur == 0.0
def test_synthesize_chapter_2d_engine_output_with_pause():
# #897 regression: real engines return (1, samples) (TTSBackend contract)
# while inter-span pause silence was built as 1-D zeros — the final hard
# concat crashed a longform chapter render with
# "RuntimeError: Tensors must have same number of dimensions: got 1 and 2".
torch = pytest.importorskip("torch")
sr = 16000
def synth(text, voice_id, speed=None):
return torch.ones(1, 1000, dtype=torch.float32)
plan = parse_audiobook_script("First. [pause 1s] Second.", default_voice="v")
audio, dur = synthesize_chapter(plan.chapters[0].spans, synth, sr)
assert audio.shape == (1, 1000 + sr + 1000)
assert dur == pytest.approx((2000 + sr) / sr)
def test_synthesize_chapter_silence_rank_matches_engine_output(monkeypatch):
# The producer half of the #897 fix: pause silence is materialized with
# the rendered audio's rank, so the parts reaching the final concat are
# rank-homogeneous even without concatenate_audio_chunks' normalization.
torch = pytest.importorskip("torch")
import services.chunked_tts as ct
seen: list[list[int]] = []
real = ct.concatenate_audio_chunks
def spy(chunks, sr, crossfade_ms=50):
seen.append([c.dim() for c in chunks])
return real(chunks, sr, crossfade_ms=crossfade_ms)
monkeypatch.setattr(ct, "concatenate_audio_chunks", spy)
plan = parse_audiobook_script("First. [pause 500ms] Second.", default_voice="v")
synthesize_chapter(plan.chapters[0].spans,
lambda t, v, s=None: torch.ones(1, 100), 16000)
assert seen, "the stitcher never reached the concat"
assert all(d == 2 for dims in seen for d in dims)
def test_parse_applies_ssml_lite_prosody():
plan = parse_audiobook_script("[slow]hush[/slow] normal [spell]USA[/spell]")
spans = plan.chapters[0].spans
+77 -1
View File
@@ -14,10 +14,23 @@ import pytest
class _FakeTrimmer:
"""Minimal non-Off LLM stand-in that trims any line to a short fixed length
so the fit loop converges deterministically."""
def chat(self, *, system, user, timeout=None):
def chat(self, *, system, user, timeout=None, temperature=None):
return "A" * 14 # ratio 0.93 for slot 1.0s @ en 15 cps → inside [0.92, 1.0]
class _FakeChatter:
"""Non-Off LLM stand-in that always returns one fixed reply; counts calls."""
def __init__(self, reply: str):
self.reply = reply
self.calls = 0
self.last_temperature = None
def chat(self, *, system, user, timeout=None, temperature=None):
self.calls += 1
self.last_temperature = temperature
return self.reply
@pytest.fixture
def speech_rate(monkeypatch):
from services import speech_rate as _sr
@@ -53,3 +66,66 @@ def test_strict_within_tolerance_is_noop(speech_rate):
res = speech_rate.adjust_for_slot("A" * 15, slot_seconds=1.0, target_lang="en", strict=True)
assert res["attempts"] == 0
assert res["rate_ratio"] == pytest.approx(1.0, abs=0.01)
# ── Divergence guard (v0.3.9 field report: Autofit invented dialogue) ───────
def test_expand_hallucination_rejected_output_stays_input(monkeypatch):
"""The reported bug: a short line over a long slot invited the LLM to
fabricate a slot-filling wall of dialogue, and the accept step took ANY
non-empty reply (the best-tracker then favored the most-padded one). The
guard must discard it and keep the input text."""
from services import speech_rate as _sr
text = "B" * 24 # 1.6s @ en 15cps over a 10s slot → ratio 0.16
fake = _FakeChatter("A" * 145) # a "perfect" slot fill = ~6× the input line
monkeypatch.setattr(_sr, "get_active_llm_backend", lambda: fake)
res = _sr.adjust_for_slot(text, slot_seconds=10.0, target_lang="en", strict=True)
assert res["text"] == text # hallucination never adopted
assert res.get("error") == "fit-diverged"
assert fake.calls == _sr.MAX_ATTEMPTS # attempts burned, loop still bounded
def test_refusal_reply_rejected_keeps_input(monkeypatch):
"""A refusal/commentary reply must not replace a long line just because
its length happens to land near the slot budget."""
from services import speech_rate as _sr
text = "C" * 60 # 4s over a 1s slot → heavy trim ask
fake = _FakeChatter("Sorry, I cannot.") # 16 chars ≈ the 15-char slot budget
monkeypatch.setattr(_sr, "get_active_llm_backend", lambda: fake)
res = _sr.adjust_for_slot(text, slot_seconds=1.0, target_lang="en")
assert res["text"] == text
assert res.get("error") == "fit-diverged"
def test_legitimate_expansion_accepted(monkeypatch):
from services import speech_rate as _sr
text = "D" * 40 # ratio 0.67 over a 4s slot @ en
fake = _FakeChatter("E" * 57) # ~1.4× the line → ratio 0.95, honest fill
monkeypatch.setattr(_sr, "get_active_llm_backend", lambda: fake)
res = _sr.adjust_for_slot(text, slot_seconds=4.0, target_lang="en", strict=True)
assert res["text"] == "E" * 57
assert "error" not in res
def test_tiny_line_skips_llm_expansion(monkeypatch):
"""A line under 15% of its slot can never honestly fill it — the LLM must
not even be asked (it could only fabricate), and the line stays short."""
from services import speech_rate as _sr
fake = _FakeChatter("A" * 150)
monkeypatch.setattr(_sr, "get_active_llm_backend", lambda: fake)
res = _sr.adjust_for_slot("Hi.", slot_seconds=10.0, target_lang="en", strict=True)
assert res["text"] == "Hi."
assert res["attempts"] == 0
assert res.get("error") == "fit-skip-short"
assert fake.calls == 0
def test_fit_llm_call_pins_low_temperature(monkeypatch):
"""The fit pass pins temperature=0.2 like the Fast translate path — the
provider default of 1.0 is part of what made expansions drift."""
from services import speech_rate as _sr
fake = _FakeChatter("A" * 14)
monkeypatch.setattr(_sr, "get_active_llm_backend", lambda: fake)
_sr.adjust_for_slot("A" * 16, slot_seconds=1.0, target_lang="en", strict=True)
assert fake.last_temperature == 0.2

Some files were not shown because too many files have changed in this diff Show More