Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a721c1fdcf | ||
|
|
80f10289fe | ||
|
|
a0ad314736 | ||
|
|
1808a373a1 | ||
|
|
67789fb31c | ||
|
|
da4bef8e42 | ||
|
|
0076d0067e | ||
|
|
faf34348c8 | ||
|
|
69ce697ee5 | ||
|
|
93a3cb260a | ||
|
|
33379890ad | ||
|
|
7f8a42ce51 | ||
|
|
4de3c824d0 | ||
|
|
11fbbd3f6d | ||
|
|
f546f04c8e | ||
|
|
270b3b1c4c | ||
|
|
5a7d9cc05c | ||
|
|
5871ec9d28 | ||
|
|
58b023ff16 | ||
|
|
b61d17dd61 | ||
|
|
93aab6dadc | ||
|
|
7098be1c8b | ||
|
|
8520b84b68 | ||
|
|
549fa4009f | ||
|
|
dfe2bd1bc3 | ||
|
|
324d417d27 | ||
|
|
637020b82b | ||
|
|
dab6456581 | ||
|
|
015d29cafb | ||
|
|
1a03c59f82 | ||
|
|
8281b7c798 | ||
|
|
efe57ee49d | ||
|
|
7f77f4d7bf | ||
|
|
087309259b | ||
|
|
453db55f12 | ||
|
|
620321a9cc | ||
|
|
738d45f1c7 | ||
|
|
d959aae41b | ||
|
|
de99bc3bd5 | ||
|
|
c47633a409 | ||
|
|
4aa9abe22a | ||
|
|
d90cfde1bb | ||
|
|
bd92729623 | ||
|
|
afe819f498 | ||
|
|
6b91205036 | ||
|
|
1dfa032be4 |
@@ -118,9 +118,12 @@ jobs:
|
||||
working-directory: frontend
|
||||
run: bun run format:check
|
||||
|
||||
# `bun run test` (frontend/package.json), not `bunx vitest` — bunx
|
||||
# resolves by npm package name and can miss workspace-hoisted bins,
|
||||
# then falls back to fetching from npm (#962 class).
|
||||
- name: Run Vitest (frontend)
|
||||
working-directory: frontend
|
||||
run: bunx vitest run
|
||||
run: bun run test
|
||||
|
||||
# Legacy node:test runner for tests/frontend/*.test.mjs
|
||||
- name: Run frontend node:test (legacy)
|
||||
@@ -146,7 +149,7 @@ jobs:
|
||||
- os: windows-2022
|
||||
label: Windows
|
||||
rust_target: x86_64-pc-windows-msvc
|
||||
- os: ubuntu-22.04
|
||||
- os: ubuntu-24.04
|
||||
label: Linux
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
@@ -213,13 +213,24 @@ jobs:
|
||||
bundles: "msi,updater"
|
||||
|
||||
# Linux: ship .AppImage only. AppImage is universal (no distro
|
||||
# package-manager dep), runs on any glibc-2.31+ host, and is the
|
||||
# package-manager dep), runs on any glibc-2.39+ host, and is the
|
||||
# Linux auto-update target. The .deb target was dropped: tauri-bundler
|
||||
# fails it with "Failed to create control scripts: No such file or
|
||||
# directory" (no custom deb config of ours is at fault) — revisit on a
|
||||
# tauri-cli bump. FUSE unavailability on GH runners is handled via
|
||||
# APPIMAGE_EXTRACT_AND_RUN=1.
|
||||
- os: ubuntu-22.04
|
||||
#
|
||||
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
|
||||
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
|
||||
# repos resolve (see the "Linux system deps" step below) — 22.04's
|
||||
# was meaningfully stale relative to what current Ubuntu/Fedora
|
||||
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
|
||||
# take priority over a healthy system WebKitGTK at runtime. Raises
|
||||
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
|
||||
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
|
||||
# something that old has come in, and the project's own install
|
||||
# docs already assume Debian 12 / Ubuntu 22.04+.
|
||||
- os: ubuntu-24.04
|
||||
arch: x86_64-unknown-linux-gnu
|
||||
label: "Linux x64"
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
|
||||
@@ -6,6 +6,99 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
|
||||
The bundled TTS model package (`pyproject.toml`) is versioned independently.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.14] — 2026-07-09
|
||||
|
||||
A fast follow to v0.3.13: **every engine family now has a visible picker.** Settings → Engines showed only a TTS table, with the ASR and LLM pickers hidden behind a low-discoverability tab — so the 10 transcription engines (including the new OpenAI-compatible backend) looked unswitchable without env vars. Now all three families get their own table. Also in: the Linux AppImage's white-screen auto-workaround now checks the WebKitGTK it actually ships (not whatever your system reports), and installing to a different drive on Windows is properly documented.
|
||||
|
||||
### Added
|
||||
|
||||
- **ASR engines get the same Settings picker TTS has.** Settings → Engines now shows a visible picker table per family — TTS, ASR, and LLM — instead of a single TTS-titled table with the other families tucked behind a tab (README even promised a Settings ASR picker that didn't exist). The OpenAI-compatible backend and the 9 local ASR engines become selectable with one click, no env vars needed; an explicit `OMNIVOICE_ASR_BACKEND` still wins over the Settings pick, so pinned setups behave exactly as before. (no issue — UX gap found during #877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage's white-screen auto-workaround now checks the right WebKitGTK.** The launcher decided whether to apply the compositing workaround by asking the *system's* `pkg-config` — but the version that actually runs is the *bundled* one, which the AppImage prioritizes. On any machine where the two diverge (e.g. building from source with newer dev packages installed), the detection read the wrong number and could skip a workaround the running library needed. The build now stamps the bundled version into the AppImage at package time, and the launcher reads that stamp — correct by construction. The launcher's shell tests also now run in CI, which they previously never did. (#961 follow-up)
|
||||
|
||||
### Docs
|
||||
|
||||
- **Windows: installing to a different drive is documented** — the wizard's directory picker works for any local drive; mapped network drives are a Windows Installer limitation (not installable-to by design); and the big data (models/voices) moves independently via Settings → Storage or Portable mode. (#938)
|
||||
|
||||
## [0.3.13] — 2026-07-09
|
||||
|
||||
The community-fixes release. Two contributors didn't just report bugs — they diagnosed them to the exact line and submitted the fixes that shipped: **voice cloning on mlx-audio's CSM model works for the first time**, and **macOS live recording finally gets its microphone permission prompt** (both @MahdiHedhli). A third reporter's A/B analysis fixed **cross-language dubs speaking the wrong language**. On top of that: a backend shutdown race that produced confusing crash-on-quit reports is fixed, the Linux AppImage stops shipping a stale WebKitGTK that white-screened current distros, and a new OpenAI-compatible transcription backend opens a path to Qwen3-ASR today. Thank you to everyone who filed, diagnosed, and contributed — this release is mostly yours.
|
||||
|
||||
### Added
|
||||
|
||||
- **A path to Qwen3-ASR today: generic OpenAI-compatible transcription.** The direct integration is still blocked on `transformers>=5.13` stabilizing upstream, but a community member proposed splitting the work — add a backend that talks to any OpenAI-compatible transcription server right now. Point OmniVoice at a self-hosted Qwen3-ASR/FunASR/SenseVoice server, or OpenAI's own API, configured in Settings → Models. No install; audio does leave your machine to whichever server you configure, unlike every other ASR engine. (#877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage no longer white-screens on current distros with a healthy system WebKitGTK.** The release build ran on an older CI base image, and the resulting AppImage bundles whatever `libwebkit2gtk` that image's apt repos resolve — which the AppImage's own `LD_LIBRARY_PATH` then prioritizes over your system's newer, healthy copy at runtime. A from-source build (which links straight against your system library) worked fine on the exact same machine where the shipped AppImage didn't — that split was the tell. Bumped the release build to a current Ubuntu LTS. Raises the AppImage's minimum host to glibc 2.39 (Ubuntu 24.04+); no reports from anyone on an older distro. (#961)
|
||||
- **Backend shutdown no longer races a still-loading model, surfacing a confusing crash on restart.** Quitting the app while a model was still loading in the background let shutdown report itself "done" while a background thread was still mid-import; tearing the process down under that thread produced a misleading error (a generic transformers import-failure message, unrelated to the real cause) that looked like a real crash rather than a timing issue. All background tasks are now properly cancelled and awaited before shutdown proceeds. (#1000, likely the same class behind #941 and #979)
|
||||
- **Cross-language dub no longer speaks the source-language reference line verbatim.** Auto-generated speaker clones pair an audio slice with the ASR segment's own text field, assuming the two agree — but ASR segment text and its timestamps routinely drift (a trailing word audible in the clip but missing from the text, or vice versa). A mismatched (reference audio, reference text) pair breaks zero-shot TTS prompt priming badly enough that the clone can emit the reference text itself instead of the target-language line it was asked to speak. Each reference clip is now re-transcribed after it's written, so the pair matches by construction — reported with an exceptionally clear root-cause diagnosis and a working A/B repro. (#1004)
|
||||
- **Voice Gallery errors now say what actually went wrong.** "Use voice", "Preview", search, upload, save, delete, and trim in the Gallery all showed the same hardcoded guess ("the engine may be loading") on ANY failure — a 500, a validation error, a genuinely unrelated bug — discarding the real, already-clean backend error message in the process. Every one of those now shows the actual error.
|
||||
- **Voice cloning on mlx-audio's CSM model no longer crashes with an opaque "list index out of range".** `MLXAudioBackend.generate()` read `voice`/`ref_audio`/`language`/`speed` from its kwargs but silently dropped `ref_text` — CSM only builds its cloning context when both `ref_audio` and `ref_text` are present, so cloning on this engine could never have worked as shipped. Reported with the exact root cause and a working fix. (#1012, #1013)
|
||||
- **A dub segment's free-text style tags no longer 400 the segment preview.** A validator-safe instruct builder already keeps Studio and Clone generation from round-tripping a 400 on unsupported free-text (a preset's raw attrs, an old profile's stray descriptive phrase) — but the Dub tab's segment preview, and saving a profile from a clone or from history, built their instruct strings directly and skipped it. Same guard now applies everywhere an instruct string is sent. (#1010)
|
||||
- **The dub editor's play button no longer sticks permanently disabled after an audio-decode hiccup.** When the initial WaveSurfer decode fails, the timeline falls back to loading pre-computed peaks — the waveform draws fine, but the button's enabled state only relied on the `ready` event firing again for that recovery load, which it didn't reliably do. Each fallback path now confirms readiness explicitly once it settles.
|
||||
- **macOS: live recording finally works — the microphone permission prompt now actually appears.** The app never showed up in System Settings → Privacy & Security → Microphone because macOS never saw a legitimate request: Tauri enables Hardened Runtime by default, which blocks microphone hardware access unless the matching entitlement is in the signed bundle — and it wasn't. Diagnosed to the exact mechanism and fixed by a community contributor (@MahdiHedhli), who also corrected our initial mis-read of this as an upstream WebKit limitation. (#1013, #1016)
|
||||
- **Quitting during a slow model load waits longer before giving up.** A post-merge code review of the shutdown-race fix flagged that its 3-second wait could still be outrun by a cold model import on a slow disk, reproducing the original confusing-crash-on-quit in rare cases. The wait is now 20 seconds — imperceptible on a normal quit (tasks finish or cancel in milliseconds), only felt in the exact case it protects. (#1020)
|
||||
|
||||
### Changed
|
||||
|
||||
- **Removed the donate heart from the nav rail.** Support OmniVoice is still one click away from Settings and the Contact page.
|
||||
|
||||
### CI
|
||||
|
||||
- **The "flaky trio" is root-caused and neutralized.** Three tests failed intermittently on CI — never locally — across unrelated PRs, costing a re-run each time. Cause: a leaked half-precision torch default from some earlier test in CI's ordering (the giveaway: a failing assertion's observed value was exactly float16(0.1)). An autouse test-suite guard now resets the leak between tests and names the offending test in CI output when it fires. (#1021)
|
||||
|
||||
## [0.3.12] — 2026-07-08
|
||||
|
||||
A community-issue sweep — nineteen open reports triaged in one pass, most fixed same-day. The through-line: **your active engine selection is now honored everywhere** (dubbing, batch, and — new in this release — MLX-Audio's own curated models are finally selectable instead of always silently defaulting to Kokoro), **first-run stops dead-ending users on restricted networks or behind corporate TLS proxies**, and a run of sharp community diagnoses (a one-line ROCm index fix, a Windows-only focus-stealing bug, a genuine crash regression) got fixed largely because reporters did the hard diagnostic work themselves. Thank you.
|
||||
|
||||
### Added
|
||||
|
||||
- **MLX-Audio's other 6 curated models are finally selectable.** The engine multiplexes Kokoro, CSM, Qwen3-TTS, Dia, Chatterbox, MeloTTS, and OuteTTS, but there was no way anywhere in the UI or API to pick which one loads — downloading a model via Settings → Models did nothing, since the backend always defaulted to Kokoro regardless. Settings → Engines now shows a model picker on the mlx-audio row; switching takes effect immediately, no restart needed. (#981)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **First-run no longer dead-ends behind restricted networks (e.g. China).** The system check probed hardcoded huggingface.co, and any failure locked the Continue button — users behind the Great Firewall were stuck on the very first screen, even when they had already configured a working mirror. The check now probes the Hugging Face endpoint actually in effect, an unreachable endpoint is a warning instead of a blocker (models already on disk keep working offline), and when huggingface.co is blocked but the hf-mirror.com community mirror answers, the wizard says so and offers a one-click mirror switch right on the check screen — no restart needed. (#984)
|
||||
- **Installs behind a corporate or antivirus TLS-inspecting proxy no longer fail with a raw SSL error.** `SSLV3_ALERT_HANDSHAKE_FAILURE` happens when a proxy re-signs HTTPS traffic with a root CA your OS trusts but Python's bundled certificate list doesn't — a different failure mode from the network-blocking case above. OmniVoice now trusts your OS's certificate store directly, which should resolve the handshake outright rather than just explain it better. (#976)
|
||||
- **The loaded-models panel now says when a resident model is not your active engine.** Switching TTS engines keeps the previous model in VRAM (so switching back is instant) — but the panel showed it with no context, so "OmniVoice TTS — 1.9 GB" after selecting VoxCPM2 looked like the selection was ignored. A field report confirmed the confusion. Resident-but-inactive models are now tagged "not active — safe to unload", and the API self-describes each entry's engine. (#985)
|
||||
- **Voices no longer ship with a hidden echo.** Every non-raw synthesis was getting a small room reverb baked in by the mastering pre-stage — on top of whatever effect preset you chose, so even "Podcast" (which promises *no reverb*) had some, and Cinematic/Warm got it twice. A field report ("a lot of echo/reverb on some of the voices") led straight to it. The mastering stage is now highpass + compressor only; reverb happens only when a preset explicitly declares it. Also documented: cloned voices reproduce the reference clip's room acoustics — dry, close-mic references clone cleanest. (#986)
|
||||
- **Your engine selection now actually applies to Dubbing and Batch TTS.** Both hardcoded OmniVoice regardless of what was picked in Settings → Engines — pick VoxCPM2, dub anyway with OmniVoice, no error. Both now resolve the active engine up front; an engine that can't clone from reference audio (KittenTTS, Sherpa-ONNX, Supertonic 3 — fixed preset voices only) fails the job immediately with a clear message naming which engines do support it, instead of silently substituting OmniVoice or mis-cloning every speaker into one voice. Batch only requires cloning when a specific voice is pinned — an unpinned batch job runs on any engine. (#987)
|
||||
- **AMD ROCm torch install no longer silently falls back to CPU.** A community member (Kaihui-AMD) diagnosed it precisely: the ROCm wheel index we pointed at tops out at PyTorch 2.5.1, but the app pins `torch==2.8.0` — the reinstall was unsatisfiable and silently kept the default CUDA build, which runs on CPU on an AMD GPU. Bumped the default index to one that actually carries the pinned version. (#972)
|
||||
- **mlx-audio no longer crashes on unsupported languages.** Selecting a language like Dutch, Spanish, or Portuguese with mlx-audio's Kokoro model crashed with a raw, unreadable internal-details dump instead of a real error — the code was guessing an ISO language code by truncating the language name, which only worked by coincidence for a few languages. Unsupported languages now fail cleanly with a message naming what's actually supported, and no engine can leak a raw crash-internals dump into an error message again. (#977)
|
||||
- **The voice-design panel no longer crashes on certain saved voice profiles.** A genuine regression: an earlier translation fix accidentally introduced a crash when a saved design profile's data was incomplete (possible from an older app version or a partial save). Fixed at every layer — the render no longer crashes, both places that restore saved data complete it first, and profiles can no longer be *saved* with incomplete data in the first place. (#983)
|
||||
- **Windows: the dictation pill no longer steals focus.** Pressing the dictation shortcut activated the pill window, which meant the auto-paste landed back in OmniVoice instead of whatever app you were dictating into, and the pill would get stuck on screen. Precisely diagnosed by a community reporter; fixed to match how this already worked on macOS. (#982)
|
||||
- **The nemo-parakeet ASR engine's install hint no longer breaks your backend.** Following the in-app "pip install nemo_toolkit[asr]" instruction silently downgraded core packages your backend needs to start — the install reported success, and the breakage only showed up on the next restart. The hint now says plainly that this isn't safe to install into the shared environment. (#974)
|
||||
- **A stuck generate now tells you the actual fix.** When a job times out from GPU/VRAM contention, the error explained why but never mentioned Flush/Unload — the one action that actually resolves it, and one the sibling ASR-timeout error already recommended. (#939)
|
||||
|
||||
### Changed
|
||||
|
||||
- **README and Linux docs no longer advertise a `.deb` package that isn't published.** `.deb` bundling is disabled in the release pipeline pending a tauri-cli fix; the docs now say so honestly instead of pointing at a file that was never in any release. (#961 investigation, #990)
|
||||
- **Linux install docs mention `yt-dlp` as an optional prerequisite** — previously only surfaced via an in-app warning after the fact. (#973)
|
||||
- **A benign Tauri startup warning no longer looks like an app problem.** On some Windows configurations, Tauri's own internal IPC fallback logs a warning that's fully harmless (it silently and successfully falls back to another transport) — it was spuriously flipping the Settings → Logs footer to show "1 warning" on every launch. Filtered out of the diagnostic capture. (#975)
|
||||
|
||||
## [0.3.11] — 2026-07-05
|
||||
|
||||
The multi-language release — dubbing into several languages at once is finally a mature, honest workflow: **"Generate N dubs" now translates each language before rendering it** (with visible per-language progress), **switching languages never destroys your work** (every track keeps its own text, subtitles, and audio cache), completed tracks always show their tabs, and dialogue stops starting seconds early because of footsteps — a community reporter's theory, confirmed exactly. Around it, a reliability sweep driven by same-day field reports: your LLM provider finally survives a restart, SOCKS-proxy users can synthesize again (installed models now load without touching the network at all), timeline boxes are visible on every WebView2 runtime, running from source works again — and when the backend crashes, **it now tells you the exit code and attaches the evidence to your bug report automatically**.
|
||||
|
||||
### Added
|
||||
|
||||
- **Backend crashes are now self-documenting.** When the local backend process dies (a native GPU abort, an out-of-memory kill), the app used to show only "Can't reach the backend" — undiagnosable without logs nobody sends. The launcher now records every unexpected backend death (exit code, how long it ran, the last 40 log lines), tells you honestly that it *crashed* and is restarting, offers a "View crash details" panel, attaches the evidence to in-app bug reports automatically (paths scrubbed), and stops silent crash-loops after 3 deaths in 10 minutes with the details on screen. Intentional shutdowns, restarts, and app quits are never misreported as crashes. (#969)
|
||||
- **"Generate N dubs" now actually translates each language first.** Multi-language generation used to synthesize every track from whatever text was in the editor — so at most one of your N dubs was really in its language. The batch now runs translate → generate per language with a visible "Translating → Bengali (2/3)…" phase, skips (and reports) any language whose translation fails instead of rendering a wrong-language track, and your multi-language picks and export-track selection are saved with the project instead of vanishing on tab switch. (#957)
|
||||
- **Switching dub languages no longer destroys your work — every track keeps its own text and audio.** Translations are now stored per language (switching the target swaps the editor text non-destructively; manual edits stay with their language), subtitles export each track's own text instead of N identical files, burned-in subs match their track, and the per-segment audio cache is keyed by language — "Regen changed" can no longer splice another language's audio into the track you're rebuilding, and staleness is tracked per track. Fully backward-compatible: existing projects and caches keep working; a pre-upgrade project's first "Regen changed" simply regenerates cleanly once. (#958)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Timeline segment boxes are visible on every WebView2 runtime.** The v0.3.10 flicker fix switched box colors to a newer CSS feature (`color-mix`) applied as an inline style — on WebView2 runtimes older than ~March 2023 (pinned enterprise/offline installs) that renders as *fully transparent*, turning "flickering boxes" into "no boxes at all" while looking perfect on up-to-date machines. Colors are now pre-blended in plain JavaScript to universally-supported `rgb()` values — pixel-identical on modern runtimes, theme-aware, and guarded by a test that fails if an engine-dependent color ever reaches the timeline again. (#968)
|
||||
- **Dubbed dialogue stops starting seconds early because of footsteps.** Dialogue starts are snapped to the first detected sound — and a single 20 ms burst (footsteps, a door, a sigh) counted as "speech", with no limit on how far a start could jump, and the snap even ran on the raw mix when vocal separation had failed. Onsets now require sustained speech-like energy, long jumps are only allowed across genuinely silent spans (so the original fix for whisper's stretched starts keeps working), and snapping turns off entirely when vocals weren't separated. Credit to the community reporter whose "footsteps theory" was exactly right. (#967)
|
||||
- **Completed dub tracks always show their video tabs.** Opening a project with a finished dubbed track hid the Original/track switcher until you re-selected the language — visibility was keyed to the language dropdown instead of the project's tracks, and restored projects couldn't set the language because the history database froze it at empty forever. Tabs now render from the tracks themselves, history keeps its language (existing projects heal without migration), restoring a project can no longer 404 the video preview, and track pills gained duration/timing tooltips plus an accurate now-playing indicator. (#956)
|
||||
- **Running from source works again, and the install docs stop lying.** `bun run desktop-prod` broke when the frontend became a workspace (`bunx` could fetch the wrong "tauri" package from npm — fixed everywhere including CI); the Linux white-screen guidance now leads with the variable that actually fixes modern Ubuntu (`WEBKIT_DISABLE_DMABUF_RENDERER=1`, with the exact `EGL_BAD_PARAMETER` error quoted); Windows docs now state plainly that GPU acceleration is NVIDIA-only there; the Linux docs document the ROCm support that already shipped (the "planned follow-up" note was stale); and prerequisites are split installer-vs-source with git and curl included. (#964)
|
||||
- **Your LLM provider now survives a restart.** Setting up Ollama (or any provider), testing it, and saving looked like it worked — then a restart forgot the selection: only the separate "Save & use for translation" button ever persisted it, and a leftover setting from the retired (≤0.3.7) translation panel could silently steal the choice back to "Custom" on every launch. An explicit save now activates the provider when none was chosen yet, the leftover legacy settings are migrated into the Custom provider once and removed, and the panel says "Saved — not yet used for translation" instead of staying silent when your edit isn't the active provider. (#965)
|
||||
- **SOCKS-proxy users can synthesize again — and an installed model can never again be blocked by a broken network stack.** With a system-wide SOCKS proxy set, clicking Synthesize 500'd with a raw "socksio not installed" error: loading an already-downloaded model still constructed a network session first, which failed at creation. The app now ships SOCKS support (including in the packaged installers), resolves installed models **cache-first** (no network session when the files are already on disk — the local-first guarantee at the loader level), warms up at startup even when the online check fails, degrades LLM extras instead of crashing on proxy errors, and classifies the error with an actionable hint if it ever does surface. (#966)
|
||||
|
||||
## [0.3.10] — 2026-07-05
|
||||
|
||||
The listening release — nine fixes in twenty-four hours, almost all driven by your v0.3.9 field reports (several with same-day turnaround). The dubbing pipeline stops lying: **Cinematic and Autofit can no longer invent dialogue**, the **speaker count you set is honored on every path** (and auto-cloning stops fabricating voices from guessed labels), and the timeline stops flashing invisible on Windows. Audiobook chapters with pauses render again. And one fix everyone should want: **updating can no longer leave you secretly running the old version** — a leftover backend from a previous install holding the port is now detected and replaced at launch. Plus: the Dub tab's LLM engine finally runs on the provider you configured in Settings, history timestamps stop reading "20617d ago", and the Engines page can't crash under concurrent load.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
**OmniVoice Studio**
|
||||
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The latest stable release is **v0.3.5**; `main` rolls ahead at **v0.3.6** (latest release + 1 patch — see the Versioning rule below).
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
|
||||
|
||||
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
|
||||
|
||||
@@ -16,175 +16,21 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
|
||||
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
|
||||
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
|
||||
- **Local-first guarantee preserved**: Auto bug reporting (new addition) must be **opt-in**, must submit only to GitHub Issues (no third-party telemetry endpoint), and the app must remain fully functional with reporting disabled. No required cloud calls, accounts, or API keys.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release (currently **v0.3.5**). ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
<!-- GSD:project-end -->
|
||||
|
||||
<!-- GSD:stack-start source:research/STACK.md -->
|
||||
## Technology Stack
|
||||
|
||||
## Recommended Stack — Per Capability
|
||||
### Capability 1 — HuggingFace Token Persistence (issue #35)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `huggingface_hub` (already pinned transitively by `transformers>=5.3.0`) | `≥1.12.x` (latest 2026) | Auth + cache + token storage | Canonical, used by every HF library already in the stack. `HfFolder` is **superseded** in v1.x by the higher-level `login()` / `auth_list()` / `auth_switch()` API. |
|
||||
| `keyring` (Python) | `≥25.x` | Optional OS-keychain backing | Only adopt if a future hardening pass wants Keychain/Credential-Manager/SecretService. **Not recommended for this milestone** — adds a native dep (`dbus`, `pywin32`) per platform with no real security win over `0600` file storage in `HF_HOME`. |
|
||||
| Shell | One-liner to persist `HF_TOKEN` |
|
||||
|-------|---------------------------------|
|
||||
| macOS zsh (default since 10.15) | `echo 'export HF_TOKEN=hf_xxx' >> ~/.zshrc && source ~/.zshrc` |
|
||||
| Linux bash | `echo 'export HF_TOKEN=hf_xxx' >> ~/.bashrc && source ~/.bashrc` |
|
||||
| Windows PowerShell (user scope) | `[Environment]::SetEnvironmentVariable("HF_TOKEN","hf_xxx","User")` (new shells only) |
|
||||
| Windows cmd | `setx HF_TOKEN "hf_xxx"` (user scope, new shells only) |
|
||||
- [HF environment variables docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH confidence (official, current)
|
||||
- [HF authentication API docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH confidence
|
||||
- [Microsoft `setx` docs](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH confidence
|
||||
### Capability 2 — In-App Structured Bug Reporting (opt-in, GitHub Issues)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| GitHub REST API `POST /repos/{owner}/{repo}/issues` | `2026-03-10` API version | Server-side issue creation | Official, stable. Requires auth. |
|
||||
| **Prefilled-URL pattern** (`github.com/{owner}/{repo}/issues/new?title=…&body=…&labels=…`) | n/a | Zero-auth fallback | **This is the recommended primary path for v0.3.x.** No token needed, no GitHub App registration needed, user's browser opens with a prefilled form, they review and click Submit. They own the issue, the OSS project gets the report, and OmniVoice never holds a credential. |
|
||||
| `gh-app-jwt` + GitHub App (Rust crate `octocrab` or Python `pygithub`) | only if we later want fully-automated submission | Programmatic posting under an app identity | **Defer to a later milestone.** Requires registering a public GitHub App, hosting a token-exchange endpoint, and managing rate-limit quotas — disproportionate for stabilization scope. |
|
||||
| `platform`, `psutil`, `torch.cuda` (already in deps) | already pinned | Capture OS, CPU/GPU/VRAM info | No new deps. |
|
||||
| `httpx` (already in `dev-dependencies`, promote to runtime if needed) | `≥0.28.1` | HTTP for the API call path (if/when we add auth) | Modern async-first, already used in test suite. |
|
||||
- ✓ No token storage in OmniVoice → no security surface
|
||||
- ✓ Opt-in by definition (user has to click Submit on github.com)
|
||||
- ✓ User owns the issue → can be replied to, edited, closed by them
|
||||
- ✓ Zero infra cost — no proxy, no app, no rate-limit management
|
||||
- ✓ Works identically on macOS / Windows / Linux via Tauri's `shell.open`
|
||||
- ✓ Survives our project being forked (just change the URL)
|
||||
- OS name + version (`platform.platform()`)
|
||||
- Python version (`sys.version`)
|
||||
- OmniVoice version (`pyproject.toml`)
|
||||
- Backend git SHA (if installed from source) or installer build ID
|
||||
- CPU model, RAM (`psutil.cpu_count()`, `psutil.virtual_memory()`)
|
||||
- GPU vendor/model/VRAM (`torch.cuda.get_device_name()`, `torch.cuda.mem_get_info()`, MPS detect)
|
||||
- Active TTS engine + list of installed engines
|
||||
- Frontend: bun version, OS shell
|
||||
- Last error message + stack trace if launched from an error toast
|
||||
- Audio file contents (privacy — reference samples may contain user's voice)
|
||||
- File paths containing `/Users/<name>/` (strip home dir → `~/`)
|
||||
- HF token, OpenAI keys, any env var matching `*TOKEN*|*KEY*|*SECRET*`
|
||||
- [GitHub URL query parameters for issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/creating-an-issue#creating-an-issue-from-a-url-query) — HIGH confidence
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (widely used reference impl)
|
||||
- [GitHub REST API: Create an issue](https://docs.github.com/en/rest/issues/issues#create-an-issue) — HIGH confidence (for the future auto-submit path)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — reviewed, **rejected for milestone** due to local-first constraint
|
||||
### Capability 3 — `uv venv` Mirror Fallback for Restricted Networks (issues #57, #60)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `uv` (already used) | `≥0.5.x` | Python+venv bootstrap | Existing dep. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` env var | uv `0.4.x`+ | Override python-build-standalone download URL | **Official, current.** Replaces `https://github.com/astral-sh/python-build-standalone/releases/download/...` in download URL construction. No built-in fallback if mirror fails. |
|
||||
| `UV_PYTHON_PREFERENCE=only-system` (or CLI flag `--python-preference only-system`) | uv `0.4.x`+ | Skip the python-build-standalone download entirely; use the user's system Python | **The reliable escape hatch** when no mirror works. Requires a compatible Python `>=3.11` to already be on PATH. |
|
||||
| `UV_HTTP_TIMEOUT`, `UV_HTTP_CONNECT_TIMEOUT`, `UV_HTTP_RETRIES` | uv `0.4.x`+ | Tune retry behavior for flaky links | Defaults are 30s / 10s / 3 — bump to 120s / 30s / 5 for restricted networks. |
|
||||
# Pseudocode for the bootstrap
|
||||
# Final fallback: don't download Python at all
|
||||
- `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (Tsinghua — fastest in China)
|
||||
- `UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple` (Aliyun fallback)
|
||||
- Russia: no major government-blessed PyPI mirror; users typically tunnel via VPN. Document this honestly rather than ship a broken default.
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (official)
|
||||
- [uv issue #5224 — python-build-standalone mirror support](https://github.com/astral-sh/uv/issues/5224) — HIGH (the feature was added)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms real user pain, no built-in fallback)
|
||||
- [uv python-versions concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH (documents `python-preference` semantics)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained mirror list; verify each URL still works before shipping)
|
||||
### Capability 4 — Supertonic-3 TTS Engine
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `supertonic` (PyPI) | `1.3.1` (latest, May 18 2026 — Phase 3 Wave 1 to verify constructor signature before bump) | Official Supertonic-3 inference SDK | Authoritative wrapper from Supertone Inc. Wraps the ONNX session orchestration so we don't have to. |
|
||||
| `onnxruntime` | `≥1.17.x` (any recent) | ONNX inference runtime | Already a transitive dep of WhisperX (via CTranslate2 path is separate, but `onnxruntime` itself ships for kittentts and audioseal). Verify with `uv tree` after adding — should resolve cleanly. |
|
||||
| `huggingface_hub` (already pinned) | `≥1.12.x` | Model weight download (~400 MB on first use) | Reuses existing HF token + cache infrastructure. The user's existing `HF_TOKEN` (Capability 1) works for the Supertonic model download too. |
|
||||
| `numpy`, `soundfile` (already pinned) | already pinned | Audio I/O + array math | No new deps. |
|
||||
- `text_encoder.onnx`
|
||||
- `latent_denoiser.onnx`
|
||||
- `voice_decoder.onnx`
|
||||
- 44.1 kHz sample rate, 24-dim latent, 128-dim style
|
||||
- ~99M parameters total
|
||||
- Tokenizer: `AutoTokenizer.from_pretrained(model_path)` — loads from `tokenizer.json` shipped with model
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official)
|
||||
- [supertonic PyPI page](https://pypi.org/project/supertonic/) — HIGH (`1.3.1` confirmed 2026-05-18; same publisher, MIT, same 4 deps)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure details)
|
||||
### Capability 5 — Cross-Platform Documentation Tooling
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| Plain Markdown in `docs/` + GitHub-rendered (current state) | n/a | Install tutorial, troubleshooting | Zero new infra. Renders inline on GitHub for issue-replies. No build step to break. |
|
||||
| Existing `scripts/smoke-test.sh` + Playwright `tests/` (already in `package.json`) | already pinned | Verify install paths actually work | **This is the real solution to "docs drift."** If smoke-test exercises the install path described in docs, docs that drift will break CI. |
|
||||
| **Future** (defer): Astro Starlight | `≥0.30` | Standalone docs site at `docs.omnivoice.studio` | Adopt only when docs exceed ~20 markdown files and need search/versioning. Tauri, the framework OmniVoice already depends on, uses Starlight — well-traveled choice. Material for MkDocs entered maintenance mode in November 2025 per Docsio's 2026 review — **avoid** for new docs. |
|
||||
| Project | What they do |
|
||||
|---------|--------------|
|
||||
| **OBS Studio** | Docs at `obsproject.com/docs` (Sphinx, separate repo). Install paths in README, wiki for community-contributed. CI doesn't gate on docs drift. |
|
||||
| **Audacity** | Manual at `manual.audacityteam.org` (MediaWiki). README is minimal. Install path = "use the installer." No automated sync. |
|
||||
| **Tauri** | Docs at `v2.tauri.app` (Astro Starlight, separate repo `tauri-apps/tauri-docs`). README is minimal. Heavy reliance on community contributions and PR review. |
|
||||
| **VS Code** | Docs at `code.visualstudio.com/docs` (separate repo, Markdown). README is minimal. Manual sync; docs team is staffed. |
|
||||
- [Tauri docs (Astro Starlight)](https://github.com/tauri-apps/tauri-docs) — HIGH (reference for "if we ever move off README")
|
||||
- [OBS Studio docs](https://docs.obsproject.com/) — HIGH (Sphinx, separate site reference)
|
||||
- [Audacity Manual](https://manual.audacityteam.org/) — HIGH (MediaWiki reference)
|
||||
- [Docsio: Material for MkDocs 2026 review (maintenance mode)](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with project's own GitHub activity)
|
||||
- [Docsio: Starlight 2026 review](https://docsio.co/blog/starlight-docs) — MEDIUM
|
||||
## Installation
|
||||
# No new Python dependencies needed for Capabilities 1, 2, 3, 5.
|
||||
# Only Capability 4 adds a runtime dep:
|
||||
# Verify no regressions:
|
||||
# Should show single versions of each; no duplicates.
|
||||
## Alternatives Considered
|
||||
| Recommended | Alternative | When to Use Alternative |
|
||||
|-------------|-------------|-------------------------|
|
||||
| HF token via in-app Settings → `huggingface_hub.login()` | OS keyring via `keyring` package | Only if a security hardening milestone later demands OS-native credential storage. Not worth the cross-platform native-dep cost for v0.3.x. |
|
||||
| Prefilled-URL GitHub Issues | GitHub App + device flow + authenticated POST | When milestone budget can afford registering a public GitHub App and hosting a token-exchange function. Defer. |
|
||||
| Prefilled-URL GitHub Issues | Sentry / `sentry-tauri` | Never — violates the "no third-party telemetry endpoint" constraint in PROJECT.md. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` chain + `only-system` fallback | Bundle Python in the Tauri installer | Adds ~30 MB to every installer for ~5% of users. Revisit if the bootstrap is still a top complaint in v0.4. |
|
||||
| In-repo Markdown docs | Astro Starlight standalone site | When docs grow past ~20 pages and need full-text search. Tauri provides a precedent if/when we get there. |
|
||||
| In-repo Markdown docs | MkDocs / Material for MkDocs | **Avoid** for new sites — Material for MkDocs is in maintenance mode as of Nov 2025. |
|
||||
## What NOT to Use
|
||||
| Avoid | Why | Use Instead |
|
||||
|-------|-----|-------------|
|
||||
| `HfFolder.save_token()` directly | Older API; v1.x `login()` does the same plus git-credential integration and is the documented path | `huggingface_hub.login(token=val, add_to_git_credential=False)` |
|
||||
| Setting `HF_TOKEN` via shell rc files as the *only* persistence mechanism | Different per OS, fragile, opaque to the user, breaks in installer-launched processes that don't source shell rc | Write to `$HF_HOME/token` via `login()`. Document env var as override only. |
|
||||
| `setx` for HF token persistence | Doesn't propagate to current shell; common source of "I set it but it's empty" bug reports | `[Environment]::SetEnvironmentVariable(...,"User")` in PowerShell, or the in-app Settings field |
|
||||
| PAT-based GitHub Issues posting from OmniVoice | Would require shipping or asking for a token; breaks local-first promise | Prefilled-URL pattern (user submits from their browser) |
|
||||
| `sentry-tauri` for OmniVoice | Third-party telemetry endpoint — violates PROJECT.md constraint | Local-only `backend.log` rotation + opt-in prefilled-URL reporter |
|
||||
| `hf_transfer` for downloads | Deprecated in favor of `hf-xet` per HF docs | Default `huggingface_hub` (uses `hf-xet` automatically when available) |
|
||||
| `--python-preference managed` (default) without mirror config in restricted-network installers | Hits GitHub CDN, times out, user sees raw `uv` error | Configure `UV_PYTHON_INSTALL_MIRROR` + retry chain + `only-system` final fallback |
|
||||
| Material for MkDocs as a *new* docs choice | Entered maintenance mode November 2025 | If docs site is eventually needed, use Astro Starlight (Tauri precedent) |
|
||||
## Stack Patterns by Variant
|
||||
- Set `UV_PYTHON_INSTALL_MIRROR` to one of the gh-proxy URLs at install time
|
||||
- Set `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (China) or document VPN requirement (Russia)
|
||||
- Fall back to `UV_PYTHON_PREFERENCE=only-system` if all mirrors fail
|
||||
- Increase `UV_HTTP_TIMEOUT=120`, `UV_HTTP_RETRIES=5`
|
||||
- Default path: in-app Settings field → `login()` → file at `$HF_HOME/token`
|
||||
- Power-user path: `export HF_TOKEN=...` in shell rc (documented but not promoted)
|
||||
- Both paths are read at HF library import time; env var wins on conflict
|
||||
- Default path: in-app "Report a bug" → prefilled GitHub Issues URL → user reviews + submits in browser
|
||||
- All optional capture toggles default ON except "include reproduction file" (privacy)
|
||||
- No path posts to any URL except `github.com/{owner}/{repo}/issues/new` (rendered locally as a URL, opened via `shell.open`)
|
||||
- `uv add supertonic` → new TTSBackend subclass in `backend/services/tts_backend.py`
|
||||
- Auto-detected and added to the engine picker in Settings
|
||||
- ~400 MB model download on first synthesize call, cached in `$HF_HUB_CACHE`
|
||||
- Existing IndexTTS/CosyVoice/etc. installs are untouched (no shared model weights)
|
||||
## Version Compatibility
|
||||
| Package A | Compatible With | Notes |
|
||||
|-----------|-----------------|-------|
|
||||
| `supertonic@1.3.1` | `onnxruntime>=1.17`, `numpy>=1.24`, `huggingface_hub>=0.20` | All deps already satisfied transitively by current `pyproject.toml`. |
|
||||
| `huggingface_hub>=1.12` | `transformers>=5.3.0` (current pin) | `HfFolder` retained as deprecated alias; `login()`/`get_token()` are the canonical APIs. |
|
||||
| `uv>=0.5` | `UV_PYTHON_INSTALL_MIRROR`, `UV_PYTHON_PREFERENCE` | Both env vars stable since uv 0.4.x. |
|
||||
| Tauri v2 + `@tauri-apps/api/shell` | `shell.open()` for the prefilled-URL pattern | Already in the desktop app; no new permission needed beyond what the existing "open external link" plugin grants. |
|
||||
## Sources
|
||||
- [Hugging Face Hub environment variables](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH (verified against v1.12.1 docs, current 2026)
|
||||
- [Hugging Face Hub authentication API](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH (verified `login()` is the canonical 1.x API)
|
||||
- [Microsoft `setx` reference](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH (confirms "current shell" gotcha)
|
||||
- [PowerShell `about_Environment_Variables`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables) — HIGH
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (verified all mirror + retry env vars)
|
||||
- [uv issue #5224 — python-build-standalone mirror](https://github.com/astral-sh/uv/issues/5224) — HIGH (feature shipped)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms user pain, justifies fallback chain)
|
||||
- [uv `python-preference` semantics](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official, 99M params, 31 languages, OpenRAIL-M)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official inference API)
|
||||
- [supertonic 1.3.1 on PyPI](https://pypi.org/project/supertonic/) — HIGH (released 2026-05-18, MIT code license; bumped from 1.2.3 after Phase 3 research)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure)
|
||||
- [GitHub Docs: Authenticating to the REST API](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api) — HIGH
|
||||
- [GitHub Docs: Generating a user access token for a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) — HIGH (device flow reference)
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (canonical prefilled-URL reference impl)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — MEDIUM (reviewed, rejected on PROJECT.md constraint, not on quality)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained, verify URLs are still live before pinning in production)
|
||||
- [Tauri 2 docs (Astro Starlight reference)](https://v2.tauri.app/) — HIGH (precedent for docs framework if we ever migrate)
|
||||
- [Docsio: Material for MkDocs entered maintenance mode Nov 2025](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with the project's own GitHub commit activity)
|
||||
The May-2026 stack research that used to live here served five capabilities that have all since shipped (HF-token Settings panel, prefilled-URL bug reporting, uv mirror fallback for restricted networks, the Supertonic-3 engine, in-repo Markdown docs). Follow the patterns in the code itself; the durable *don'ts* that research established:
|
||||
|
||||
- **No third-party telemetry endpoints, ever** (`sentry-tauri` was evaluated and rejected) — bug reporting stays opt-in via prefilled GitHub-issue URLs.
|
||||
- **No PAT/token-based GitHub posting from the app** — the user submits from their own browser.
|
||||
- **Don't recommend `setx` for env vars on Windows** (silent truncation, no current-shell propagation) — use the in-app Settings panel or PowerShell `[Environment]::SetEnvironmentVariable`.
|
||||
- **Don't adopt Material for MkDocs** for any future docs site (maintenance mode since Nov 2025) — Astro Starlight is the precedent if docs ever outgrow the repo.
|
||||
- **`hf_transfer` is deprecated** — default `huggingface_hub` (hf-xet) handles downloads.
|
||||
|
||||
For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/package.json`, and check `uv tree` for conflicts before adding a dependency.
|
||||
<!-- GSD:stack-end -->
|
||||
|
||||
<!-- GSD:conventions-start source:CONVENTIONS.md -->
|
||||
@@ -222,16 +68,9 @@ No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skill
|
||||
<!-- GSD:skills-end -->
|
||||
|
||||
<!-- GSD:workflow-start source:GSD defaults -->
|
||||
## GSD Workflow Enforcement
|
||||
## Workflow
|
||||
|
||||
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
|
||||
|
||||
Use these entry points:
|
||||
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
|
||||
- `/gsd-debug` for investigation and bug fixing
|
||||
- `/gsd-execute-phase` for planned phase work
|
||||
|
||||
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
|
||||
Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command gate that used to live here referenced `/gsd-quick` / `/gsd-debug` / `/gsd-execute-phase` skills that are not installed in this environment; the owner chose to keep working directly rather than restore them. The working conventions that matter are in **Conventions** above — versioning, docs-sync, changelog, localization, fix quality, keep-main-green — plus: gate every merge on the "Tests (backend + frontend)" check passing and the PR being MERGEABLE, and check the open-PR queue before implementing any community-reported fix (contributors may have already submitted one).
|
||||
<!-- GSD:workflow-end -->
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ Thanks for your interest in improving OmniVoice Studio! This guide covers everyt
|
||||
### Prerequisites
|
||||
|
||||
- [Git](https://git-scm.com/)
|
||||
- `curl` (used by the Bun / uv / rustup install one-liners on macOS and Linux)
|
||||
- [Bun](https://bun.sh/) (frontend package manager)
|
||||
- [uv](https://docs.astral.sh/uv/) (Python environment manager)
|
||||
- [ffmpeg](https://ffmpeg.org/) (audio/video processing)
|
||||
|
||||
@@ -170,7 +170,7 @@ The eight headliners — and twelve more waiting under the fold.
|
||||
- 📦 **Batch Queue** — drop 50 videos, walk away; per-job progress bars.
|
||||
- 🛡️ **AI Watermark** — AudioSeal (Meta): invisible, survives compression.
|
||||
- 🔬 **Diagnostics** — self-check suite, error journal, scrubbed diagnostic bundles.
|
||||
- ⚡ **GPU Auto-Detect** — CUDA · MPS · ROCm · CPU; ≤8 GB VRAM auto-offloads.
|
||||
- ⚡ **GPU Auto-Detect** — CUDA · MPS · ROCm (Linux, opt-in) · CPU; ≤8 GB VRAM auto-offloads.
|
||||
- 🧭 **Engine routing** — preflight GPU check per engine; no silent CPU fallback.
|
||||
- 🧩 **Extensible** — subclass `TTSBackend`, add any engine in ~50 lines.
|
||||
- 🎒 **Portable personas** — export voices as `.ovsvoice` bundles: identity + watermark.
|
||||
@@ -190,7 +190,6 @@ The eight headliners — and twelve more waiting under the fold.
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
|
||||
<br/>
|
||||
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy & Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
|
||||
<br/>
|
||||
@@ -244,7 +243,7 @@ ElevenLabs charges **$5–$330/mo** and processes your audio on their servers. O
|
||||
| **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
|
||||
| **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
|
||||
| **API Keys** | Required | Not needed |
|
||||
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm · CPU |
|
||||
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm (Linux) · CPU |
|
||||
| **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
|
||||
| **TTS Engines** | 1 | **14** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS) |
|
||||
| **ASR Engines** | 1 | **9** (WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR, isolated Faster-Whisper, sherpa-onnx live dictation) |
|
||||
@@ -267,16 +266,19 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
| | **Minimum** | **Recommended** |
|
||||
|---|---|---|
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 20.04+ | Any modern 64-bit OS |
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
|
||||
| **RAM** | 8 GB | 16 GB+ |
|
||||
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
|
||||
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
|
||||
| **Python** | 3.10+ (managed by `uv`) | 3.11–3.12 |
|
||||
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm |
|
||||
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm (Linux only) |
|
||||
|
||||
> [!TIP]
|
||||
> On GPUs with **≤8 GB VRAM**, OmniVoice automatically offloads TTS to CPU during transcription — no config needed. A dedicated GPU is not required; the entire pipeline runs on CPU (just slower).
|
||||
|
||||
> [!NOTE]
|
||||
> **AMD GPUs:** ROCm acceleration is **Linux-only and opt-in** — pick **"AMD GPU (ROCm)"** on the first-run setup screen or set `OMNIVOICE_TORCH_VARIANT=rocm` ([docs/install/linux.md](docs/install/linux.md#amd-gpu-rocm)). **On Windows, AMD GPUs (incl. Ryzen AI iGPUs) run CPU-only**: PyTorch has no Windows ROCm wheels, so Windows GPU acceleration is NVIDIA/CUDA-only ([docs/install/windows.md](docs/install/windows.md#gpu-support)).
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **macOS Intel (x86_64) is unsupported for the local backend:** the app UI installs, but the Python backend cannot run because PyTorch no longer ships Intel-Mac wheels ([#889](https://github.com/debpalash/OmniVoice-Studio/issues/889)). Intel-Mac users can still point the UI at a remote backend on another machine — see [docs/install/macos.md](docs/install/macos.md).
|
||||
|
||||
@@ -284,7 +286,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🗣️ TTS Engines
|
||||
|
||||
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
|
||||
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var — the selection applies everywhere synthesis happens: single-clip generation, Voice Cloning, Video Dubbing, and Batch TTS.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
|
||||
@@ -310,6 +312,8 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
|
||||
>
|
||||
> **Clone** matters beyond single-clip generation: Video Dubbing (and any Batch job with a pinned voice) needs reference-audio cloning to preserve speaker identity, so picking a Clone-less engine (KittenTTS, Sherpa-ONNX, Supertonic 3) as the active engine fails those jobs up front with an actionable message instead of silently falling back to OmniVoice.
|
||||
>
|
||||
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
|
||||
|
||||
</details>
|
||||
@@ -318,10 +322,10 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🎧 ASR Engines
|
||||
|
||||
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
|
||||
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines** (the ASR Engines table — same picker TTS has), or pin one with the `OMNIVOICE_ASR_BACKEND` env var (the env var wins over the Settings pick). Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
|
||||
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -336,6 +340,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
|
||||
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
|
||||
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
|
||||
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
|
||||
|
||||
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
|
||||
|
||||
|
||||
@@ -33,6 +33,14 @@ hiddenimports = [
|
||||
'uvicorn.lifespan', 'uvicorn.lifespan.on',
|
||||
'fastapi', 'fastapi.responses', 'starlette',
|
||||
'multipart',
|
||||
# SOCKS proxy support (#959). httpx imports socksio lazily inside a
|
||||
# try/except (only when a socks5:// proxy env var is set), so
|
||||
# PyInstaller's static tracer never sees it — without this entry the
|
||||
# frozen installers keep raising "Using SOCKS proxy, but the 'socksio'
|
||||
# package is not installed" on every model load under a SOCKS proxy,
|
||||
# even though pyproject.toml ships the package. Guarded by
|
||||
# tests/test_socks_proxy.py.
|
||||
'socksio',
|
||||
|
||||
# Core
|
||||
'uuid', 'asyncio',
|
||||
|
||||
@@ -179,6 +179,21 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
job["status"] = "failed"
|
||||
return
|
||||
|
||||
# ── Engine resolution (issue #312 class) ────────────────────────────
|
||||
# Batch used to hardcode OmniVoice via get_model() regardless of the
|
||||
# engine selected in Settings → Engines. require_cloning only when a
|
||||
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
|
||||
# any active engine. Resolved ONCE for the whole job (every language
|
||||
# below shares the same active engine); an uncaught ValueError here
|
||||
# propagates to _worker()'s existing except-Exception handling, which
|
||||
# already records a structured job failure via core.failure.build_failure.
|
||||
from services.tts_backend import resolve_generation_backend
|
||||
backend = await resolve_generation_backend(
|
||||
require_cloning=bool(job.get("voice_id")),
|
||||
cloning_purpose="this batch job's pinned voice",
|
||||
)
|
||||
sr = backend.sample_rate
|
||||
|
||||
# ── 3. Translate + Generate per language ───────────────────────────
|
||||
total_langs = len(langs)
|
||||
outputs = {}
|
||||
@@ -243,13 +258,10 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
total_segments=len(translated_segments),
|
||||
)
|
||||
|
||||
from services.model_manager import get_model
|
||||
from services.audio_dsp import apply_mastering, normalize_audio
|
||||
from services.audio_io import atomic_save_wav
|
||||
import torch
|
||||
|
||||
_model = await get_model()
|
||||
sr = _model.sampling_rate
|
||||
total_samples = int(duration * sr)
|
||||
full_audio = torch.zeros(1, total_samples)
|
||||
total_segs = len(translated_segments)
|
||||
@@ -295,22 +307,16 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
ref_text = row.get("ref_text")
|
||||
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
duration=dur, num_step=16,
|
||||
guidance_scale=2.0, speed=1.0,
|
||||
denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered = apply_mastering(
|
||||
audio_out,
|
||||
sample_rate=sr,
|
||||
)
|
||||
return normalize_audio(mastered, target_dBFS=-2.0)
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
audio_out = apply_mastering(audio_out, sample_rate=sr)
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
except Exception as e:
|
||||
logger.warning("TTS failed for seg %d (lang=%s): %s", i, lang, e)
|
||||
return torch.zeros(1, int(dur * sr))
|
||||
|
||||
@@ -452,6 +452,11 @@ async def dub_transcribe_stream(
|
||||
asr_audio_target = job.get("vocals_path")
|
||||
if not asr_audio_target or not os.path.exists(asr_audio_target):
|
||||
asr_audio_target = job.get("audio_path")
|
||||
# #963: onset snapping is only trustworthy on the Demucs vocals
|
||||
# track. When separation failed/was skipped, dub_pipeline sets
|
||||
# vocals_path to the mixed audio_path — so compare paths instead
|
||||
# of trusting the key's presence.
|
||||
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
|
||||
if not asr_audio_target or not os.path.exists(asr_audio_target):
|
||||
preflight_error = "No audio available for transcription."
|
||||
else:
|
||||
@@ -649,9 +654,12 @@ async def dub_transcribe_stream(
|
||||
# leading music/silence (classic case: speech begins at 0:03,
|
||||
# transcript says 0.0 → the dub plays 3 s early). Snap starts
|
||||
# forward to the actual speech onset. `audio_np` is the same
|
||||
# track ASR ran on — vocals.wav when Demucs succeeded.
|
||||
# track ASR ran on — vocals.wav when Demucs succeeded. #963:
|
||||
# when it didn't (mixed audio), snapping is disabled — every
|
||||
# footstep/sigh/score cue is a false onset candidate there.
|
||||
try:
|
||||
snap_segment_starts(chunk_segs, audio_np, sr)
|
||||
snap_segment_starts(chunk_segs, audio_np, sr,
|
||||
separated_vocals=asr_on_vocals)
|
||||
except Exception as e:
|
||||
logger.warning("onset alignment skipped for chunk %d: %s", i, e)
|
||||
# Provisional per-chunk labels for the streaming UI only — the
|
||||
@@ -998,6 +1006,11 @@ async def dub_transcribe_stream(
|
||||
clones = done.pop().result()
|
||||
break
|
||||
yield _sse_event("ping", {})
|
||||
if clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(clones, _asr_backend),
|
||||
)
|
||||
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
|
||||
# own reference from the vocals so the dub of each line matches the
|
||||
# prosody of its source line. Short lines fall back to the
|
||||
@@ -1017,6 +1030,10 @@ async def dub_transcribe_stream(
|
||||
),
|
||||
)
|
||||
if seg_clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
seg_clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(seg_clones, _asr_backend),
|
||||
)
|
||||
job["segment_clones"] = seg_clones
|
||||
except Exception as e:
|
||||
logger.warning("per-segment clone refs skipped: %s", e)
|
||||
@@ -1129,6 +1146,9 @@ async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
|
||||
asr_audio_target = job.get("vocals_path")
|
||||
if not asr_audio_target or not os.path.exists(asr_audio_target):
|
||||
asr_audio_target = job.get("audio_path")
|
||||
# #963: same source-awareness as the SSE endpoint — vocals_path
|
||||
# falls back to the mixed audio_path when Demucs failed/skipped.
|
||||
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
|
||||
|
||||
import torch
|
||||
|
||||
@@ -1173,10 +1193,13 @@ async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
|
||||
segments = segment_transcript(result, duration=job.get("duration", 0.0), scene_cuts=scene_cuts)
|
||||
|
||||
# #280: snap segment starts forward to the actual speech onset so the
|
||||
# dub doesn't begin seconds before the original speaker does.
|
||||
# dub doesn't begin seconds before the original speaker does. #963:
|
||||
# only on the separated vocals track — on mixed audio every ambient
|
||||
# sound is a false onset candidate, so snapping is disabled.
|
||||
try:
|
||||
audio_for_onset, onset_sr = sf.read(asr_audio_target, dtype="float32")
|
||||
snap_segment_starts(segments, audio_for_onset, onset_sr)
|
||||
snap_segment_starts(segments, audio_for_onset, onset_sr,
|
||||
separated_vocals=asr_on_vocals)
|
||||
except Exception as e:
|
||||
logger.warning("onset alignment skipped: %s", e)
|
||||
|
||||
|
||||
@@ -170,8 +170,42 @@ async def dub_list_tracks(job_id: str):
|
||||
return {"tracks": job.get("dubbed_tracks", {})}
|
||||
|
||||
|
||||
def _segments_for_lang(job: dict, lang: "str | None") -> list:
|
||||
"""Job segments with `text` overlaid from ``job["segments_i18n"][lang]``.
|
||||
|
||||
P1.2 — ``job["segments"]`` is single-slot: it holds whichever language was
|
||||
generated LAST, so exporting subtitles for track A after generating track B
|
||||
emitted B's text under A's language label (the "N identical subtitle
|
||||
files" class). ``segments_i18n`` ({lang: {segKey: text}}, written by
|
||||
``dub_generate._sync_job_segments``) preserves each generated track's text;
|
||||
this overlays it non-destructively when present.
|
||||
|
||||
Back-compat: no lang requested, no ``segments_i18n`` on the job (predates
|
||||
the field), no entry for this lang, or no text for a given segment — each
|
||||
falls back to the segment as-is, i.e. exactly today's behaviour.
|
||||
Segment keys are the stable id (str) with the list index (str) as the
|
||||
legacy fallback, mirroring how the map is written.
|
||||
"""
|
||||
segments = job.get("segments", [])
|
||||
if not lang:
|
||||
return segments
|
||||
i18n = job.get("segments_i18n")
|
||||
lang_texts = i18n.get(lang) if isinstance(i18n, dict) else None
|
||||
if not isinstance(lang_texts, dict) or not lang_texts:
|
||||
return segments
|
||||
out = []
|
||||
for i, seg in enumerate(segments):
|
||||
key = str(seg.get("id")) if seg.get("id") is not None else str(i)
|
||||
txt = lang_texts.get(key)
|
||||
if txt is None:
|
||||
txt = lang_texts.get(str(i))
|
||||
out.append(dict(seg, text=txt) if isinstance(txt, str) and txt.strip() else seg)
|
||||
return out
|
||||
|
||||
|
||||
def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
|
||||
fitted_segments: "list[dict] | None" = None) -> str | None:
|
||||
fitted_segments: "list[dict] | None" = None,
|
||||
lang: "str | None" = None) -> str | None:
|
||||
"""Build a temp SRT from job segments for use with ffmpeg's subtitles filter.
|
||||
|
||||
Returned path is already ffmpeg-filter-safe (plain ASCII basename under exports_dir).
|
||||
@@ -181,8 +215,11 @@ def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
|
||||
fitted timeline — when provided, cue times come from there instead of
|
||||
the original ``job["segments"]`` timings, so burned subs track the
|
||||
retimed video / fitted audio rather than the source timeline.
|
||||
|
||||
``lang`` (P1.2): burn the named track's text (see ``_segments_for_lang``)
|
||||
instead of whatever language generated last.
|
||||
"""
|
||||
segments = job.get("segments", [])
|
||||
segments = _segments_for_lang(job, lang)
|
||||
if not segments:
|
||||
return None
|
||||
if fitted_segments:
|
||||
@@ -486,7 +523,9 @@ async def dub_download(
|
||||
# Smart Fit: cue times come from the fitted timeline — that's where the
|
||||
# dubbed audio actually sits, whether or not the video retime succeeds.
|
||||
fitted_segments = _fitted_segments_for(job, default_track) if default_track and default_track != "original" else None
|
||||
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments) if burn_subs else None
|
||||
# Burn the DEFAULT track's text (P1.2) — it's the audio the viewer hears.
|
||||
_burn_lang = default_track if default_track and default_track != "original" else None
|
||||
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments, lang=_burn_lang) if burn_subs else None
|
||||
|
||||
# ── Smart Fit video retime (two-tier) ─────────────────────────────────
|
||||
# Tier 1 (≤48 chunks): single filter_complex graph inlined into the mux
|
||||
@@ -1125,20 +1164,40 @@ async def dub_get_audio(job_id: str):
|
||||
raise HTTPException(status_code=404, detail="Audio file not found")
|
||||
return FileResponse(audio, media_type="audio/wav")
|
||||
|
||||
def _seg_wav_candidates(job: dict, lang: "str | None", seg_keys: tuple) -> list:
|
||||
"""Per-segment WAV name candidates, language-keyed first (P1.3).
|
||||
|
||||
Generation writes ``seg_{lang}_{id}.wav`` now; ``lang`` defaults to the
|
||||
job's last-generated track. Legacy un-keyed names (``seg_{id}.wav`` /
|
||||
``seg_{index}.wav``) stay as fallbacks so jobs rendered by previous
|
||||
builds keep serving their audio — these read-only endpoints keep the
|
||||
permissive fallback that matches their historic behaviour (the strict
|
||||
single-track gate lives on the generate splice path, where a wrong-
|
||||
language read would be baked into a track).
|
||||
"""
|
||||
lang = lang or job.get("language_code")
|
||||
keys = []
|
||||
if lang:
|
||||
keys.extend(f"{lang}_{k}" for k in seg_keys)
|
||||
keys.extend(seg_keys)
|
||||
return keys
|
||||
|
||||
|
||||
@router.get("/dub/preview/{job_id}/{segment_index}")
|
||||
async def dub_preview_segment(job_id: str, segment_index: int):
|
||||
async def dub_preview_segment(job_id: str, segment_index: int, lang: str = Query(None)):
|
||||
job = _get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
# Resolve the stable-id-named WAV via the render manifest; fall back to the
|
||||
# legacy index name for jobs rendered before id-based naming (#185). Each
|
||||
# candidate is realpath-normalised and containment-checked BEFORE any
|
||||
# filesystem access, so the guard dominates every path sink.
|
||||
# Resolve the stable-id-named WAV via the render manifest — language-keyed
|
||||
# name first (P1.3), then the legacy id/index names for jobs rendered
|
||||
# before per-language (and before id-based, #185) naming. Each candidate
|
||||
# is realpath-normalised and containment-checked BEFORE any filesystem
|
||||
# access, so the guard dominates every path sink.
|
||||
order = job.get("seg_order") or []
|
||||
seg_id = order[segment_index] if 0 <= segment_index < len(order) else segment_index
|
||||
base = os.path.realpath(DUB_DIR)
|
||||
seg_path = None
|
||||
for _sid in (seg_id, segment_index):
|
||||
for _sid in _seg_wav_candidates(job, lang, (seg_id, segment_index)):
|
||||
cand = os.path.realpath(dub_seg_path(job_id, _sid))
|
||||
if cand.startswith(base + os.sep) and os.path.exists(cand):
|
||||
seg_path = cand
|
||||
@@ -1342,13 +1401,16 @@ def _fitted_cue_times(job: dict, lang: str | None) -> list | None:
|
||||
async def dub_export_srt(
|
||||
job_id: str,
|
||||
dual: bool = False,
|
||||
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
|
||||
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
|
||||
):
|
||||
job = _get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
segments = job.get("segments", [])
|
||||
# P1.2 — text follows the REQUESTED track, not whichever language was
|
||||
# generated last (job["segments"] is single-slot). Legacy jobs without
|
||||
# segments_i18n fall back to today's behaviour.
|
||||
segments = _segments_for_lang(job, lang)
|
||||
if not segments:
|
||||
raise HTTPException(status_code=400, detail="No transcript segments available")
|
||||
|
||||
@@ -1391,13 +1453,14 @@ def _format_vtt_time(seconds):
|
||||
async def dub_export_vtt(
|
||||
job_id: str,
|
||||
dual: bool = False,
|
||||
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
|
||||
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
|
||||
):
|
||||
job = _get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
segments = job.get("segments", [])
|
||||
# Same per-track text resolution as /dub/srt (see comment there, P1.2).
|
||||
segments = _segments_for_lang(job, lang)
|
||||
if not segments:
|
||||
raise HTTPException(status_code=400, detail="No transcript segments available")
|
||||
|
||||
@@ -1427,7 +1490,7 @@ async def dub_export_vtt(
|
||||
|
||||
|
||||
@router.get("/dub/export-segments/{job_id}")
|
||||
async def dub_export_segments_zip(job_id: str):
|
||||
async def dub_export_segments_zip(job_id: str, lang: str = Query(None)):
|
||||
import zipfile
|
||||
job = _get_job(job_id)
|
||||
if not job:
|
||||
@@ -1445,7 +1508,7 @@ async def dub_export_segments_zip(job_id: str):
|
||||
seg_id = order[i] if i < len(order) else i
|
||||
# realpath + containment guard before any filesystem access.
|
||||
seg_path = None
|
||||
for _sid in (seg_id, i):
|
||||
for _sid in _seg_wav_candidates(job, lang, (seg_id, i)):
|
||||
cand = os.path.realpath(dub_seg_path(job_id, _sid))
|
||||
if cand.startswith(base + os.sep) and os.path.exists(cand):
|
||||
seg_path = cand
|
||||
|
||||
@@ -11,7 +11,8 @@ from core.db import db_conn
|
||||
from core.config import DUB_DIR, VOICES_DIR, dub_seg_path
|
||||
from core.tasks import task_manager
|
||||
from schemas.requests import DubRequest
|
||||
from services.model_manager import get_model, _gpu_pool, run_on_gpu_pool_guarded
|
||||
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
|
||||
from services.tts_backend import resolve_generation_backend
|
||||
from services.audio_dsp import apply_mastering, normalize_audio, apply_effects_chain, get_effect_chain
|
||||
from services.audio_io import atomic_save_wav, _safe_torchaudio_save
|
||||
from services.ffmpeg_utils import (
|
||||
@@ -87,6 +88,62 @@ def _sync_job_segments(job: dict, req: DubRequest) -> None:
|
||||
merged.append(row)
|
||||
job["segments"] = merged
|
||||
|
||||
# P1.2 — per-language text, additively. `job["segments"]` stays the flat
|
||||
# single-slot map every existing consumer reads (last generated language);
|
||||
# `job["segments_i18n"]` preserves EACH generated track's text so
|
||||
# /dub/srt|vtt?lang= can emit that language instead of N identical files.
|
||||
# Shape: { langCode: { segKey: text } } where segKey is the segment's
|
||||
# stable id (str) or, for id-less legacy segments, its list index (str).
|
||||
# The whole per-language map is rebuilt on every generate of that language
|
||||
# (the request always carries the full segment list), so deleted segments
|
||||
# never linger. Jobs predating this field simply lack it — every reader
|
||||
# falls back to `job["segments"]`.
|
||||
lang = (req.language_code or "und").strip() or "und"
|
||||
i18n = job.setdefault("segments_i18n", {})
|
||||
i18n[lang] = {
|
||||
(str(row["id"]) if row.get("id") is not None else str(i)): row["text"]
|
||||
for i, row in enumerate(merged)
|
||||
}
|
||||
|
||||
|
||||
def _seg_hashes_by_lang(job: dict) -> dict:
|
||||
"""Per-language segment fingerprints: { langCode: { segId: hash } }.
|
||||
|
||||
Additive migration (P1.3): jobs written by previous builds carry ONE flat
|
||||
`seg_hashes` map that was overwritten by whichever language generated
|
||||
last. That flat map can only describe the job's last-generated track, so
|
||||
it is attributed to `job["language_code"]` (which generate has always
|
||||
kept in lock-step with the last run). When even that is unknown the
|
||||
legacy hashes are dropped — segments then read as stale and regenerate
|
||||
cleanly, which is safer than guessing a language and splicing wrong-track
|
||||
audio. Note the legacy hashes also predate language-scoped fingerprints
|
||||
(see services.incremental.segment_fingerprint), so they compare stale
|
||||
once regardless — carrying them over just preserves the job shape.
|
||||
"""
|
||||
by_lang = job.get("seg_hashes_by_lang")
|
||||
if not isinstance(by_lang, dict):
|
||||
by_lang = {}
|
||||
legacy = job.get("seg_hashes")
|
||||
prev_lang = job.get("language_code")
|
||||
if isinstance(legacy, dict) and legacy and prev_lang:
|
||||
by_lang[prev_lang] = dict(legacy)
|
||||
job["seg_hashes_by_lang"] = by_lang
|
||||
return by_lang
|
||||
|
||||
|
||||
def _legacy_seg_cache_ok(job: dict, lang_code: str) -> bool:
|
||||
"""May this run reuse legacy un-keyed ``seg_<id>.wav`` files?
|
||||
|
||||
Only when no OTHER language's audio could be sitting in them: the job has
|
||||
no dubbed track in a different language. Single-language jobs rendered by
|
||||
previous builds therefore keep their whole on-disk cache; the moment a
|
||||
job carries a second language the un-keyed files are ambiguous (they hold
|
||||
whichever language wrote them last) and must never be spliced into a
|
||||
track again — the P1.3 cross-contamination class.
|
||||
"""
|
||||
tracks = job.get("dubbed_tracks") or {}
|
||||
return not any(lc != lang_code for lc in tracks)
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -100,13 +157,37 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
detail="This dub session has expired or was never created. Re-upload the video to start a new one.",
|
||||
)
|
||||
|
||||
_model = await get_model()
|
||||
# ── Engine resolution (issue #312 class) ────────────────────────────────
|
||||
# Dub used to hardcode OmniVoice via get_model() regardless of the engine
|
||||
# selected in Settings → Engines — a SILENT fallback. Every real dub
|
||||
# segment's ref_audio resolves to either an auto:<speaker>/auto-seg:<id>
|
||||
# clone cut from the source video or a saved voice-profile row (see
|
||||
# `_gen` below), so require_cloning=True: an engine that can't clone
|
||||
# would either mis-clone per segment or fail deep into the job. Checked
|
||||
# ONCE here, before the streaming task starts, so a doomed job fails fast
|
||||
# with one clear message instead of N per-segment ones.
|
||||
try:
|
||||
backend = await resolve_generation_backend(require_cloning=True)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
async def _stream(task_id):
|
||||
total = len(req.segments)
|
||||
all_segment_wavs = []
|
||||
sync_scores = []
|
||||
|
||||
# Track language for this run. Everything per-track — the per-segment
|
||||
# WAV cache, fingerprints, seg_wav_kind — is keyed by it (P1.3) so a
|
||||
# multi-language job's tracks can't cross-contaminate.
|
||||
lang_code = req.language_code or "und"
|
||||
|
||||
def _seg_lang_path(seg_key) -> str:
|
||||
# Per-language per-segment WAV: seg_{lang}_{id}.wav. Built through
|
||||
# dub_seg_path so the sanitisation + DUB_DIR containment guard
|
||||
# apply to the combined key. Legacy un-keyed seg_{id}.wav files
|
||||
# remain readable via the gated fallback (_legacy_seg_cache_ok).
|
||||
return dub_seg_path(job_id, f"{lang_code}_{seg_key}")
|
||||
|
||||
# Throttle the device cache flush. empty_cache() is a synchronous
|
||||
# device stall, so calling it every segment (as the old code did)
|
||||
# serialised the GPU loop; the batched-I/O design it replaced kept
|
||||
@@ -236,7 +317,16 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# double-compress. Force one full regen; afterwards seg_wav_kind is
|
||||
# "natural" and partial regen / fit-only re-mix (regen_only=[]) work.
|
||||
# Jobs predating this field have unknown kind → also regen once.
|
||||
if strategy == "smart_fit" and regen_only is not None and job.get("seg_wav_kind") != "natural":
|
||||
# P1.3: the kind is per-track now (each language renders under its own
|
||||
# strategy); the flat job["seg_wav_kind"] is only consulted for jobs
|
||||
# written before the per-language map existed — once the map is
|
||||
# present, a language without an entry has unknown-kind WAVs (or none
|
||||
# at all) and must regen once, exactly like the pre-field case.
|
||||
_kind_map = job.get("seg_wav_kind_by_lang")
|
||||
_wav_kind = (
|
||||
_kind_map.get(lang_code) if isinstance(_kind_map, dict) else job.get("seg_wav_kind")
|
||||
)
|
||||
if strategy == "smart_fit" and regen_only is not None and _wav_kind != "natural":
|
||||
regen_only = None
|
||||
# Manifest: stable segment id per current index. Per-segment WAVs are
|
||||
# named by stable id (dub_seg_path) so regen reuses the right audio after
|
||||
@@ -266,7 +356,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
|
||||
seg_duration = seg.end - seg.start
|
||||
if seg_duration <= 0.05 or not seg.text.strip():
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
# max(0, …): a zero/negative-duration slot must not feed a
|
||||
# negative length to torch.zeros (raises) — _store_mix_wav
|
||||
# turns the empty buffer into a harmless in-memory entry.
|
||||
@@ -283,31 +373,38 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# Partial regen: if this segment isn't in the allow-list, reuse its
|
||||
# previously-rendered WAV so the final mix still covers the timeline.
|
||||
if regen_only is not None and seg_id not in regen_only:
|
||||
seg_wav_path = dub_seg_path(job_id, seg_id)
|
||||
if not os.path.exists(seg_wav_path):
|
||||
# Back-compat: jobs rendered before id-named files used seg_{index}.wav.
|
||||
_legacy = dub_seg_path(job_id, i)
|
||||
if os.path.exists(_legacy):
|
||||
seg_wav_path = _legacy
|
||||
# This track's own cache first (seg_{lang}_{id}.wav). Legacy
|
||||
# un-keyed files (seg_{id}.wav / seg_{index}.wav) are reused
|
||||
# ONLY when no other-language track exists on the job — a
|
||||
# multi-track job's un-keyed files hold whichever language
|
||||
# rendered last, and splicing them here was exactly how
|
||||
# "Regen N changed" mixed language B into track A (P1.3).
|
||||
seg_wav_path = _seg_lang_path(seg_id)
|
||||
if not os.path.exists(seg_wav_path) and _legacy_seg_cache_ok(job, lang_code):
|
||||
for _legacy_key in (seg_id, i):
|
||||
_legacy = dub_seg_path(job_id, _legacy_key)
|
||||
if os.path.exists(_legacy):
|
||||
seg_wav_path = _legacy
|
||||
break
|
||||
if os.path.exists(seg_wav_path):
|
||||
try:
|
||||
_t_cache_0 = time.perf_counter()
|
||||
cached_wav, cached_sr = torchaudio.load(seg_wav_path)
|
||||
if cached_sr != _model.sampling_rate:
|
||||
if cached_sr != backend.sample_rate:
|
||||
import torchaudio.functional as AF
|
||||
cached_wav = AF.resample(cached_wav, cached_sr, _model.sampling_rate)
|
||||
cached_wav = AF.resample(cached_wav, cached_sr, backend.sample_rate)
|
||||
# Pad/trim to slot — except smart_fit, whose mix
|
||||
# loop needs the natural-rate length to compute the
|
||||
# audio/video split (the seg_wav_kind guard above
|
||||
# guarantees these cached WAVs are natural-rate).
|
||||
if strategy != "smart_fit":
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = cached_wav.shape[-1]
|
||||
if target_samples > current_samples:
|
||||
cached_wav = torch.nn.functional.pad(cached_wav, (0, target_samples - current_samples))
|
||||
elif current_samples > target_samples:
|
||||
cached_wav = cached_wav[..., :target_samples]
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, _model.sampling_rate, f"mix_{seg_id}"))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, backend.sample_rate, f"mix_{seg_id}"))
|
||||
try:
|
||||
del cached_wav
|
||||
except Exception:
|
||||
@@ -320,7 +417,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# Fall through to a silent placeholder if the cached WAV
|
||||
# is broken — cleaner than aborting the whole mix.
|
||||
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'cached seg lost, padding silence: {str(e)[:120]}'})}\n\n"
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
silence = torch.zeros(1, max(0, int(seg_duration * sr)))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, silence, sr, f"mix_{seg_id}"))
|
||||
try:
|
||||
@@ -404,25 +501,23 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
torch.manual_seed(used_seed)
|
||||
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang if lang != "Auto" else None,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
instruct=instruct_str if instruct_str else None,
|
||||
duration=dur_s, num_step=nstep, guidance_scale=cfg,
|
||||
speed=spd, denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
|
||||
sr = backend.sample_rate
|
||||
|
||||
# Apply per-segment DSP effect preset (default: broadcast)
|
||||
seg_effect_preset = effect_preset or "broadcast"
|
||||
if seg_effect_preset == "raw":
|
||||
return audio_out
|
||||
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
mastered_audio = audio_out
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
effect_chain = get_effect_chain(seg_effect_preset)
|
||||
if effect_chain:
|
||||
mastered_audio = apply_effects_chain(
|
||||
@@ -453,24 +548,22 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
nstep, retry_steps,
|
||||
)
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang if lang != "Auto" else None,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
instruct=instruct_str if instruct_str else None,
|
||||
duration=dur_s, num_step=retry_steps, guidance_scale=cfg,
|
||||
speed=spd, denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
|
||||
sr = backend.sample_rate
|
||||
|
||||
seg_effect_preset = effect_preset or "broadcast"
|
||||
if seg_effect_preset == "raw":
|
||||
return audio_out
|
||||
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
mastered_audio = audio_out
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
effect_chain = get_effect_chain(seg_effect_preset)
|
||||
if effect_chain:
|
||||
mastered_audio = apply_effects_chain(
|
||||
@@ -557,7 +650,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
yield f"data: {json.dumps({'type': 'cancelled', 'segments_processed': i + 1})}\n\n"
|
||||
return
|
||||
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = audio_tensor.shape[-1]
|
||||
|
||||
if strategy == "strict_slot":
|
||||
@@ -575,7 +668,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# trim, slip, stretch the video, or split audio/video
|
||||
# retiming (smart_fit) to accommodate it.
|
||||
|
||||
generated_dur = audio_tensor.shape[-1] / _model.sampling_rate
|
||||
generated_dur = audio_tensor.shape[-1] / backend.sample_rate
|
||||
sync_ratio = round(generated_dur / max(seg_duration, 0.01), 3)
|
||||
|
||||
sync_scores.append(sync_ratio)
|
||||
@@ -584,6 +677,9 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# and job flush to the batch-write phase after the GPU loop.
|
||||
_seg_fp = None
|
||||
try:
|
||||
# track_lang scopes the hash to THIS track (P1.3); the
|
||||
# client-side recompute (/tools/incremental) sends the
|
||||
# same code, so parity (#281 class) holds per language.
|
||||
_seg_fp = segment_fingerprint({
|
||||
"text": seg.text,
|
||||
"target_lang": getattr(seg, "target_lang", None),
|
||||
@@ -592,24 +688,24 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
"speed": getattr(seg, "speed", None),
|
||||
"direction": getattr(seg, "direction", None),
|
||||
"effect_preset": getattr(seg, "effect_preset", None),
|
||||
})
|
||||
}, track_lang=lang_code)
|
||||
except Exception as e:
|
||||
logger.debug("seg fingerprint skipped for %s: %s", seg_id, e)
|
||||
|
||||
_pending_seg_writes.append((i, _model.sampling_rate, seg_id, _seg_fp, _num_step))
|
||||
_pending_seg_writes.append((i, backend.sample_rate, seg_id, _seg_fp, _num_step))
|
||||
|
||||
# RVC needs the WAV on disk, so write it immediately only
|
||||
# when RVC is active (uncommon path).
|
||||
if rvc_is_enabled():
|
||||
seg_wav_path = dub_seg_path(job_id, seg_id)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
|
||||
seg_wav_path = _seg_lang_path(seg_id)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
|
||||
try:
|
||||
await loop.run_in_executor(_gpu_pool, apply_rvc, seg_wav_path)
|
||||
rvc_wav, rvc_sr = torchaudio.load(seg_wav_path)
|
||||
if rvc_sr == _model.sampling_rate:
|
||||
if rvc_sr == backend.sample_rate:
|
||||
audio_tensor = rvc_wav
|
||||
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = audio_tensor.shape[-1]
|
||||
if target_samples > current_samples:
|
||||
audio_tensor = torch.nn.functional.pad(audio_tensor, (0, target_samples - current_samples))
|
||||
@@ -619,31 +715,32 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'RVC skipped: {str(e)[:120]}'})}\n\n"
|
||||
|
||||
# Watermark this FRESH TTS output exactly once, right before it
|
||||
# is persisted. The same seg_<id>.wav is BOTH the downloadable
|
||||
# per-segment file AND the assembly input for the final track,
|
||||
# so marking it here (and nowhere else) gives the downloadable
|
||||
# WAV its mark back and the final mix inherits it — no double-
|
||||
# mark. Cached-reuse audio is already marked; silence/zero slots
|
||||
# carry no speech to mark, so neither is re-watermarked.
|
||||
audio_tensor = embed_watermark(audio_tensor, _model.sampling_rate)
|
||||
# is persisted. The same seg_{lang}_{id}.wav is BOTH the
|
||||
# downloadable per-segment file AND the assembly input for the
|
||||
# final track, so marking it here (and nowhere else) gives the
|
||||
# downloadable WAV its mark back and the final mix inherits it —
|
||||
# no double-mark. Cached-reuse audio is already marked;
|
||||
# silence/zero slots carry no speech to mark, so neither is
|
||||
# re-watermarked.
|
||||
audio_tensor = embed_watermark(audio_tensor, backend.sample_rate)
|
||||
|
||||
seg_wav_path = dub_seg_path(job_id, seg_id)
|
||||
seg_wav_path = _seg_lang_path(seg_id)
|
||||
try:
|
||||
# Keep the existing per-segment WAV contract for previews
|
||||
# and partial regeneration, but do not keep the tensor in RAM.
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
|
||||
except Exception as e:
|
||||
logger.warning("seg write failed for %s: %s", seg_id, e)
|
||||
# If the durable segment write fails, still preserve a mix
|
||||
# copy so this generation can finish.
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, _model.sampling_rate, f"mix_{seg_id}"))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, backend.sample_rate, f"mix_{seg_id}"))
|
||||
try:
|
||||
del audio_tensor
|
||||
except Exception:
|
||||
pass
|
||||
_release_audio_tensors()
|
||||
else:
|
||||
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, _model.sampling_rate))
|
||||
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, backend.sample_rate))
|
||||
try:
|
||||
del audio_tensor
|
||||
except Exception:
|
||||
@@ -651,7 +748,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
_release_audio_tensors()
|
||||
except Exception as e:
|
||||
yield f"data: {json.dumps({'type': 'error', 'segment': i, 'error': str(e)})}\n\n"
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, torch.zeros(1, max(0, int(seg_duration * sr))), sr, f"mix_{seg_id}"))
|
||||
sync_scores.append(1.0)
|
||||
|
||||
@@ -663,17 +760,24 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# Per-segment WAVs were written during the loop to keep RAM bounded.
|
||||
# Flush only lightweight fingerprints/quality metadata here.
|
||||
_t_diskw_0 = time.perf_counter()
|
||||
hashes = job.setdefault("seg_hashes", {})
|
||||
# P1.3 — fingerprints live per language so each track's staleness is
|
||||
# judged against ITS OWN last generate. The flat job["seg_hashes"] is
|
||||
# kept as a mirror of the CURRENT track's map: every existing consumer
|
||||
# (the `done` event, dub-history restore, older frontends) already
|
||||
# treats it as "the hashes of the language generated last", which is
|
||||
# exactly what it now provably contains.
|
||||
hashes = _seg_hashes_by_lang(job).setdefault(lang_code, {})
|
||||
quality_map = job.setdefault("seg_num_step", {})
|
||||
for (_si, _sr, _sid, _fp, _nstep) in _pending_seg_writes:
|
||||
if _fp is not None:
|
||||
hashes[_sid] = _fp
|
||||
quality_map[_sid] = _nstep
|
||||
job["seg_hashes"] = dict(hashes)
|
||||
# Single job flush instead of one per 8 segments.
|
||||
_save_job(job_id, job)
|
||||
_t_diskw = time.perf_counter() - _t_diskw_0
|
||||
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
slot_fit = (req.slot_fit or "time_stretch").lower()
|
||||
overflow_budget_s = max(0.0, float(req.overflow_budget_s or 0.0))
|
||||
|
||||
@@ -759,7 +863,6 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# not from the plan — so subtitles land exactly on the audio.
|
||||
fitted_cues: list[dict] = []
|
||||
|
||||
lang_code = req.language_code or "und"
|
||||
track_path = os.path.join(DUB_DIR, job_id, f"dubbed_{lang_code}.wav")
|
||||
os.makedirs(os.path.dirname(track_path), exist_ok=True)
|
||||
|
||||
@@ -1049,8 +1152,12 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
job["dubbed_tracks"][lang_code]["fit_fp"] = fit_fp
|
||||
# Record what kind of per-segment WAVs are on disk so a later
|
||||
# smart_fit run knows whether partial regen / fit-only re-mix can
|
||||
# reuse them ("natural") or must regen once ("slotted").
|
||||
job["seg_wav_kind"] = "slotted" if strategy == "strict_slot" else "natural"
|
||||
# reuse them ("natural") or must regen once ("slotted"). Per-track
|
||||
# (P1.3) — each language renders under its own strategy; the flat
|
||||
# field stays in lock-step for older readers.
|
||||
_kind = "slotted" if strategy == "strict_slot" else "natural"
|
||||
job.setdefault("seg_wav_kind_by_lang", {})[lang_code] = _kind
|
||||
job["seg_wav_kind"] = _kind
|
||||
_save_job(job_id, job)
|
||||
|
||||
_t_total = time.perf_counter() - _t_start
|
||||
@@ -1098,7 +1205,13 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
_model = await get_model()
|
||||
# See the /dub/generate/{job_id} resolution above (issue #312 class) —
|
||||
# a segment preview resolves ref_audio from the same auto-clone /
|
||||
# voice-profile sources, so it needs the same cloning-capable gate.
|
||||
try:
|
||||
backend = await resolve_generation_backend(require_cloning=True)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
def _gen():
|
||||
ref_audio = None
|
||||
@@ -1133,7 +1246,7 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
instruct_str = row["instruct"]
|
||||
|
||||
lang = req.language if req.language != "Auto" else None
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=req.text,
|
||||
language=lang,
|
||||
ref_audio=ref_audio,
|
||||
@@ -1146,21 +1259,15 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
denoise=True,
|
||||
postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered = apply_mastering(
|
||||
audio_out,
|
||||
sample_rate=getattr(_model, "sampling_rate", 24000),
|
||||
)
|
||||
return normalize_audio(mastered, target_dBFS=-2.0)
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
audio_out = apply_mastering(audio_out, sample_rate=backend.sample_rate)
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
|
||||
# Bounded + pool-reset on hang so a wedged preview generate can't starve the
|
||||
# GPU pool and brick the backend (#730 class).
|
||||
audio_tensor = await run_on_gpu_pool_guarded(_gen, what="Dub preview generate")
|
||||
|
||||
sr = getattr(_model, "sampling_rate", 24000)
|
||||
sr = backend.sample_rate
|
||||
buf = io.BytesIO()
|
||||
_safe_torchaudio_save(buf, audio_tensor, sr, format="wav")
|
||||
buf.seek(0)
|
||||
|
||||
@@ -16,6 +16,7 @@ Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`,
|
||||
a backend without Settings silently undoing it.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from time import perf_counter
|
||||
|
||||
@@ -429,6 +430,11 @@ def engine_selftest(engine_id: str):
|
||||
class SelectEngineRequest(BaseModel):
|
||||
family: str # "tts" | "asr" | "llm"
|
||||
backend_id: str
|
||||
# Only meaningful for family="tts", backend_id="mlx-audio" (#981) — picks
|
||||
# which of mlx-audio's curated models is actually loaded. A curated key
|
||||
# ("kokoro") or a raw HF repo id ("mlx-community/Kokoro-82M-bf16") — the
|
||||
# same tolerance MLXAudioBackend.__init__ already has. Ignored otherwise.
|
||||
model_id: str | None = None
|
||||
|
||||
|
||||
class SelectEngineResponse(BaseModel):
|
||||
@@ -471,6 +477,24 @@ def select_engine(req: SelectEngineRequest):
|
||||
f"Backend {req.backend_id} can't run on this machine: {why}. "
|
||||
f"Pick an engine with a CPU path, or one that supports this host's GPU.",
|
||||
)
|
||||
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
|
||||
# persist the model pick alongside the backend id so the UI can actually
|
||||
# select which curated model gets loaded (previously it always defaulted
|
||||
# to Kokoro no matter what the user downloaded in Settings → Models).
|
||||
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
|
||||
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
|
||||
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
|
||||
# tolerance MLXAudioBackend.__init__ already has for power users.
|
||||
# Anything else (typo'd key, malformed id) is rejected outright
|
||||
# rather than silently persisted as a "custom repo" that then fails
|
||||
# to resolve at load time.
|
||||
if req.model_id not in known_keys and not re.fullmatch(r"[\w.-]+/[\w.-]+", req.model_id):
|
||||
raise HTTPException(
|
||||
400,
|
||||
f"Unknown mlx-audio model: {req.model_id!r}. Expected one of "
|
||||
f"{sorted(known_keys)} or a HF repo id like 'owner/name'.",
|
||||
)
|
||||
prefs.set_("mlx_audio_model_id", req.model_id)
|
||||
prefs.set_(pref_key, req.backend_id)
|
||||
return {
|
||||
"family": req.family,
|
||||
|
||||
@@ -105,8 +105,8 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
|
||||
|
||||
``skip_mastering`` honors a backend's ``applies_own_mastering`` flag
|
||||
(issue #312): studio engines (e.g. VoxCPM2's native 48 kHz output)
|
||||
opt out of the broadcast Compressor + Reverb chain that's tuned for
|
||||
OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
opt out of the broadcast highpass + Compressor pre-stage that's tuned
|
||||
for OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
it's a benign peak scale. Mirrors ``_run_tts`` in openai_compat.py.
|
||||
"""
|
||||
from services.audio_dsp import (
|
||||
@@ -143,6 +143,28 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
|
||||
|
||||
def _safe_exc_text(e: BaseException) -> str:
|
||||
"""``f"{type(e).__name__}: {e}"`` — the house style used for
|
||||
unrecognized-error formatting throughout the backend (grep
|
||||
``type(e).__name__`` in settings.py / asr_backend.py / model_manager.py
|
||||
/ engines.py) — with a guard against leaking a raw container repr.
|
||||
|
||||
#977: an AssertionError raised deep inside a vendored dependency
|
||||
(mlx-audio's Kokoro pipeline) had ``.args`` shaped like
|
||||
``('du', {'a': 'American English', ...})`` — a tuple containing a dict.
|
||||
``str(e)`` on that renders the WHOLE table straight into the user-facing
|
||||
message. Any engine's ``generate()`` can raise something shaped like
|
||||
this (not just Kokoro), so guard generically: if any element of
|
||||
``e.args`` is a container rather than a plain string, don't interpolate
|
||||
``str(e)`` at all — name the exception type and point at the log
|
||||
instead.
|
||||
"""
|
||||
args = getattr(e, "args", ())
|
||||
if any(isinstance(a, (dict, list, tuple, set, frozenset)) for a in args):
|
||||
return f"{type(e).__name__} — see Settings → Logs → Backend for details"
|
||||
return f"{type(e).__name__}: {e}"
|
||||
|
||||
|
||||
def _exception_chain(e):
|
||||
"""Yield ``e`` plus every ``__cause__``/``__context__`` beneath it
|
||||
(cycle-safe). Engines and hub libraries routinely wrap the original
|
||||
@@ -412,7 +434,7 @@ def _oom_friendly_reraise(e):
|
||||
raise RuntimeError(
|
||||
f"TTS engine stopped mid-generation with an error OmniVoice doesn't "
|
||||
f"recognize. Retry once; if it keeps failing, please report it with "
|
||||
f"the full trace. Underlying error: {e}"
|
||||
f"the full trace. Underlying error: {_safe_exc_text(e)}"
|
||||
) from e
|
||||
|
||||
|
||||
@@ -934,7 +956,7 @@ async def generate_speech(
|
||||
status_code=500,
|
||||
detail=(
|
||||
f"Couldn't synthesize audio. See Settings → Logs → Backend for the full trace. "
|
||||
f"Underlying error: {e}"
|
||||
f"Underlying error: {_safe_exc_text(e)}"
|
||||
),
|
||||
)
|
||||
finally:
|
||||
|
||||
@@ -237,10 +237,10 @@ def _run_tts(backend, text: str, kw: dict):
|
||||
sr = backend.sample_rate
|
||||
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
|
||||
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
|
||||
# That chain's Compressor + 8% Reverb is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump and a
|
||||
# reverb tail that degrade the very output we want clean. Loudness
|
||||
# normalisation still runs — it's a benign peak scale, not dynamics.
|
||||
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump that
|
||||
# degrades the very output we want clean. Loudness normalisation still
|
||||
# runs — it's a benign peak scale, not dynamics.
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
wav = apply_mastering(wav, sample_rate=sr)
|
||||
wav = normalize_audio(wav, target_dBFS=-2.0)
|
||||
|
||||
@@ -73,6 +73,17 @@ async def create_profile(
|
||||
raise ValueError("not an object")
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=422, detail="vd_states must be a JSON object")
|
||||
# Root-cause close for #983: a design profile must never be PERSISTED
|
||||
# with a partial vd_states shape, regardless of which client (older
|
||||
# frontend build, hand-edited payload, third-party API caller) created
|
||||
# it — a missing category key crashes DesignMethodPanel's render on
|
||||
# every future client that selects this profile. CATEGORY_ORDER is the
|
||||
# same single source of truth the frontend's CATEGORIES keys mirror
|
||||
# (core/describe_voice.py), so this can't drift from the picker.
|
||||
from core.describe_voice import CATEGORY_ORDER
|
||||
for _cat in CATEGORY_ORDER:
|
||||
parsed.setdefault(_cat, "Auto")
|
||||
vd_states = _json.dumps(parsed)
|
||||
# An all-Auto design (every category left on "Auto") yields an empty
|
||||
# instruct — that's still a valid, saveable voice: synthesis falls back
|
||||
# to neutral instruct-only conditioning (see generation.py design path).
|
||||
|
||||
@@ -282,7 +282,8 @@ def save_llm_provider(provider_id: str, body: _LLMProviderBody):
|
||||
A None field is left unchanged; an empty api_key clears the stored key.
|
||||
"""
|
||||
from services import llm_providers
|
||||
if llm_providers.get_provider(provider_id) is None:
|
||||
p = llm_providers.get_provider(provider_id)
|
||||
if p is None:
|
||||
raise HTTPException(status_code=404, detail=f"unknown provider {provider_id!r}")
|
||||
if body.api_key is not None:
|
||||
llm_providers.save_key(provider_id, body.api_key.strip())
|
||||
@@ -290,7 +291,17 @@ def save_llm_provider(provider_id: str, body: _LLMProviderBody):
|
||||
provider_id, base_url=body.base_url, model=body.model,
|
||||
account_id=body.account_id,
|
||||
)
|
||||
if body.make_active:
|
||||
# An explicit save also claims the active slot when the user has never
|
||||
# chosen a provider (#963). Without this, a saved-and-tested local
|
||||
# provider (Ollama/LM Studio) evaporates on restart: active_provider_id()
|
||||
# deliberately excludes local providers from auto-select, so the plain
|
||||
# "Save" left nothing persisted to resolve. Gated on the STORED selection
|
||||
# only — an explicit prior choice is never stolen by a plain save, and an
|
||||
# unconfigured provider can't claim the slot.
|
||||
if body.make_active or (
|
||||
llm_providers.stored_active_provider_id() is None
|
||||
and llm_providers.is_configured(p)
|
||||
):
|
||||
llm_providers.set_active_provider(provider_id)
|
||||
return list_llm_providers()
|
||||
|
||||
@@ -739,6 +750,51 @@ def set_hf_mirror(body: _HFMirrorBody):
|
||||
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
|
||||
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
|
||||
# without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
# base_url/model are plain settings_store text rows; the key is encrypted via
|
||||
# settings_store.set_secret — same convention as /llm-providers, never
|
||||
# returned to the client, '' clears it, omitted/None leaves it unchanged.
|
||||
|
||||
|
||||
class _ASROpenAICompatBody(BaseModel):
|
||||
base_url: str | None = None
|
||||
model: str | None = None
|
||||
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
|
||||
|
||||
|
||||
@router.get("/asr-openai-compat")
|
||||
def get_asr_openai_compat():
|
||||
from services import asr_backend
|
||||
|
||||
return {
|
||||
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
|
||||
"model": asr_backend.resolve_openai_compat_asr_model(),
|
||||
"has_key": asr_backend.openai_compat_asr_has_key(),
|
||||
}
|
||||
|
||||
|
||||
@router.put("/asr-openai-compat")
|
||||
def set_asr_openai_compat(body: _ASROpenAICompatBody):
|
||||
from services import asr_backend, settings_store
|
||||
|
||||
if body.base_url is not None:
|
||||
url = body.base_url.strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
|
||||
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
|
||||
if body.model is not None:
|
||||
settings_store.set_text(
|
||||
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
|
||||
)
|
||||
if body.api_key is not None:
|
||||
settings_store.set_secret(
|
||||
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
|
||||
)
|
||||
return get_asr_openai_compat()
|
||||
|
||||
|
||||
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
|
||||
# (feat/safe-updates). Both are read-only, local-first surfaces for
|
||||
# Settings → Updates: the "What's new" viewer reads the CHANGELOG.md that
|
||||
|
||||
@@ -530,14 +530,16 @@ async def install_model(req: InstallModelRequest):
|
||||
_install_cooldowns[req.repo_id] = _time_fail.time()
|
||||
# #874: when the install failed because the configured HF mirror is
|
||||
# unreachable, name the mirror + the setting instead of leaking the
|
||||
# raw connectivity error. No-op for every other failure.
|
||||
from core.failure import append_hf_mirror_hint
|
||||
# raw connectivity error. #959: likewise for the SOCKS-proxy class
|
||||
# (missing socksio fails the download's session construction).
|
||||
# No-op for every other failure.
|
||||
from core.failure import append_hint
|
||||
hf_progress.emit({
|
||||
"repo_id": req.repo_id,
|
||||
"filename": req.repo_id,
|
||||
"downloaded": 0, "total": 0, "pct": 0.0,
|
||||
"phase": "install_error",
|
||||
"error": append_hf_mirror_hint(str(e)),
|
||||
"error": append_hint(str(e)),
|
||||
})
|
||||
finally:
|
||||
_cancelled.discard(req.repo_id)
|
||||
|
||||
@@ -168,16 +168,40 @@ def _detect_gpu() -> dict:
|
||||
return info
|
||||
|
||||
|
||||
def _probe_network(host: str = "huggingface.co", timeout: float = 2.0) -> bool:
|
||||
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 2.0) -> bool:
|
||||
"""Tiny TCP connect test."""
|
||||
import socket
|
||||
try:
|
||||
with socket.create_connection((host, 443), timeout=timeout):
|
||||
with socket.create_connection((host, port), timeout=timeout):
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _hf_endpoint_host() -> tuple[str, int]:
|
||||
"""Host/port of the Hugging Face endpoint actually in effect.
|
||||
|
||||
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
|
||||
point HF_ENDPOINT at a mirror via Settings → Models → Hugging Face
|
||||
mirror. Probing hardcoded huggingface.co would fail them even when their
|
||||
configured mirror works fine.
|
||||
"""
|
||||
try:
|
||||
from core.failure import configured_hf_mirror
|
||||
mirror = configured_hf_mirror()
|
||||
except Exception:
|
||||
mirror = ""
|
||||
if mirror:
|
||||
try:
|
||||
from urllib.parse import urlsplit
|
||||
u = urlsplit(mirror)
|
||||
if u.hostname:
|
||||
return u.hostname, u.port or (80 if u.scheme == "http" else 443)
|
||||
except Exception:
|
||||
pass
|
||||
return "huggingface.co", 443
|
||||
|
||||
|
||||
def _ram_gb() -> float:
|
||||
try:
|
||||
import psutil
|
||||
@@ -400,15 +424,49 @@ def preflight():
|
||||
"status": r_status, "detail": r_detail, "fix": r_fix,
|
||||
})
|
||||
|
||||
# ── Network
|
||||
net_ok = _probe_network()
|
||||
# ── Network — probes the HF endpoint actually in effect (mirror-aware),
|
||||
# and a dead network is a WARNING, not a blocker. The app is local-first:
|
||||
# already-downloaded models work offline, and a hard fail here dead-ends
|
||||
# restricted-network users (e.g. China, where huggingface.co is blocked)
|
||||
# on the very first screen — before they can reach the mirror setting
|
||||
# that fixes it. Model downloads surface their own actionable errors.
|
||||
net_host, net_port = _hf_endpoint_host()
|
||||
net_ok = _probe_network(net_host, net_port)
|
||||
mirror_reachable = False
|
||||
if not net_ok and net_host == "huggingface.co":
|
||||
# Official endpoint blocked — if the community mirror is reachable,
|
||||
# tell the user exactly which switch unblocks them.
|
||||
mirror_reachable = _probe_network("hf-mirror.com")
|
||||
if net_ok:
|
||||
net_fix = None
|
||||
elif mirror_reachable:
|
||||
net_fix = (
|
||||
"huggingface.co is blocked on this network, but the hf-mirror.com "
|
||||
"community mirror is reachable — apply it below and re-check. "
|
||||
"Model downloads will use the mirror immediately."
|
||||
)
|
||||
elif net_host != "huggingface.co":
|
||||
net_fix = (
|
||||
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
|
||||
"— it may be down or blocked. Pick another mirror or the official "
|
||||
"endpoint below, or continue offline: models already downloaded "
|
||||
"keep working."
|
||||
)
|
||||
else:
|
||||
net_fix = (
|
||||
"Check internet connection, VPN, or corporate firewall whitelist "
|
||||
"for huggingface.co. You can continue — models already downloaded "
|
||||
"keep working offline; new downloads need a connection or a "
|
||||
"mirror (configurable below)."
|
||||
)
|
||||
checks.append({
|
||||
"id": "network", "label": "Network (huggingface.co)",
|
||||
"status": "pass" if net_ok else "fail",
|
||||
"detail": "Reachable" if net_ok else "Unreachable on port 443",
|
||||
"fix": None if net_ok else
|
||||
"Check internet connection, VPN, or corporate firewall "
|
||||
"whitelist for huggingface.co.",
|
||||
"id": "network", "label": f"Network ({net_host})",
|
||||
"status": "pass" if net_ok else "warn",
|
||||
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
|
||||
"fix": net_fix,
|
||||
# Frontend affordance hint: the wizard offers the mirror quick-pick
|
||||
# when the endpoint is unreachable (PreflightCheck allows extras).
|
||||
"mirror_reachable": mirror_reachable,
|
||||
})
|
||||
|
||||
# Aggregate
|
||||
|
||||
@@ -77,6 +77,10 @@ async def probe(req: ProbeReq):
|
||||
class IncrementalReq(BaseModel):
|
||||
segments: list[dict]
|
||||
stored_hashes: Optional[dict[str, str]] = None
|
||||
# P1.3 — the ACTIVE track's language code. When set, fingerprints are
|
||||
# scoped to that language (pass that language's stored hashes alongside);
|
||||
# omitted → legacy language-agnostic hashing, kept for old callers.
|
||||
lang: Optional[str] = None
|
||||
|
||||
|
||||
@router.post("/tools/incremental")
|
||||
@@ -84,6 +88,7 @@ def plan_incremental(req: IncrementalReq):
|
||||
return incremental.plan_incremental(
|
||||
req.segments,
|
||||
stored_hashes=req.stored_hashes or {},
|
||||
track_lang=req.lang,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -41,6 +41,8 @@ _HINTS: dict[str, str] = {
|
||||
"COMPUTE_TYPE_UNSUPPORTED": "Your GPU doesn't support float16 — OmniVoice retried on int8. If transcription still fails, set OMNIVOICE/ASR_COMPUTE_TYPE=int8 or use CPU.",
|
||||
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
|
||||
"OS_INVALID_ARGUMENT": "The OS rejected a file operation (Errno 22 / invalid argument) — in the transcribe path this is the temporary WAV write before ASR. It's almost always the temp directory: missing, read-only, on a full or removed drive, or blocked by antivirus. Check that your system TEMP/TMP folder exists and is writable and the drive has free space (add an OmniVoice antivirus exclusion if you use one), then retry.",
|
||||
"SOCKS_PROXY_SUPPORT_MISSING": "A SOCKS proxy is configured in your environment (ALL_PROXY/HTTPS_PROXY=socks5://…) and the backend's HTTP client is missing SOCKS support. Newer OmniVoice builds ship SOCKS support (the socksio package) — update the app. If you still see this, unset ALL_PROXY/HTTPS_PROXY for OmniVoice, or run `uv pip install 'httpx[socks]'` in the backend venv, then restart.",
|
||||
"SSL_HANDSHAKE_FAILURE": "A corporate or antivirus proxy is intercepting HTTPS traffic and re-signing certificates with its own CA — your OS trusts that CA, but Python's bundled certifi CA list doesn't, so the TLS handshake fails even though the connection reached the server. Newer OmniVoice builds trust the OS certificate store at startup (the truststore package), which should already fix this — update the app and retry. If you still see this, add an HTTPS-scanning exclusion for OmniVoice/Python in your antivirus, or ask IT for the proxy's CA bundle and set SSL_CERT_FILE to it, then restart.",
|
||||
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
|
||||
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
|
||||
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
|
||||
@@ -168,6 +170,34 @@ def append_hf_mirror_hint(text: str) -> str:
|
||||
return f"{text} — {hint}" if hint else text
|
||||
|
||||
|
||||
# Classes whose hint is safe to attach on the CONTEXT-FREE surfaces (the
|
||||
# global 500 handler in main.py, the model-install SSE in setup/download.py),
|
||||
# where all we have is a raw error string with no stage. Only classes whose
|
||||
# classify() trigger is unmistakable belong here — e.g. VIDEO_DOWNLOAD_NETWORK
|
||||
# must NOT be added: its bare "timed out" trigger would stamp a "video server"
|
||||
# hint on a model-load timeout that leaks through the 500 handler.
|
||||
_CONTEXT_FREE_HINT_CLASSES = frozenset({
|
||||
"SOCKS_PROXY_SUPPORT_MISSING",
|
||||
"SSL_HANDSHAKE_FAILURE",
|
||||
})
|
||||
|
||||
|
||||
def append_hint(text: str) -> str:
|
||||
"""``"{text} — {hint}"`` for raw-string surfaces (the global 500 handler,
|
||||
the model-install SSE): the dynamic mirror hint (#874) when that class
|
||||
applies, else a context-free static class hint (#959). ``text`` unchanged
|
||||
otherwise — a no-op for every other error. Never raises."""
|
||||
try:
|
||||
hint = hf_mirror_hint(text)
|
||||
if not hint:
|
||||
topic = classify(text)
|
||||
if topic in _CONTEXT_FREE_HINT_CLASSES:
|
||||
hint = _HINTS.get(topic, "")
|
||||
except Exception:
|
||||
return text
|
||||
return f"{text} — {hint}" if hint else text
|
||||
|
||||
|
||||
def classify(reason: str) -> str:
|
||||
"""Map a failure reason to a docs-taxonomy key, or "" when unknown.
|
||||
|
||||
@@ -220,6 +250,28 @@ def classify(reason: str) -> str:
|
||||
)
|
||||
):
|
||||
return "TRANSFORMERS_IMPORT"
|
||||
# #959: httpx raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS
|
||||
# proxy, but the 'socksio' package is not installed. Make sure to install
|
||||
# httpx using `pip install httpx[socks]`.") when ALL_PROXY/HTTPS_PROXY is
|
||||
# socks5:// and socksio isn't importable. It surfaced from
|
||||
# huggingface_hub's get_session() inside model load — a bare 500 on
|
||||
# /generate with no next step. Checked BEFORE the HF-auth/mirror rules so
|
||||
# a message that also carries HF wording still names this class.
|
||||
if "socks proxy" in low or "socksio" in low:
|
||||
return "SOCKS_PROXY_SUPPORT_MISSING"
|
||||
# #976: a TLS handshake failing AFTER the TCP connection succeeds — the
|
||||
# signature of a corporate/antivirus proxy that TLS-inspects traffic and
|
||||
# re-signs certificates with a CA the OS trusts but Python's bundled
|
||||
# certifi list doesn't (a different failure mode from #984's TCP-level
|
||||
# "can't reach the host at all"). Requires "ssl" plus a handshake/cert-
|
||||
# verify marker so a generic connection error isn't mislabelled.
|
||||
if "ssl" in low and (
|
||||
"handshake" in low
|
||||
or "certificate verify failed" in low
|
||||
or "sslv3_alert" in low
|
||||
or "sslcertverificationerror" in low
|
||||
):
|
||||
return "SSL_HANDSHAKE_FAILURE"
|
||||
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
|
||||
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
|
||||
):
|
||||
|
||||
@@ -24,7 +24,7 @@ from pathlib import Path
|
||||
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
|
||||
# release.yml's version-bump job, so it stays equal to
|
||||
# pyproject/tauri.conf/Cargo/package.json.
|
||||
_FALLBACK_VERSION = "0.3.10"
|
||||
_FALLBACK_VERSION = "0.3.14"
|
||||
|
||||
|
||||
def _fallback_version() -> str:
|
||||
|
||||
@@ -70,6 +70,7 @@ class Supertonic3Backend(SubprocessBackend):
|
||||
id = "supertonic3"
|
||||
display_name = "Supertonic-3 (31 langs, CPU ONNX, 7 preset voices, OpenRAIL-M)"
|
||||
supports_voice_design = False # preset voices only
|
||||
supports_cloning = False # preset voices only; generate() never reads ref_audio
|
||||
# TTS-04: honest hardware reporting. Supertonic-3 has no CUDA / MPS
|
||||
# path in the SDK ‑‑ ONNX Runtime CPU EP only.
|
||||
gpu_compat: tuple[str, ...] = ("cpu",)
|
||||
|
||||
+96
-14
@@ -139,6 +139,24 @@ os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
|
||||
os.environ.setdefault("HF_HUB_ETAG_TIMEOUT", "15")
|
||||
os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "30")
|
||||
|
||||
# ── OS trust store for TLS (#976) ───────────────────────────────────────────
|
||||
# Users behind a corporate/antivirus proxy that TLS-inspects HTTPS traffic get
|
||||
# a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure"
|
||||
# on every model install — the TCP connection succeeds (a different failure
|
||||
# mode from #984's TCP-level blocked-host case), but the proxy re-signs the
|
||||
# certificate with its own root CA, which the OS trusts (Windows CryptoAPI/
|
||||
# SChannel) and Python's bundled `certifi` CA list does not. `inject_into_ssl`
|
||||
# patches `ssl.SSLContext` process-wide to verify against the OS trust store
|
||||
# instead, which is the actual fix (not just a nicer error message). Must run
|
||||
# here — at MODULE level, before huggingface_hub/requests/httpx do any network
|
||||
# I/O — not inside lifespan(), which runs too late. Not platform-gated: it's a
|
||||
# correctness improvement everywhere. Best-effort: never block startup.
|
||||
try:
|
||||
import truststore
|
||||
|
||||
truststore.inject_into_ssl()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Prevent torchaudio from lazy-importing torchcodec (broken on some installs).
|
||||
# Proper fix = exclude torchcodec in pyproject.toml; this is a belt-and-braces guard.
|
||||
@@ -155,6 +173,16 @@ from logging.handlers import RotatingFileHandler
|
||||
# written to prefs.json so they survive backend restarts. Read them back
|
||||
# here — before any user code reads os.environ — so the values are available
|
||||
# from startup.
|
||||
#
|
||||
# Legacy (≤v0.3.7) Translation-LLM rows (env.TRANSLATE_*) must migrate into
|
||||
# the custom LLM provider's settings store BEFORE the re-import below — once
|
||||
# TRANSLATE_BASE_URL lands in os.environ it hijacks the LLM provider
|
||||
# selection for the whole session (#963). Real env vars are untouched.
|
||||
try:
|
||||
from services.llm_providers import migrate_legacy_translate_prefs
|
||||
migrate_legacy_translate_prefs()
|
||||
except Exception:
|
||||
pass # never block startup on the migration; it retries next launch
|
||||
_PERSISTED_ENV_PREFIX = "env."
|
||||
try:
|
||||
from core.prefs import _load as _load_all_prefs
|
||||
@@ -476,6 +504,35 @@ async def _start_mcp_session_manager(session_manager, *, timeout: float):
|
||||
return task, stop, mounted
|
||||
|
||||
|
||||
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
|
||||
"""Cancel each background task and give it a bounded chance to actually
|
||||
finish before shutdown proceeds — ``None`` entries are skipped (a task
|
||||
that's conditionally created, e.g. ``capture_preload_task``, may not
|
||||
exist).
|
||||
|
||||
``task.cancel()`` alone is not enough for a task awaiting
|
||||
``run_in_executor()``: once the underlying OS thread is inside blocking
|
||||
native/import work, cancellation can't stop it, so cancel-and-move-on lets
|
||||
shutdown finish while that thread is still running — invisible to
|
||||
asyncio, but very much alive when the interpreter starts tearing down
|
||||
module state under it (#1000 class). Awaiting with a bound (instead of
|
||||
just cancelling) gives an early-stage task a real chance to exit cleanly
|
||||
first; a task that's genuinely still deep in blocking work times out here
|
||||
same as before, and the caller's own GPU-pool reset handles that case.
|
||||
"""
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
t.cancel()
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=timeout)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
|
||||
@@ -550,6 +607,7 @@ async def lifespan(app: FastAPI):
|
||||
# lean and the first dictation is instant instead of a cold model load.
|
||||
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
|
||||
# under 4 GB free RAM (checked at warm time, not boot time).
|
||||
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
|
||||
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
|
||||
async def _preload_capture_asr():
|
||||
await asyncio.sleep(_capture_preload_delay_s())
|
||||
@@ -618,14 +676,33 @@ async def lifespan(app: FastAPI):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
idle_task.cancel()
|
||||
worker_task.cancel()
|
||||
# Wait for tasks to finish their current iteration
|
||||
for t in (idle_task, worker_task):
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=3.0)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
# preload_task/capture_preload_task matter most here (#1000 class): a quit
|
||||
# mid-preload used to fall straight through to "Shutdown: done." while the
|
||||
# model load was still running on a GPU-pool thread — cancel() can't stop
|
||||
# a thread already inside blocking import/load work, so the process
|
||||
# reported a clean exit while that background thread was still mid-
|
||||
# `import transformers`, and got torn down by interpreter finalization
|
||||
# instead. That surfaced as a misleading "Could not import module
|
||||
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
|
||||
# not a real dependency problem. Awaiting here lets an early-stage load
|
||||
# (still importing, not yet mid weight-download) finish cleanly before we
|
||||
# report done; a load that's genuinely deep into a multi-GB download still
|
||||
# times out — _reset_gpu_pool() below abandons it either way.
|
||||
#
|
||||
# 20s, not the original 3s (code-review finding post-merge): a cold
|
||||
# transformers import alone can take longer than 3s on a slow disk or a
|
||||
# first-ever launch, so the original bound left a real residual window —
|
||||
# cancellation detaches the asyncio task, but the underlying OS thread
|
||||
# keeps running past it, and shutdown could still report "done" while
|
||||
# that thread was alive. Python cannot forcibly kill a running thread, so
|
||||
# no finite bound eliminates this outright — 20s just shrinks the window
|
||||
# from "any preload" to "an unusually slow cold-import," which is the
|
||||
# practical ceiling before a longer shutdown itself becomes the
|
||||
# complaint. A thread that's still running past 20s was never going to
|
||||
# finish in a shutdown-appropriate timeframe regardless.
|
||||
await _cancel_and_await_tasks(
|
||||
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
|
||||
)
|
||||
# Unload the model and free GPU memory
|
||||
try:
|
||||
import services.model_manager as mm
|
||||
@@ -633,6 +710,10 @@ async def lifespan(app: FastAPI):
|
||||
mm.model = None
|
||||
logger.info("Shutdown: model unloaded.")
|
||||
mm.free_vram()
|
||||
# Abandon a still-running preload's GPU-pool thread (Python can't kill
|
||||
# a thread mid blocking call) so it can't outlive this shutdown block
|
||||
# holding a reference into module state that's about to be torn down.
|
||||
mm._reset_gpu_pool()
|
||||
except Exception:
|
||||
pass
|
||||
# Run GC to release any remaining references
|
||||
@@ -718,13 +799,14 @@ async def global_exception_handler(request: Request, exc: Exception):
|
||||
# #874: a model download that failed because the CONFIGURED Hugging Face
|
||||
# mirror (HF_ENDPOINT) is unreachable used to leak the raw transformers
|
||||
# message ("We couldn't connect to 'https://hf-mirror.com' …") as the 500
|
||||
# detail with no next step. Appending the shared mirror hint HERE covers
|
||||
# every route that can leak a model-load/download error (generate, dub,
|
||||
# archetypes, …), not just TTS generate. append_hf_mirror_hint is a no-op
|
||||
# for every other error and never raises.
|
||||
from core.failure import append_hf_mirror_hint
|
||||
# detail with no next step. #959: same story for the SOCKS-proxy class
|
||||
# ("Using SOCKS proxy, but the 'socksio' package is not installed").
|
||||
# Appending the shared hints HERE covers every route that can leak a
|
||||
# model-load/download error (generate, dub, archetypes, …), not just TTS
|
||||
# generate. append_hint is a no-op for every other error and never raises.
|
||||
from core.failure import append_hint
|
||||
return JSONResponse(
|
||||
{"detail": append_hf_mirror_hint(str(exc)), "error_class": _entry.get("error_class")},
|
||||
{"detail": append_hint(str(exc)), "error_class": _entry.get("error_class")},
|
||||
status_code=500,
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
@@ -29,6 +29,7 @@ import os
|
||||
import re
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.asr")
|
||||
|
||||
@@ -1634,6 +1635,161 @@ class FunASRBackend(ASRBackend):
|
||||
pass
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote transcription (#877 — Qwen3-ASR / FunASR / any
|
||||
# compatible server, today, without waiting on transformers to catch up) ──
|
||||
#
|
||||
# transformers doesn't yet ship a stable Qwen3-ASR integration (issue #877),
|
||||
# but a self-hosted Qwen3-ASR/FunASR/SenseVoice server exposing an
|
||||
# OpenAI-compatible `POST /v1/audio/transcriptions` endpoint — or OpenAI's own
|
||||
# Whisper API — is usable right now. This backend is a pure network client:
|
||||
# no model runs locally, so it needs no install and claims no GPU.
|
||||
#
|
||||
# Settings mirror the LLM-providers convention exactly (services/
|
||||
# llm_providers.py): base_url/model are plain settings_store text rows; the
|
||||
# API key is Fernet-encrypted via settings_store.set_secret/get_secret — never
|
||||
# a .env row, never echoed back to the client. Optional: some self-hosted
|
||||
# servers (vLLM, LM Studio-style) don't check the key at all.
|
||||
|
||||
_ASR_OPENAI_COMPAT_BASE_URL_KEY = "asr.openai_compat.base_url"
|
||||
_ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_base_url() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_BASE_URL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_BASE_URL_KEY)
|
||||
or ""
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_model() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_MODEL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_MODEL_KEY)
|
||||
or "whisper-1"
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_api_key() -> Optional[str]:
|
||||
"""Env → encrypted stored key → None. Unlike LLM providers, no 'local'
|
||||
sentinel: many self-hosted transcription servers accept an empty/omitted
|
||||
Authorization header outright, so the OpenAI SDK is constructed with
|
||||
``api_key="not-needed"`` (a non-empty placeholder the SDK requires) when
|
||||
this returns None, rather than treating a keyless server as unconfigured.
|
||||
"""
|
||||
from services import settings_store
|
||||
return os.environ.get("ASR_OPENAI_COMPAT_API_KEY") or settings_store.get_secret(
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME
|
||||
)
|
||||
|
||||
|
||||
def openai_compat_asr_has_key() -> bool:
|
||||
"""Whether a key is configured, without ever decrypting it — mirrors
|
||||
llm_providers.has_key()'s no-plaintext-round-trip contract."""
|
||||
from services import settings_store
|
||||
if os.environ.get("ASR_OPENAI_COMPAT_API_KEY"):
|
||||
return True
|
||||
return _ASR_OPENAI_COMPAT_SECRET_NAME in settings_store.list_secret_names()
|
||||
|
||||
|
||||
class OpenAICompatASRBackend(ASRBackend):
|
||||
"""Remote transcription via any OpenAI-compatible server.
|
||||
|
||||
Adapts whatever the server returns into this module's expected shape.
|
||||
Prefers `response_format="verbose_json"` for real per-segment timestamps
|
||||
(OpenAI's own API and most compatible servers support it); falls back to
|
||||
plain text with rough single-segment bounds — mirroring
|
||||
MoonshineASRBackend's degraded shape — for minimal servers that reject it.
|
||||
"""
|
||||
id = "openai-compat-asr"
|
||||
display_name = "OpenAI-compatible (remote server)"
|
||||
gpu_compat = ("cpu",) # network client only — no local compute
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = resolve_openai_compat_asr_base_url()
|
||||
self._model = resolve_openai_compat_asr_model()
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
if not resolve_openai_compat_asr_base_url():
|
||||
return False, "Configure a server endpoint in Settings → Engines"
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
except ImportError:
|
||||
return False, "openai package not installed. Install with: uv pip install openai"
|
||||
return True, "ready"
|
||||
|
||||
def _client(self):
|
||||
from openai import OpenAI
|
||||
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
|
||||
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
|
||||
# rate-limited/slow server retrying inside the SDK would blow past
|
||||
# whatever bounded timeout the caller (dub transcribe, dictation)
|
||||
# expects from a single call.
|
||||
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
logger.info(
|
||||
"OpenAI-compat ASR transcribing %s (base_url=%s, model=%s)",
|
||||
audio_path, self._base_url, self._model,
|
||||
)
|
||||
client = self._client()
|
||||
try:
|
||||
with open(audio_path, "rb") as f:
|
||||
try:
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="verbose_json",
|
||||
)
|
||||
except Exception:
|
||||
# Minimal/older compatible servers reject verbose_json
|
||||
# outright — retry plain before treating it as a real
|
||||
# failure. Re-open: the SDK may have partially consumed
|
||||
# the file handle on the first attempt.
|
||||
f.seek(0)
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="json",
|
||||
)
|
||||
except Exception as exc:
|
||||
# Never leak a raw SDK/httpx exception object (auth headers,
|
||||
# connection internals) straight into a user-facing message —
|
||||
# same convention as generation.py's _safe_exc_text (#977 class).
|
||||
raise RuntimeError(
|
||||
f"OpenAI-compatible ASR server at {self._base_url!r} failed: "
|
||||
f"{type(exc).__name__}: {exc}"
|
||||
) from exc
|
||||
return self._adapt_response(resp)
|
||||
|
||||
@staticmethod
|
||||
def _adapt_response(resp) -> dict:
|
||||
segments_out = []
|
||||
# verbose_json: resp.segments is a list of objects with start/end/text.
|
||||
raw_segments = getattr(resp, "segments", None)
|
||||
if raw_segments:
|
||||
for seg in raw_segments:
|
||||
seg_dict = seg if isinstance(seg, dict) else seg.model_dump()
|
||||
segments_out.append({
|
||||
"text": (seg_dict.get("text") or "").strip(),
|
||||
"start": seg_dict.get("start", 0.0),
|
||||
"end": seg_dict.get("end", 0.0),
|
||||
"words": [], # word-level timing isn't part of this API
|
||||
})
|
||||
else:
|
||||
# Plain text response (json/text format) — single-segment shape,
|
||||
# matching MoonshineASRBackend's degraded fallback exactly.
|
||||
text = (getattr(resp, "text", None) or "").strip()
|
||||
if text:
|
||||
segments_out.append({"text": text, "start": 0.0, "end": None, "words": []})
|
||||
chunks = [
|
||||
{"text": seg["text"], "timestamp": (seg["start"], seg["end"])}
|
||||
for seg in segments_out
|
||||
]
|
||||
language = getattr(resp, "language", None) or "en"
|
||||
return {"chunks": chunks, "segments": segments_out, "language": language}
|
||||
|
||||
|
||||
def _isolated_faster_whisper():
|
||||
"""Lazy import so the subprocess_asr → subprocess_backend chain isn't
|
||||
pulled in at registry definition time."""
|
||||
@@ -1689,6 +1845,7 @@ _REGISTRY: dict[str, type[ASRBackend]] = _LazyASRRegistry({
|
||||
"moonshine": MoonshineASRBackend,
|
||||
"funasr": FunASRBackend,
|
||||
"sherpa-onnx-asr": SherpaDictationBackend,
|
||||
"openai-compat-asr": OpenAICompatASRBackend,
|
||||
# "faster-whisper-isolated": resolved lazily (crash-isolated subprocess).
|
||||
})
|
||||
|
||||
@@ -1700,10 +1857,26 @@ _INSTALL_HINTS: dict[str, str] = {
|
||||
"faster-whisper": "pip install faster-whisper (CTranslate2; cross-platform, CUDA or CPU)",
|
||||
"mlx-whisper": "pip install mlx-whisper (Apple Silicon only)",
|
||||
"pytorch-whisper": "Bundled with transformers — no extra install (CUDA/MPS/CPU)",
|
||||
"nemo-parakeet": "pip install nemo_toolkit[asr] (NVIDIA Parakeet; CUDA or CPU)",
|
||||
"nemo-parakeet": (
|
||||
"No safe install path in this app yet — nemo_toolkit's ASR extras pin "
|
||||
"transformers>=4.57,<4.58, which conflicts with OmniVoice's own "
|
||||
"transformers>=5.3 requirement and WILL break the backend "
|
||||
"(ImportError on startup) if installed into this shared venv. Do NOT "
|
||||
"install nemo_toolkit here. If you want to try Parakeet TDT, set it "
|
||||
"up in a separate/dedicated Python environment — not the one "
|
||||
"OmniVoice manages; in-app isolation for this engine is tracked "
|
||||
"separately."
|
||||
),
|
||||
"moonshine": "pip install useful-moonshine (edge/CPU-optimized ASR)",
|
||||
"funasr": "pip install funasr (SenseVoiceSmall + FSMN-VAD; CUDA or CPU)",
|
||||
"sherpa-onnx-asr": "uv add sherpa-onnx (ONNX live dictation; CPU, cross-platform)",
|
||||
"openai-compat-asr": (
|
||||
"No install needed — configure a server endpoint in Settings → "
|
||||
"Engines. Points OmniVoice at any OpenAI-compatible transcription "
|
||||
"server (a self-hosted Qwen3-ASR/FunASR/SenseVoice server, OpenAI's "
|
||||
"own Whisper API, or similar) — a path to Qwen3-ASR today, without "
|
||||
"waiting on a direct transformers integration."
|
||||
),
|
||||
"faster-whisper-isolated": (
|
||||
"No extra install (reuses faster-whisper). Escape hatch for hanging "
|
||||
"transcribes: runs ASR in a separate process that can be force-killed "
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
"""
|
||||
Audio DSP pipeline — broadcast-grade mastering + configurable effects chain.
|
||||
|
||||
The default `apply_mastering()` is the same chain shipped since v0.1.0
|
||||
(highpass + compressor + light reverb). The new `apply_effects_chain()`
|
||||
lets callers build custom pipelines from a list of named effects.
|
||||
`apply_mastering()` is the shared pre-stage that runs before the user's
|
||||
effect preset: highpass + gentle compression only (see `MASTERING_CHAIN`).
|
||||
Reverb is deliberately NOT part of it — it is preset-declared only (e.g.
|
||||
cinematic, warm); a hidden reverb here used to bake echo into every non-raw
|
||||
synthesis, which field reports flagged. `apply_effects_chain()` lets callers
|
||||
build custom pipelines from a list of named effects.
|
||||
|
||||
All effects use Spotify's `pedalboard` library. When pedalboard isn't
|
||||
installed, every function degrades gracefully (returns audio unmodified).
|
||||
@@ -97,24 +100,26 @@ def get_effect_chain(preset_id: str) -> list[dict]:
|
||||
|
||||
# ── Core DSP functions ──────────────────────────────────────────────────
|
||||
|
||||
#: Shared pre-preset mastering stage: highpass + gentle compression ONLY.
|
||||
#: Reverb must never live here — a hidden Reverb in this chain baked echo
|
||||
#: into every non-raw synthesis regardless of the chosen preset (field
|
||||
#: reports of echoey voices; the podcast preset even promises "no reverb").
|
||||
#: Reverb is preset-declared only (see EFFECT_PRESETS: cinematic, warm).
|
||||
MASTERING_CHAIN = [
|
||||
{"type": "highpass", "cutoff_hz": 60},
|
||||
{"type": "compressor", "threshold_db": -15, "ratio": 1.5, "attack_ms": 2.0, "release_ms": 100},
|
||||
]
|
||||
|
||||
|
||||
def apply_mastering(audio_tensor, sample_rate=24000):
|
||||
"""Applies professional Broadcast-grade DSP (EQ, Compressor, light Reverb) to the clone voice."""
|
||||
"""Applies the broadcast pre-stage (highpass + gentle compression) to the clone voice.
|
||||
|
||||
Reverb is intentionally absent — only user-chosen effect presets declare
|
||||
it. Degrades gracefully: pedalboard missing or any DSP error returns the
|
||||
input unmodified.
|
||||
"""
|
||||
try:
|
||||
from pedalboard import Pedalboard, Compressor, Reverb, HighpassFilter
|
||||
import numpy as np
|
||||
board = Pedalboard([
|
||||
HighpassFilter(cutoff_frequency_hz=60),
|
||||
Compressor(threshold_db=-15, ratio=1.5, attack_ms=2.0, release_ms=100),
|
||||
Reverb(room_size=0.10, wet_level=0.08, dry_level=0.95)
|
||||
])
|
||||
audio_np = audio_tensor.cpu().numpy()
|
||||
if audio_np.ndim == 1:
|
||||
audio_np = audio_np[np.newaxis, :]
|
||||
effected = board(audio_np, sample_rate, reset=False)
|
||||
return torch.from_numpy(effected).to(audio_tensor.device)
|
||||
except ImportError:
|
||||
return audio_tensor # Fail gracefully if pedalboard isn't installed
|
||||
return apply_effects_chain(audio_tensor, sample_rate, MASTERING_CHAIN)
|
||||
except Exception as e:
|
||||
logger.warning("Mastering DSP Error: %s", e)
|
||||
return audio_tensor
|
||||
|
||||
@@ -187,6 +187,14 @@ def put_job(job_id: str, job: dict) -> None:
|
||||
def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0, content_hash: str = "") -> None:
|
||||
"""Persist dub job state to SQLite so it survives restarts. Uses UPSERT
|
||||
on `id` so repeated saves in a session keep the latest snapshot.
|
||||
|
||||
language / language_code / content_hash only update when the incoming
|
||||
value is non-empty: the ingest-time insert runs before the target
|
||||
language is known (both columns ""), generation sets them on the job
|
||||
dict, and a later save from a job that lost them (e.g. hydrated from an
|
||||
old row) must not clobber the healed columns back to "". The frontend
|
||||
keys history restore off language_code, so a frozen "" hid finished
|
||||
tracks until the user re-picked a language.
|
||||
"""
|
||||
try:
|
||||
segments = job.get("segments") or []
|
||||
@@ -200,6 +208,8 @@ def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0,
|
||||
filename=excluded.filename,
|
||||
duration=excluded.duration,
|
||||
segments_count=excluded.segments_count,
|
||||
language=CASE WHEN excluded.language != '' THEN excluded.language ELSE dub_history.language END,
|
||||
language_code=CASE WHEN excluded.language_code != '' THEN excluded.language_code ELSE dub_history.language_code END,
|
||||
tracks=excluded.tracks,
|
||||
job_data=excluded.job_data,
|
||||
content_hash=CASE WHEN excluded.content_hash != '' THEN excluded.content_hash ELSE dub_history.content_hash END""",
|
||||
|
||||
@@ -49,7 +49,7 @@ def _canon_value(field: str, value):
|
||||
return value
|
||||
|
||||
|
||||
def segment_fingerprint(seg: dict) -> str:
|
||||
def segment_fingerprint(seg: dict, track_lang: str | None = None) -> str:
|
||||
"""Deterministic hash of the inputs that actually affect TTS output.
|
||||
|
||||
Any change to `_GEN_INPUT_FIELDS` flips the hash and the segment becomes
|
||||
@@ -61,8 +61,20 @@ def segment_fingerprint(seg: dict) -> str:
|
||||
so a fingerprint computed from the generate request (server defaults
|
||||
filled in) matches one recomputed later from the client's raw segment
|
||||
state — the root cause of #281's "1 edit re-dubs all N lines".
|
||||
|
||||
``track_lang`` (P1.3) is the TRACK's language code (`req.language_code`,
|
||||
e.g. "es"). It is part of the fingerprint because the same segment text
|
||||
renders different audio per language — without it, a bn hash could
|
||||
vouch for an es WAV on a multi-track job. It is only mixed in when
|
||||
provided, so hashes computed by legacy callers (and hashes stored by
|
||||
previous builds, which never carried a language) keep their old values;
|
||||
a legacy hash therefore never matches a lang-scoped fingerprint and the
|
||||
segment reads as stale — the safe direction (one clean regen, never a
|
||||
wrong-language splice).
|
||||
"""
|
||||
payload = {k: _canon_value(k, seg.get(k)) for k in _GEN_INPUT_FIELDS}
|
||||
if track_lang:
|
||||
payload["track_lang"] = str(track_lang)
|
||||
blob = json.dumps(payload, sort_keys=True, ensure_ascii=False)
|
||||
return hashlib.sha1(blob.encode("utf-8"), usedforsecurity=False).hexdigest()[:16]
|
||||
|
||||
@@ -120,6 +132,7 @@ def plan_incremental(
|
||||
segments: list[dict],
|
||||
*,
|
||||
stored_hashes: dict[str, str] | None = None,
|
||||
track_lang: str | None = None,
|
||||
) -> dict:
|
||||
"""Return `{stale, fresh, total, fingerprints}` where:
|
||||
|
||||
@@ -133,6 +146,13 @@ def plan_incremental(
|
||||
`stored_hashes` may come from the caller's own bookkeeping (e.g. the
|
||||
`dub_history.job_data["seg_hashes"]` we'll start writing in Phase 4.5).
|
||||
When missing, every segment is considered stale (first run).
|
||||
|
||||
`track_lang` (P1.3) scopes the plan to ONE dub track: pass the track's
|
||||
language code together with THAT language's stored hashes
|
||||
(`job_data["seg_hashes_by_lang"][lang]`) so staleness is judged against
|
||||
the active track, never against whatever language was generated last.
|
||||
Must match the language the generate run hashed with, or every segment
|
||||
reads stale (#281 parity class).
|
||||
"""
|
||||
stored = stored_hashes or {}
|
||||
stale: list[str] = []
|
||||
@@ -142,7 +162,7 @@ def plan_incremental(
|
||||
sid = str(seg.get("id", ""))
|
||||
if not sid:
|
||||
continue
|
||||
fp = segment_fingerprint(seg)
|
||||
fp = segment_fingerprint(seg, track_lang=track_lang)
|
||||
fingerprints[sid] = fp
|
||||
prev = stored.get(sid)
|
||||
if prev == fp:
|
||||
|
||||
@@ -24,10 +24,13 @@ users.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.llm_providers")
|
||||
|
||||
# Settings-store row names (non-secret overrides live in the plaintext table;
|
||||
# keys live in the encrypted secret table under ``llm_key.<id>``).
|
||||
_ACTIVE_PROVIDER_KEY = "llm.active_provider"
|
||||
@@ -248,6 +251,19 @@ def is_configured(p: Provider) -> bool:
|
||||
|
||||
# ── Active provider selection ─────────────────────────────────────────────
|
||||
|
||||
def stored_active_provider_id() -> Optional[str]:
|
||||
"""The user's explicitly-persisted selection ONLY — no env pin, no legacy
|
||||
TRANSLATE_* fallback, no auto-detect.
|
||||
|
||||
``None`` means the user has never chosen a provider. This is what gates
|
||||
save-activates in the settings router (#963): an explicit save may claim
|
||||
the *empty* slot, but must never steal it from a made choice.
|
||||
"""
|
||||
from services import settings_store
|
||||
stored = settings_store.get_text(_ACTIVE_PROVIDER_KEY)
|
||||
return stored if stored and stored in _BY_ID else None
|
||||
|
||||
|
||||
def active_provider_id() -> Optional[str]:
|
||||
"""The provider Cinematic/Autofit should use.
|
||||
|
||||
@@ -255,12 +271,11 @@ def active_provider_id() -> Optional[str]:
|
||||
configured provider → None. Legacy ``TRANSLATE_BASE_URL`` users with no
|
||||
explicit selection resolve to ``custom`` (its envs are TRANSLATE_*).
|
||||
"""
|
||||
from services import settings_store
|
||||
env_pick = os.environ.get("LLM_DEFAULT_PROVIDER")
|
||||
if env_pick and env_pick in _BY_ID:
|
||||
return env_pick
|
||||
stored = settings_store.get_text(_ACTIVE_PROVIDER_KEY)
|
||||
if stored and stored in _BY_ID:
|
||||
stored = stored_active_provider_id()
|
||||
if stored:
|
||||
return stored
|
||||
# Legacy: a lone TRANSLATE_BASE_URL means the old single-endpoint setup.
|
||||
if os.environ.get("TRANSLATE_BASE_URL"):
|
||||
@@ -354,3 +369,69 @@ def describe(p: Provider) -> dict:
|
||||
d["account_id"] = resolve_account_id(p)
|
||||
d["account_from_env"] = bool(p.account_env and os.environ.get(p.account_env))
|
||||
return d
|
||||
|
||||
|
||||
# ── Legacy TRANSLATE_* prefs migration (#963) ──────────────────────────────
|
||||
|
||||
# prefs.json row → the custom-provider field it becomes.
|
||||
_LEGACY_TRANSLATE_PREFS: tuple[tuple[str, str], ...] = (
|
||||
("env.TRANSLATE_BASE_URL", "base_url"),
|
||||
("env.TRANSLATE_MODEL", "model"),
|
||||
("env.TRANSLATE_API_KEY", "api_key"),
|
||||
)
|
||||
|
||||
|
||||
def migrate_legacy_translate_prefs() -> bool:
|
||||
"""Move the retired (≤v0.3.7) Translation-LLM panel's prefs rows into the
|
||||
``custom`` provider's own settings-store rows, then delete them.
|
||||
|
||||
Those ``env.TRANSLATE_*`` rows in prefs.json are re-imported into
|
||||
``os.environ`` on every launch (main.py), and a live ``TRANSLATE_BASE_URL``
|
||||
makes :func:`active_provider_id` resolve to ``custom`` ahead of the stored
|
||||
selection fallbacks — silently hijacking the active slot on every restart
|
||||
(issue #963, "Ollama works until I restart"). Must run BEFORE main.py's
|
||||
prefs→env import so the rows never reach the environment.
|
||||
|
||||
Semantics:
|
||||
* Each value is copied only where the store has no value yet — a user's
|
||||
later edit of the custom provider always wins over legacy leftovers.
|
||||
* The prefs row is deleted afterwards either way, so it can never be
|
||||
re-imported as env again (the migration is one-shot per row).
|
||||
* Real process env vars are NEVER touched — a shell/.env
|
||||
``TRANSLATE_BASE_URL`` keeps its documented override behavior.
|
||||
* A row whose store write fails is kept in prefs (it still works via the
|
||||
env import this launch and the migration retries next launch).
|
||||
|
||||
Returns True if any prefs row was migrated/removed.
|
||||
"""
|
||||
from core import prefs
|
||||
from services import settings_store
|
||||
|
||||
changed = False
|
||||
for prefs_key, field in _LEGACY_TRANSLATE_PREFS:
|
||||
try:
|
||||
raw = prefs.get(prefs_key)
|
||||
except Exception:
|
||||
logger.exception("legacy TRANSLATE prefs read failed (%s)", prefs_key)
|
||||
return changed
|
||||
if raw is None:
|
||||
continue
|
||||
val = str(raw).strip()
|
||||
try:
|
||||
if val:
|
||||
if field == "base_url":
|
||||
if not settings_store.get_text(_BASE_URL_KEY + "custom"):
|
||||
save_overrides("custom", base_url=val)
|
||||
elif field == "model":
|
||||
if not settings_store.get_text(_MODEL_KEY + "custom"):
|
||||
save_overrides("custom", model=val)
|
||||
else: # api_key — encrypted store, never overwrite an existing one
|
||||
if not _key_in_store("custom"):
|
||||
save_key("custom", val)
|
||||
prefs.delete(prefs_key)
|
||||
changed = True
|
||||
except Exception:
|
||||
# Store not ready (e.g. settings table missing) — keep the prefs
|
||||
# row so the legacy env import still works and we retry next boot.
|
||||
logger.exception("legacy TRANSLATE prefs migration failed (%s)", prefs_key)
|
||||
return changed
|
||||
|
||||
@@ -242,8 +242,24 @@ def resolve_skill_client(skill_id: str) -> Optional[SkillClient]:
|
||||
# skill's wall-clock budget (the cinematic pass budget, the glossary call
|
||||
# timeout) from inside one request. Fail fast — the per-call timeout and the
|
||||
# pass-level budget are the only bounds we want. Mirrors OpenAICompatBackend.
|
||||
#
|
||||
# #959 class guard: OpenAI() eagerly builds its httpx client, which can
|
||||
# raise AT CONSTRUCTION for environment-shaped reasons — the reported one
|
||||
# is httpx's ImportError under ALL_PROXY/HTTPS_PROXY=socks5:// without
|
||||
# socksio; a malformed proxy URL or broken cert bundle fails the same way.
|
||||
# The contract here is already "None == LLM unavailable, degrade" — a bad
|
||||
# proxy env must degrade the skill, never 500 the calling feature.
|
||||
try:
|
||||
client = OpenAI(max_retries=0, **kw)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"LLM client construction failed for skill %s (provider %s): %s — "
|
||||
"treating the skill as unavailable.",
|
||||
skill_id, res.provider.id, exc,
|
||||
)
|
||||
return None
|
||||
return SkillClient(
|
||||
client=OpenAI(max_retries=0, **kw),
|
||||
client=client,
|
||||
model=llm_providers.resolve_model(res.provider),
|
||||
provider_id=res.provider.id,
|
||||
timeout=_default_timeout(),
|
||||
|
||||
@@ -45,11 +45,33 @@ def _asr_device() -> str:
|
||||
return "cpu"
|
||||
|
||||
|
||||
def _active_tts_id() -> Optional[str]:
|
||||
"""Configured TTS engine id, or None if it can't be resolved. Attribution
|
||||
is advisory — a prefs/import hiccup must never break /model/loaded."""
|
||||
try:
|
||||
from services.tts_backend import active_backend_id
|
||||
return active_backend_id()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _tts_attribution(engine_id: str, active: Optional[str]) -> dict:
|
||||
"""Per-entry engine attribution for TTS-family models. A model can stay
|
||||
resident in VRAM after the user switches engines (freed only by unload/
|
||||
idle-evict), so the panel needs to know which entry synthesis actually
|
||||
routes to. ``is_active_engine`` is None when the active id is unknown."""
|
||||
return {
|
||||
"engine_id": engine_id,
|
||||
"is_active_engine": (engine_id == active) if active is not None else None,
|
||||
}
|
||||
|
||||
|
||||
def list_loaded() -> dict:
|
||||
"""Enumerate every currently-loaded model. Shape: ``{"models": [...],
|
||||
"count": n}`` with per-model id/name/checkpoint/device/vram_mb/unloadable
|
||||
(+ optional ``note``)."""
|
||||
models: list[dict] = []
|
||||
active_tts = _active_tts_id()
|
||||
|
||||
# 1. In-process TTS model (OmniVoice)
|
||||
if mm.model is not None:
|
||||
@@ -64,6 +86,7 @@ def list_loaded() -> dict:
|
||||
"device": device,
|
||||
"vram_mb": round(_tts_vram_mb(), 1),
|
||||
"unloadable": True,
|
||||
**_tts_attribution("omnivoice", active_tts),
|
||||
})
|
||||
|
||||
# 2. ASR (WhisperX) — co-loaded with and released alongside the TTS model.
|
||||
@@ -105,6 +128,7 @@ def list_loaded() -> dict:
|
||||
"device": get_best_device(),
|
||||
"vram_mb": round(float(s.get("vram_mb") or 0), 1),
|
||||
"unloadable": True,
|
||||
**_tts_attribution(s["id"], active_tts),
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -277,9 +277,11 @@ def _timeout_guidance(what: str, timeout: float) -> str:
|
||||
)
|
||||
return common + (
|
||||
"most often the GPU is VRAM-starved (a resident model and this job "
|
||||
"contend for memory). For a durable fix try shorter text, a lighter "
|
||||
"engine, or set the engine to CPU in Settings → Models. (Raise "
|
||||
"OMNIVOICE_GENERATE_TIMEOUT_S for very long single generations.)"
|
||||
"contend for memory). For a durable fix, Flush caches / Unload the "
|
||||
"resident model (top toolbar or Settings → Models) before retrying, "
|
||||
"try shorter text, a lighter engine, or set the engine to CPU in "
|
||||
"Settings → Models. (Raise OMNIVOICE_GENERATE_TIMEOUT_S for very "
|
||||
"long single generations.)"
|
||||
)
|
||||
|
||||
|
||||
@@ -955,7 +957,13 @@ def _load_model_sync():
|
||||
except Exception: # never let failure-formatting mask the real error
|
||||
err_msg = str(exc)
|
||||
_set_loading("error", "Model loading failed", error=err_msg)
|
||||
logger.error("Model loading failed: %s", str(exc))
|
||||
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
|
||||
# to an inner import (including one interrupted by process teardown)
|
||||
# in a generic "Could not import module X. Are this object's
|
||||
# requirements defined correctly?" — logging only str(exc) discarded
|
||||
# the real cause in __cause__/__context__ and made a shutdown race
|
||||
# look like a broken install. exc_info surfaces the full chain.
|
||||
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
|
||||
raise
|
||||
finally:
|
||||
unregister_listener(lid)
|
||||
@@ -1022,6 +1030,22 @@ async def get_model():
|
||||
return model
|
||||
|
||||
|
||||
def _checkpoint_in_local_cache(checkpoint: str) -> bool:
|
||||
"""True when ``checkpoint`` is loadable with NO network: an existing local
|
||||
directory, or a COMPLETE HF cache snapshot. ``snapshot_download(...,
|
||||
local_files_only=True)`` never constructs an HTTP session, so a broken
|
||||
proxy env (#959: ``ALL_PROXY``/``HTTPS_PROXY=socks5://`` without socksio)
|
||||
can't false-negative this probe. Never raises."""
|
||||
if os.path.isdir(checkpoint):
|
||||
return True
|
||||
try:
|
||||
from huggingface_hub import snapshot_download
|
||||
snapshot_download(checkpoint, local_files_only=True)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
async def preload_model():
|
||||
"""Background model warm-up — call from lifespan startup.
|
||||
|
||||
@@ -1042,10 +1066,27 @@ async def preload_model():
|
||||
try:
|
||||
from huggingface_hub import model_info
|
||||
model_info(checkpoint, timeout=5)
|
||||
except Exception:
|
||||
# Model not downloaded yet — skip preload
|
||||
logger.info("Preload skipped: %s not available locally.", checkpoint)
|
||||
return
|
||||
except Exception as probe_err:
|
||||
# The probe failing does NOT mean the model isn't installed — it
|
||||
# means the Hub API wasn't reachable from this process. The #959
|
||||
# class: under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio,
|
||||
# hf_hub's get_session() raises ImportError AT CLIENT CONSTRUCTION;
|
||||
# same story for offline mode, DNS, or firewall failures. Fall back
|
||||
# to a cache-only probe (no HTTP session involved) and warm up
|
||||
# anyway when the model is locally present, instead of silently
|
||||
# skipping and letting the first /generate eat the full load.
|
||||
if not _checkpoint_in_local_cache(checkpoint):
|
||||
logger.info(
|
||||
"Preload skipped: %s not available locally (network probe "
|
||||
"failed: %s: %s).",
|
||||
checkpoint, type(probe_err).__name__, probe_err,
|
||||
)
|
||||
return
|
||||
logger.warning(
|
||||
"Network probe for %s failed (%s: %s) — model found in the "
|
||||
"local cache; warming up from cache.",
|
||||
checkpoint, type(probe_err).__name__, probe_err,
|
||||
)
|
||||
|
||||
logger.info("Preloading TTS model in background…")
|
||||
_last_used = time.time()
|
||||
@@ -1054,7 +1095,11 @@ async def preload_model():
|
||||
model = await _load_model_with_timeout()
|
||||
logger.info("Preload complete — model ready.")
|
||||
except Exception as e:
|
||||
logger.warning("Model preload failed (non-fatal): %s", e)
|
||||
# See the matching exc_info note on the _load_model_sync handler above
|
||||
# (#1000 class) — the full chain, not just str(e), is what actually
|
||||
# distinguishes a real dependency problem from a shutdown-interrupted
|
||||
# import.
|
||||
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
|
||||
|
||||
def get_model_status():
|
||||
is_loaded = model is not None
|
||||
|
||||
@@ -10,8 +10,8 @@ plays the moment the video begins and everything feels desynchronised.
|
||||
``snap_segment_starts`` post-processes segments against the actual audio
|
||||
(ideally the Demucs-isolated vocals track, which the dub pipeline already
|
||||
produces): for each segment it scans the waveform inside ``[start, end]``
|
||||
for the first frame whose RMS rises above an adaptive threshold and moves
|
||||
``start`` forward to just before that onset.
|
||||
for the first *sustained* rise of frame RMS above an adaptive threshold
|
||||
and moves ``start`` forward to just before that onset.
|
||||
|
||||
Design constraints:
|
||||
|
||||
@@ -23,6 +23,27 @@ Design constraints:
|
||||
(no frame above the absolute floor) are left untouched.
|
||||
* **Pure NumPy.** No model, no platform-specific code — identical
|
||||
behaviour on macOS / Windows / Linux, trivially unit-testable.
|
||||
|
||||
Robustness against non-speech onsets (#963): a field report showed dubbed
|
||||
lines starting seconds off because "when a noise is heard (a sigh or
|
||||
footsteps), it's interpreted as the start of the conversation". Three
|
||||
guards address that class of failure:
|
||||
|
||||
* **Sustained energy.** A frame only counts as an onset when the energy
|
||||
stays up for a speech-like duration (``SUSTAIN_MIN_S`` within the
|
||||
following ``SUSTAIN_WINDOW_S``). Footsteps/door thuds/clicks light up
|
||||
one or two 20 ms frames and die; syllables keep the energy up.
|
||||
* **Bounded snap distance.** Shifts beyond ``MAX_SNAP_S`` are only
|
||||
trusted when everything being skipped is (near-)silence — the genuine
|
||||
#280 whisper start-stretch, where Demucs removed the leading music and
|
||||
left real silence on the vocals track. Jumping far over *audible*
|
||||
content (e.g. quiet speech sitting under the relative threshold) would
|
||||
play the dub seconds late, so it is refused.
|
||||
* **Source-aware.** Snapping only runs on a separated vocals track
|
||||
(``separated_vocals=True``). On mixed/original audio — Demucs skipped
|
||||
or failed — music, ambience and room tone are all legitimate sustained
|
||||
energy, so any detected "onset" is as likely the score as the speaker;
|
||||
whisper's own timestamps beat a confidently wrong snap.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -51,6 +72,23 @@ RELATIVE_THRESHOLD = 0.10
|
||||
# the whole window is treated as silence and left alone (we'd only be
|
||||
# snapping to noise).
|
||||
ABS_RMS_FLOOR = 1e-3
|
||||
# An onset must be *sustained* to count as speech (#963): within the
|
||||
# SUSTAIN_WINDOW_S that follows a candidate frame, at least SUSTAIN_MIN_S
|
||||
# worth of frames must also sit above the threshold. A ~100 ms footstep
|
||||
# burst fails this; real speech (syllables every few hundred ms) passes.
|
||||
SUSTAIN_WINDOW_S = 0.30
|
||||
SUSTAIN_MIN_S = 0.16
|
||||
# Snaps larger than this are only trusted when the skipped span is
|
||||
# (near-)silence — see _region_mostly_silent (#963).
|
||||
MAX_SNAP_S = 1.5
|
||||
# The skipped span counts as "mostly silent" when at most this fraction of
|
||||
# its frames is audible. Non-zero so an isolated transient bleeding through
|
||||
# separation (a footstep) doesn't block a genuine long silence-trim…
|
||||
SKIPPED_AUDIBLE_FRAC = 0.10
|
||||
# …where "audible" = above max(ABS_RMS_FLOOR, this fraction of the span's
|
||||
# own peak); the relative term keeps a slightly raised residual noise floor
|
||||
# from reading as content.
|
||||
SKIPPED_FLOOR_PEAK_FRAC = 0.02
|
||||
|
||||
|
||||
def _frame_rms(x: np.ndarray, frame_len: int) -> np.ndarray:
|
||||
@@ -70,6 +108,15 @@ def detect_speech_onset(
|
||||
) -> float | None:
|
||||
"""Return the absolute time (s) of the first speech-like frame inside
|
||||
``[start_s, end_s]``, or ``None`` when the window is empty / silent.
|
||||
|
||||
"Speech-like" requires *sustained* energy (#963): within the
|
||||
``SUSTAIN_WINDOW_S`` look-ahead after a candidate frame, at least
|
||||
``SUSTAIN_MIN_S`` worth of frames must also exceed the threshold.
|
||||
Short broadband transients — footsteps, door thuds, mouse clicks —
|
||||
light up one or two 20 ms frames and then die, so they no longer read
|
||||
as "the conversation started here"; real speech keeps the energy up
|
||||
across syllables. A candidate too close to the window's end to prove
|
||||
sustain is rejected (conservative: the ASR timestamp stands).
|
||||
"""
|
||||
if sr <= 0 or end_s <= start_s:
|
||||
return None
|
||||
@@ -86,10 +133,21 @@ def detect_speech_onset(
|
||||
if peak < ABS_RMS_FLOOR:
|
||||
return None # whole window is effectively silent
|
||||
threshold = max(RELATIVE_THRESHOLD * peak, ABS_RMS_FLOOR)
|
||||
above = np.nonzero(rms >= threshold)[0]
|
||||
if above.size == 0:
|
||||
above = rms >= threshold
|
||||
candidates = np.nonzero(above)[0]
|
||||
if candidates.size == 0:
|
||||
return None
|
||||
return start_s + float(above[0]) * (frame_len / sr)
|
||||
frame_s = frame_len / sr
|
||||
win_frames = max(1, int(round(SUSTAIN_WINDOW_S / frame_s)))
|
||||
need_frames = max(1, int(round(SUSTAIN_MIN_S / frame_s)))
|
||||
# counts[k] = above-threshold frames within rms[c : c + win_frames]
|
||||
# for candidate c — O(n) via a cumulative sum, no per-candidate scan.
|
||||
cum = np.concatenate(([0], np.cumsum(above)))
|
||||
counts = cum[np.minimum(candidates + win_frames, above.size)] - cum[candidates]
|
||||
sustained = candidates[counts >= need_frames]
|
||||
if sustained.size == 0:
|
||||
return None # only transient bursts in this window
|
||||
return start_s + float(sustained[0]) * frame_s
|
||||
|
||||
|
||||
# Hysteresis for full-track onset listing: after a frame crosses the
|
||||
@@ -138,21 +196,60 @@ def detect_speech_onsets(audio: np.ndarray, sr: int) -> list[float]:
|
||||
return onsets
|
||||
|
||||
|
||||
def _region_mostly_silent(
|
||||
audio: np.ndarray,
|
||||
sr: int,
|
||||
start_s: float,
|
||||
end_s: float,
|
||||
) -> bool:
|
||||
"""True when ``[start_s, end_s]`` contains (almost) no audible content.
|
||||
|
||||
Gates long snaps (> ``MAX_SNAP_S``, #963): jumping far forward is only
|
||||
trustworthy when everything being skipped is silence — the genuine
|
||||
whisper start-stretch of #280, where Demucs stripped the leading music
|
||||
and left real silence on the vocals track. A small fraction of audible
|
||||
frames is tolerated so an isolated transient bleeding through
|
||||
separation (a footstep) doesn't block the trim; *sustained* audible
|
||||
content — e.g. quiet speech sitting below the relative onset
|
||||
threshold — does block it, because skipping past it would desync the
|
||||
dub by the full jump.
|
||||
"""
|
||||
i0 = max(0, int(start_s * sr))
|
||||
i1 = min(len(audio), int(end_s * sr))
|
||||
if i1 <= i0:
|
||||
return True
|
||||
rms = _frame_rms(audio[i0:i1], max(1, int(FRAME_S * sr)))
|
||||
if rms.size == 0:
|
||||
return True
|
||||
floor = max(ABS_RMS_FLOOR, SKIPPED_FLOOR_PEAK_FRAC * float(rms.max()))
|
||||
return float((rms >= floor).mean()) <= SKIPPED_AUDIBLE_FRAC
|
||||
|
||||
|
||||
def snap_segment_starts(
|
||||
segments: Sequence[dict],
|
||||
audio: np.ndarray,
|
||||
sr: int,
|
||||
*,
|
||||
min_shift_s: float = MIN_SHIFT_S,
|
||||
separated_vocals: bool = True,
|
||||
) -> int:
|
||||
"""Snap each segment's ``start`` forward to the actual speech onset.
|
||||
|
||||
Mutates the segment dicts in place (the shape the dub pipeline passes
|
||||
around). Returns the number of segments adjusted.
|
||||
|
||||
``audio`` should be mono float; the Demucs vocals track gives the best
|
||||
signal but the mixed track still beats nothing.
|
||||
``audio`` should be the mono-float **separated vocals** track. When the
|
||||
caller only has mixed/original audio (Demucs skipped or failed), pass
|
||||
``separated_vocals=False``: snapping is then disabled entirely (#963) —
|
||||
on a mixed track music, ambience and footsteps are all sustained energy,
|
||||
so a detected "onset" is as likely the score as the speaker, and
|
||||
whisper's own timestamps beat a confidently wrong snap.
|
||||
"""
|
||||
if not separated_vocals:
|
||||
logger.info(
|
||||
"onset-align: skipped — audio is not a separated vocals track "
|
||||
"(Demucs unavailable/failed); keeping ASR timestamps as-is")
|
||||
return 0
|
||||
if sr <= 0 or audio is None or len(audio) == 0:
|
||||
return 0
|
||||
if audio.ndim > 1:
|
||||
@@ -174,6 +271,13 @@ def snap_segment_starts(
|
||||
shift = new_start - start
|
||||
if shift < min_shift_s:
|
||||
continue
|
||||
if shift > MAX_SNAP_S and not _region_mostly_silent(audio, sr, start, onset):
|
||||
# Long jump over audible content (#963): the "onset" is more
|
||||
# likely a louder late event than the true start — quiet speech
|
||||
# under the relative threshold would be skipped wholesale and
|
||||
# the dub would play seconds LATE. Bounded corrections are fine;
|
||||
# unbounded ones only over true silence (the #280 case).
|
||||
continue
|
||||
# Preserve a minimum playable duration.
|
||||
new_start = min(new_start, end - MIN_SEG_DUR_S)
|
||||
if new_start - start < min_shift_s:
|
||||
|
||||
@@ -223,6 +223,60 @@ def extract_segment_refs(
|
||||
return out
|
||||
|
||||
|
||||
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
|
||||
"""Re-transcribe a written reference clip and return that transcript.
|
||||
|
||||
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
|
||||
the ASR segment's OWN text field, on the assumption that the segment's
|
||||
timestamps and its transcribed text agree. They routinely don't — Whisper
|
||||
(and friends) frequently drift on segment boundaries: a trailing word
|
||||
audible in `[start, end]` but missing from `text`, or vice versa. When the
|
||||
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
|
||||
down and the clone can speak the mismatched reference text itself instead
|
||||
of the target-language text it was given to synthesize (issue #1004).
|
||||
|
||||
Re-transcribing the *actual written clip* guarantees the pair matches by
|
||||
construction — the model doesn't care whether the original ASR text was
|
||||
right, only that ref_text is what's really in ref_audio. `asr_backend` is
|
||||
the caller's already-loaded active backend (duck-typed:
|
||||
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
|
||||
`{"text": ...}`); the model is already warm, so this costs one more short
|
||||
transcribe call, not a fresh load. Falls back to `fallback_text` — never
|
||||
raises — so a re-transcribe failure is a strict no-op, never a regression
|
||||
from the original (matching) behavior.
|
||||
"""
|
||||
if asr_backend is None:
|
||||
return fallback_text
|
||||
try:
|
||||
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
|
||||
text = " ".join(
|
||||
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
|
||||
).strip()
|
||||
return text or fallback_text
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
|
||||
ref_audio_path, e,
|
||||
)
|
||||
return fallback_text
|
||||
|
||||
|
||||
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
|
||||
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
|
||||
|
||||
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
|
||||
or per-segment `seg_clones` from `extract_segment_refs`) into the single
|
||||
executor round-trip the caller submits to the GPU pool, rather than one
|
||||
dispatch per reference. Mutates and returns `clones` for a convenient
|
||||
call-and-reassign at the call site.
|
||||
"""
|
||||
for entry in clones.values():
|
||||
entry["ref_text"] = refine_ref_text(
|
||||
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
|
||||
)
|
||||
return clones
|
||||
|
||||
|
||||
# ── Internals ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -143,12 +143,19 @@ class TTSBackend(ABC):
|
||||
supports_voice_design: bool = False
|
||||
|
||||
#: Whether this engine already emits mastered, studio-grade audio and should
|
||||
#: therefore skip the shared apply_mastering() chain (Compressor + Reverb,
|
||||
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
|
||||
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
|
||||
#: 48 kHz) set this True so their clean output isn't pumped/reverbed. Loudness
|
||||
#: 48 kHz) set this True so their clean output isn't pumped. Loudness
|
||||
#: normalisation is applied regardless — it's a benign peak scale.
|
||||
applies_own_mastering: bool = False
|
||||
|
||||
#: Whether this engine can clone an arbitrary voice from reference audio
|
||||
#: (`ref_audio=`), as opposed to only offering a fixed set of preset
|
||||
#: voices. Default True — most engines clone. Dub/batch gate on this
|
||||
#: (issue #312 class) before committing to a job that needs it, instead
|
||||
#: of silently falling back to OmniVoice or mis-cloning per segment.
|
||||
supports_cloning: bool = True
|
||||
|
||||
#: GPU/accelerator targets the engine can run on. Surfaced via the
|
||||
#: Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) so users can
|
||||
#: tell at a glance which engines will use their hardware. Defaults to
|
||||
@@ -603,6 +610,7 @@ class KittenTTSBackend(TTSBackend):
|
||||
display_name = "KittenTTS (English, 8 preset voices, CPU realtime)"
|
||||
# KittenTTS ships as an ONNX CPU graph; no CUDA/MPS path today.
|
||||
gpu_compat = ("cpu",)
|
||||
supports_cloning = False # fixed preset voices only; ref_audio is ignored
|
||||
|
||||
PRESET_VOICES = [
|
||||
"expr-voice-2-m", "expr-voice-2-f",
|
||||
@@ -683,6 +691,54 @@ class KittenTTSBackend(TTSBackend):
|
||||
# ── MLX-Audio (mac-ARM engine multiplexer) ──────────────────────────────────
|
||||
|
||||
|
||||
# #977: Kokoro's own ALIASES table (mlx_audio.tts.models.kokoro.pipeline) only
|
||||
# recognizes ISO-ish tokens ("en", "es", "fr-fr", "pt-br", …) — it has no idea
|
||||
# what a full language name is. OmniVoice's `language` kwarg is normally a
|
||||
# full display name from frontend/src/languages.json (e.g. "Dutch",
|
||||
# "Spanish"), forwarded verbatim by the frontend and by
|
||||
# `OmniVoiceBackend.generate()`. Translate the subset Kokoro actually
|
||||
# supports to the ISO token its own ALIASES expects; a caller that already
|
||||
# passes an ISO code (or one of Kokoro's own single-letter codes) is
|
||||
# resolved unchanged by `resolve_kokoro_lang_code()` below.
|
||||
_KOKORO_ISO_BY_FULL_NAME = {
|
||||
"english": "en",
|
||||
"spanish": "es",
|
||||
"french": "fr",
|
||||
"hindi": "hi",
|
||||
"italian": "it",
|
||||
"portuguese": "pt",
|
||||
"japanese": "ja",
|
||||
"chinese": "zh",
|
||||
}
|
||||
|
||||
|
||||
def resolve_kokoro_lang_code(language: str) -> str:
|
||||
"""Map a full language name / ISO code to Kokoro's single-letter
|
||||
`lang_code`, against the AUTHORITATIVE table read from the installed
|
||||
mlx-audio package (never a hardcoded guess — the vendored table is the
|
||||
only source of truth and can change across mlx-audio versions).
|
||||
|
||||
Raises ``ValueError`` for anything Kokoro doesn't support, naming what
|
||||
it *does* support — instead of forwarding a bogus code into Kokoro's
|
||||
`assert lang_code in LANG_CODES`, which crashes with an unreadable
|
||||
``(lang_code, LANG_CODES)`` tuple/dict repr (#977).
|
||||
"""
|
||||
from mlx_audio.tts.models.kokoro.pipeline import ALIASES, LANG_CODES
|
||||
|
||||
key = language.strip().lower()
|
||||
iso = _KOKORO_ISO_BY_FULL_NAME.get(key, key)
|
||||
code = ALIASES.get(iso, iso)
|
||||
if code not in LANG_CODES:
|
||||
supported = ", ".join(sorted(name.title() for name in _KOKORO_ISO_BY_FULL_NAME))
|
||||
raise ValueError(
|
||||
f"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) doesn't "
|
||||
f"support language={language!r}. Kokoro supports: {supported}. "
|
||||
f"Pick one of those, leave language as 'Auto', or switch to a "
|
||||
f"multilingual engine (e.g. OmniVoice) for other languages."
|
||||
)
|
||||
return code
|
||||
|
||||
|
||||
class MLXAudioBackend(TTSBackend):
|
||||
"""Blaizzy/mlx-audio — Apple-Silicon-only wrapper over 14+ TTS engines
|
||||
(Kokoro, CSM, Dia, Qwen3-TTS, Chatterbox, MeloTTS, OuteTTS, Spark,
|
||||
@@ -721,7 +777,16 @@ class MLXAudioBackend(TTSBackend):
|
||||
def __init__(self):
|
||||
self._model = None
|
||||
self._sr = 24000 # most mlx-audio engines emit 24 kHz mono
|
||||
key = os.environ.get("OMNIVOICE_MLX_AUDIO_MODEL", self.DEFAULT_MODEL_KEY)
|
||||
# Env var > persisted UI choice (#981 — Settings → Engines curated-
|
||||
# model picker) > default. Mirrors active_backend_id()'s resolution
|
||||
# order exactly so power-users can still pin a model without the UI
|
||||
# silently undoing it.
|
||||
from core import prefs
|
||||
key = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=self.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
# Accept either a curated key ("kokoro") or a full HF repo id
|
||||
# ("mlx-community/Kokoro-82M-bf16") — flexibility for power users.
|
||||
self._model_id = self.CURATED_MODELS.get(key, key)
|
||||
@@ -759,6 +824,18 @@ class MLXAudioBackend(TTSBackend):
|
||||
# silently ignores languages it doesn't know.
|
||||
return ["multi"]
|
||||
|
||||
@property
|
||||
def supports_cloning(self) -> bool:
|
||||
"""Model-dependent — this adapter multiplexes 7+ curated models and
|
||||
only some take a reference-audio speaker prompt. `generate()` passes
|
||||
`ref_audio` through when present (~kwargs below) but silently retries
|
||||
without it on a TypeError, so an engine picked for cloning that's
|
||||
actually running Kokoro/Qwen3-TTS/etc. would clone nothing. Of the
|
||||
curated set, only CSM (`mlx-community/csm-1b-8bit`) is confirmed to
|
||||
accept a reference prompt — default False for every other model,
|
||||
curated or user-supplied, until positively confirmed."""
|
||||
return self._model_id == self.CURATED_MODELS.get("csm")
|
||||
|
||||
def _ensure_loaded(self):
|
||||
if self._model is not None:
|
||||
return
|
||||
@@ -772,6 +849,7 @@ class MLXAudioBackend(TTSBackend):
|
||||
|
||||
voice = kw.get("voice")
|
||||
ref_audio = kw.get("ref_audio")
|
||||
ref_text = kw.get("ref_text")
|
||||
language = kw.get("language")
|
||||
speed = float(kw.get("speed", 1.0))
|
||||
|
||||
@@ -782,7 +860,31 @@ class MLXAudioBackend(TTSBackend):
|
||||
kwargs = {"text": text, "speed": speed}
|
||||
if voice: kwargs["voice"] = voice
|
||||
if ref_audio: kwargs["ref_audio"] = ref_audio
|
||||
if language: kwargs["lang_code"] = language[:2].lower()
|
||||
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
|
||||
# AND ref_text are present — with ref_text missing, its context list
|
||||
# stays empty and indexing into it raises an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio,
|
||||
# instead of ever attempting the clone. Community-diagnosed (#1012).
|
||||
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
|
||||
if language and language != "Auto":
|
||||
if self._model_id == self.CURATED_MODELS.get("kokoro"):
|
||||
# Kokoro's vendored pipeline hard-asserts `lang_code` against
|
||||
# its own single-letter table — a bogus code crashes with an
|
||||
# unreadable AssertionError instead of failing cleanly
|
||||
# (#977). Resolve against the authoritative installed table
|
||||
# instead of guessing via `language[:2]`.
|
||||
kwargs["lang_code"] = resolve_kokoro_lang_code(language)
|
||||
else:
|
||||
# `lang_code`-as-2-letter-truncation is Kokoro's own
|
||||
# convention, not mlx-audio's in general — other curated
|
||||
# models either ignore unrecognized kwargs (CSM/Dia/OuteTTS
|
||||
# accept **kwargs and drop it) or expect something else
|
||||
# entirely (Qwen3-TTS's own docstring: "lang_code: Language
|
||||
# code (auto, chinese, english, etc.)" — a full name, not a
|
||||
# 2-letter code). Kokoro's strict validation doesn't apply to
|
||||
# them, so don't reject a language that's valid for whatever
|
||||
# model is actually active.
|
||||
kwargs["lang_code"] = language[:2].lower()
|
||||
|
||||
pieces = []
|
||||
try:
|
||||
@@ -1111,6 +1213,7 @@ class SherpaOnnxBackend(TTSBackend):
|
||||
# Sherpa-ONNX uses the onnxruntime providers — CPU is the universal
|
||||
# baseline; CUDA provider is available on Linux/Windows installs.
|
||||
gpu_compat = ("cuda", "cpu")
|
||||
supports_cloning = False # VITS speaker-id only; no ref_audio support
|
||||
|
||||
def __init__(self):
|
||||
self._tts = None
|
||||
@@ -1366,6 +1469,21 @@ _SETUP_SNIPPETS: dict[str, str] = {
|
||||
}
|
||||
|
||||
|
||||
# Short, readable labels for mlx-audio's curated models (#981) — surfaced in
|
||||
# the Settings → Engines model picker so users see more than a bare key.
|
||||
# Single-sourced here rather than on MLXAudioBackend.CURATED_MODELS itself so
|
||||
# the class dict stays a plain key → repo-id map (what __init__ needs).
|
||||
_MLX_AUDIO_MODEL_LABELS: dict[str, str] = {
|
||||
"kokoro": "Kokoro (default, fast)",
|
||||
"csm": "CSM (voice cloning)",
|
||||
"qwen3-tts": "Qwen3-TTS (voice design)",
|
||||
"dia": "Dia",
|
||||
"chatterbox": "Chatterbox",
|
||||
"melotts": "MeloTTS (lightweight)",
|
||||
"outetts": "OuteTTS",
|
||||
}
|
||||
|
||||
|
||||
def list_backends() -> list[dict]:
|
||||
"""Enumerate every registered backend with its availability state.
|
||||
|
||||
@@ -1449,6 +1567,22 @@ def list_backends() -> list[dict]:
|
||||
# effective_device / routing_status / routing_reason (scrubbed):
|
||||
**routing_fields(gpu_compat, caps),
|
||||
})
|
||||
# #981: mlx-audio multiplexes 7+ curated models behind one backend id
|
||||
# — surface the roster + the currently-active pick so Settings can
|
||||
# render a model picker instead of always defaulting to Kokoro.
|
||||
# mlx-audio ONLY; every other backend loads a single fixed model.
|
||||
if bid == "mlx-audio":
|
||||
from core import prefs
|
||||
active_model = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=cls.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
out[-1]["curated_models"] = [
|
||||
{"key": key, "label": _MLX_AUDIO_MODEL_LABELS.get(key, key), "repo_id": repo_id}
|
||||
for key, repo_id in cls.CURATED_MODELS.items()
|
||||
]
|
||||
out[-1]["active_model_id"] = active_model
|
||||
return out
|
||||
|
||||
|
||||
@@ -1458,6 +1592,29 @@ def get_backend_class(backend_id: str) -> type[TTSBackend]:
|
||||
return _REGISTRY[backend_id]
|
||||
|
||||
|
||||
def cloning_capable_engine_ids() -> list[str]:
|
||||
"""Engine ids that support reference-audio voice cloning — used to build
|
||||
an actionable error when the active engine can't (dub/batch gating).
|
||||
|
||||
Iterates the same registry ``list_backends()`` uses, via ``.items()`` so
|
||||
lazy entries resolve through ``_LazyRegistry``'s snapshot-safe iteration
|
||||
(see ``_LazyRegistry.__iter__``) exactly like every other registry scan
|
||||
in this module.
|
||||
|
||||
A class-level ``getattr`` on a *property* returns the descriptor object
|
||||
itself (always truthy) rather than its computed value — so a
|
||||
model-dependent adapter like ``MLXAudioBackend`` (only some of its 7+
|
||||
curated models can clone) would always show up here regardless of which
|
||||
model is actually configured. Excluded rather than falsely recommended:
|
||||
``isinstance(..., bool)`` is False for a descriptor, True for a plain
|
||||
class attribute.
|
||||
"""
|
||||
return [
|
||||
bid for bid, cls in _REGISTRY.items()
|
||||
if isinstance((v := getattr(cls, "supports_cloning", True)), bool) and v
|
||||
]
|
||||
|
||||
|
||||
def active_routing() -> dict | None:
|
||||
"""Routing verdict for the currently-active TTS engine, or ``None`` if it
|
||||
can't be determined (no engine / probe failure).
|
||||
@@ -1526,15 +1683,21 @@ def active_backend_id() -> str:
|
||||
# call the outgoing engine's unload() before switching.
|
||||
_active_instance: "TTSBackend | None" = None
|
||||
_active_instance_id: "str | None" = None
|
||||
# mlx-audio multiplexes 7+ curated models behind one backend id — a model-only
|
||||
# switch (same "mlx-audio" id, different curated model) must also invalidate
|
||||
# the cache, or picking a different model in Settings has no effect until the
|
||||
# app restarts (#981). Only meaningful when _active_instance_id == "mlx-audio".
|
||||
_active_mlx_model_key: "str | None" = None
|
||||
|
||||
|
||||
def reset_active_backend() -> None:
|
||||
"""Unload + clear the cached active backend. For app shutdown and tests.
|
||||
Idempotent and best-effort — a raising unload() never propagates."""
|
||||
global _active_instance, _active_instance_id
|
||||
global _active_instance, _active_instance_id, _active_mlx_model_key
|
||||
inst = _active_instance
|
||||
_active_instance = None
|
||||
_active_instance_id = None
|
||||
_active_mlx_model_key = None
|
||||
if inst is not None:
|
||||
try:
|
||||
inst.unload()
|
||||
@@ -1552,13 +1715,32 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
``model=`` (caller already holds a loaded model), we return a fresh view
|
||||
over the shared singleton rather than caching it — but a switch *away from*
|
||||
a different engine still triggers that engine's unload().
|
||||
|
||||
For mlx-audio specifically, the backend id alone doesn't capture *which*
|
||||
curated model is loaded (#981) — so we also track the resolved model key
|
||||
and treat a model-only change as a switch, reusing the exact same
|
||||
unload-and-reconstruct path as an id switch.
|
||||
"""
|
||||
global _active_instance, _active_instance_id
|
||||
global _active_instance, _active_instance_id, _active_mlx_model_key
|
||||
bid = active_backend_id()
|
||||
|
||||
# Switching engines: release the outgoing one first. Best-effort so a bad
|
||||
# unload() can never block the switch.
|
||||
if _active_instance is not None and _active_instance_id != bid:
|
||||
mlx_model_key = None
|
||||
if bid == "mlx-audio":
|
||||
from core import prefs
|
||||
mlx_model_key = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=MLXAudioBackend.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
|
||||
# Switching engines (or, for mlx-audio, switching curated models): release
|
||||
# the outgoing one first. Best-effort so a bad unload() can never block
|
||||
# the switch.
|
||||
switching = _active_instance is not None and (
|
||||
_active_instance_id != bid
|
||||
or (bid == "mlx-audio" and mlx_model_key != _active_mlx_model_key)
|
||||
)
|
||||
if switching:
|
||||
try:
|
||||
_active_instance.unload()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
@@ -1566,6 +1748,7 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
type(_active_instance).__name__, exc)
|
||||
_active_instance = None
|
||||
_active_instance_id = None
|
||||
_active_mlx_model_key = None
|
||||
|
||||
cls = get_backend_class(bid)
|
||||
if cls is OmniVoiceBackend and model is not None:
|
||||
@@ -1577,9 +1760,82 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
if _active_instance is None or _active_instance_id != bid:
|
||||
_active_instance = OmniVoiceBackend(model=model) if cls is OmniVoiceBackend else cls()
|
||||
_active_instance_id = bid
|
||||
_active_mlx_model_key = mlx_model_key
|
||||
return _active_instance
|
||||
|
||||
|
||||
# ── Shared generation-time engine resolution (issue #312 class) ───────────
|
||||
#
|
||||
# dub_generate.py and batch.py used to call services.model_manager.get_model()
|
||||
# directly, hardcoding OmniVoice regardless of the engine selected in
|
||||
# Settings → Engines — a SILENT fallback: pick VoxCPM2, dub anyway with
|
||||
# OmniVoice, no error. This is the single resolution path both routers now
|
||||
# call instead, mirroring generation.py's /generate resolution (engine id →
|
||||
# is_available() → routing gate) plus a voice-cloning capability gate that
|
||||
# /generate doesn't need (OmniVoice's native path always clones).
|
||||
|
||||
|
||||
async def resolve_generation_backend(
|
||||
*, require_cloning: bool = False, cloning_purpose: str = "dubbing",
|
||||
) -> TTSBackend:
|
||||
"""Resolve + validate the active TTS engine for a generation call.
|
||||
|
||||
Returns the live backend instance (:func:`get_active_tts_backend`) —
|
||||
cached, and properly unload()ed on an engine switch. Raises ``ValueError``
|
||||
with an actionable message (never silently falls back to OmniVoice) when:
|
||||
|
||||
* the configured engine id is unknown (bad env var / stale pref),
|
||||
* the engine reports itself unavailable (``is_available()``),
|
||||
* the engine needs an accelerator this host lacks and has no CPU path
|
||||
(``routing_status == "unavailable"``),
|
||||
* ``require_cloning`` is True and the resolved backend can't clone
|
||||
from reference audio (``supports_cloning`` False) — checked on the
|
||||
live *instance*, not the class, so a model-dependent adapter like
|
||||
MLX-Audio (Kokoro vs. CSM) is judged by what's actually loaded.
|
||||
"""
|
||||
engine_id = active_backend_id()
|
||||
try:
|
||||
backend_cls = get_backend_class(engine_id)
|
||||
except ValueError as e:
|
||||
raise ValueError(
|
||||
f"Active TTS engine '{engine_id}' is not a recognized backend ({e}). "
|
||||
"Check Settings → Engines or the OMNIVOICE_TTS_BACKEND env var."
|
||||
) from e
|
||||
|
||||
try:
|
||||
ok, msg = backend_cls.is_available()
|
||||
except Exception as exc: # noqa: BLE001 — surface as an actionable ValueError
|
||||
ok, msg = False, f"{type(exc).__name__}: {exc}"
|
||||
if not ok:
|
||||
raise ValueError(f"TTS engine '{engine_id}' is not available: {_mask_hf_tokens(msg)}")
|
||||
|
||||
from core.device_caps import detect_host_caps
|
||||
from services.engine_routing import resolve_routing
|
||||
routing = resolve_routing(getattr(backend_cls, "gpu_compat", ("cpu",)), detect_host_caps())
|
||||
if routing["routing_status"] == "unavailable":
|
||||
raise ValueError(routing["routing_reason"])
|
||||
|
||||
_model = None
|
||||
if backend_cls is OmniVoiceBackend:
|
||||
# OmniVoice needs its model pre-loaded before construction: called
|
||||
# from an async context, OmniVoiceBackend._ensure_loaded() refuses to
|
||||
# bootstrap its own event loop (see its docstring) — same reason
|
||||
# generation.py's /generate special-cases this backend.
|
||||
from services.model_manager import get_model
|
||||
_model = await get_model()
|
||||
backend = get_active_tts_backend(model=_model)
|
||||
|
||||
if require_cloning and not getattr(backend, "supports_cloning", True):
|
||||
raise ValueError(
|
||||
f"The active TTS engine '{engine_id}' doesn't support voice cloning, "
|
||||
f"so {cloning_purpose} can't preserve speaker voices. Switch to one "
|
||||
f"of: {', '.join(cloning_capable_engine_ids())} in Settings → "
|
||||
"Engines, or use OmniVoice for this job."
|
||||
)
|
||||
|
||||
return backend
|
||||
|
||||
|
||||
# ── PEP 562 lazy attribute re-export ───────────────────────────────────────
|
||||
#
|
||||
# Allows ``from services.tts_backend import IndexTTS2Backend`` to keep
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
},
|
||||
"frontend": {
|
||||
"name": "omnivoice-studio",
|
||||
"version": "0.3.10",
|
||||
"version": "0.3.11",
|
||||
"dependencies": {
|
||||
"@fontsource-variable/inter": "^5.2.8",
|
||||
"@fontsource-variable/source-serif-4": "^5.2.9",
|
||||
|
||||
@@ -82,8 +82,12 @@ If `huggingface.co` is slow or blocked, point the client at a mirror:
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
```
|
||||
|
||||
Set it as an environment variable (or in **Settings → environment**) before
|
||||
downloading. Caveats:
|
||||
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets
|
||||
included), or as an environment variable before launching. On first run, the
|
||||
setup wizard offers the same mirror quick-pick right on the system-check
|
||||
screen when the endpoint is unreachable — the network check is a warning, not
|
||||
a blocker, so an offline or firewalled machine can still finish setup once
|
||||
models are available (mirror, or manual download below). Caveats:
|
||||
|
||||
- A mirror serves the **classic** download path, **not Xet** — you lose
|
||||
chunk-dedup and Xet's parallel fetch, but you gain reachability. On the
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# OmniVoice Studio — OpenAI-Compatible Remote ASR
|
||||
|
||||
A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own
|
||||
Whisper API — today, without waiting on `transformers` to ship a direct
|
||||
Qwen3-ASR integration (tracked separately). Unlike every other ASR engine,
|
||||
this one runs no model locally: it's a pure network client that calls any
|
||||
server exposing an OpenAI-compatible `POST /v1/audio/transcriptions`
|
||||
endpoint.
|
||||
|
||||
## Setup
|
||||
|
||||
No install step — configure it directly:
|
||||
|
||||
1. Open **Settings → Models** and find **OpenAI-compatible ASR (remote
|
||||
server)**.
|
||||
2. Set **Server URL** to your server's base URL (e.g.
|
||||
`http://localhost:8000/v1` for a local Qwen3-ASR/FunASR server, or
|
||||
`https://api.openai.com/v1` for OpenAI's own API).
|
||||
3. Set **Model** to whatever your server expects (`whisper-1` for OpenAI's
|
||||
API; check your self-hosted server's docs otherwise).
|
||||
4. **API key** is optional — many self-hosted servers accept requests
|
||||
without one. Set it if your server requires auth, or if you're using
|
||||
OpenAI's own API.
|
||||
5. Activate the engine in **Settings → Engines** — click **Use** on
|
||||
**OpenAI-compatible ASR** in the ASR Engines table (the same picker TTS
|
||||
engines have). Power users can pin it instead by setting
|
||||
`OMNIVOICE_ASR_BACKEND=openai-compat-asr` before launching — the env var
|
||||
always wins over the Settings pick.
|
||||
|
||||
## Response format
|
||||
|
||||
The backend prefers `response_format=verbose_json` for real per-segment
|
||||
timestamps (OpenAI's API and most compatible servers support it) and falls
|
||||
back to plain text automatically if your server rejects that format. Neither
|
||||
path returns word-level timestamps — that's not part of this API.
|
||||
|
||||
## Privacy note
|
||||
|
||||
Unlike every other ASR engine in OmniVoice, audio sent through this backend
|
||||
leaves your machine — to whatever server you configured. If that's a
|
||||
self-hosted server on your own network, nothing leaves your control; if
|
||||
it's a third-party API (OpenAI's, or someone else's), review their data
|
||||
handling before sending anything sensitive.
|
||||
@@ -74,6 +74,8 @@ asr_engines:
|
||||
readme: FunASR
|
||||
- id: sherpa-onnx-asr
|
||||
readme: "**sherpa-onnx** (live dictation)"
|
||||
- id: openai-compat-asr
|
||||
readme: "**OpenAI-compatible** ⚠️ remote"
|
||||
|
||||
# Doc files that must exist (the install path users are sent to).
|
||||
docs:
|
||||
|
||||
@@ -56,6 +56,8 @@ Priority: `duration` > `speed`.
|
||||
| `preprocess_prompt` | bool | True | Whether to apply preprocessing to the voice-clone prompt audio (remove long silences in reference audio, add punctuation in the end of reference text). |
|
||||
| `postprocess_output` | bool | True | Apply post-processing to generated audio (remove long silences). |
|
||||
|
||||
> **Tip — reference-clip quality transfers.** Zero-shot cloning mirrors the acoustics of the reference clip, not just the voice: a clip recorded in an echoey room clones echoey. Record dry and close-mic for clean output. No effect preset adds reverb unless you choose one that declares it (Cinematic, Warm).
|
||||
|
||||
## Long-Form Generation
|
||||
|
||||
To support stable long-form speech generation with low VRAM consumption, the text is automatically split into smaller segments when the estimated duration of the generated speech exceeds `audio_chunk_duration`, with each segment producing approximately `audio_chunk_duration` seconds of audio. This approach allows the model to accept arbitrarily long text and generate arbitrarily long speech with near-constant VRAM consumption.
|
||||
|
||||
+109
-33
@@ -5,13 +5,32 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Using the AppImage
|
||||
|
||||
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
|
||||
a Tauri / WebKitGTK app.
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
|
||||
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
|
||||
- Optional: **yt-dlp** for downloading YouTube/video clips directly in the
|
||||
Voice Gallery and Dub tabs — `sudo apt install yt-dlp` (Debian/Ubuntu),
|
||||
`sudo dnf install yt-dlp` (Fedora), or `sudo pacman -S yt-dlp` (Arch).
|
||||
Without it those downloads fail; everything else works fine.
|
||||
|
||||
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
|
||||
by the app itself on first launch. No toolchain needed.
|
||||
|
||||
### Building from source
|
||||
|
||||
Everything above, plus the toolchain:
|
||||
|
||||
- **git** — `sudo apt install git` (Debian/Ubuntu), `sudo dnf install git` (Fedora), or `sudo pacman -S git` (Arch).
|
||||
- **curl** — usually preinstalled; used by the Bun and rustup install one-liners below.
|
||||
- **Python 3.11+** — typically `sudo apt install python3.11` on Debian/Ubuntu,
|
||||
`sudo dnf install python3.11` on Fedora, or already installed on Arch.
|
||||
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
|
||||
- **FFmpeg** — `sudo apt install ffmpeg` (Debian/Ubuntu), `sudo dnf install ffmpeg-free` (Fedora), or `sudo pacman -S ffmpeg` (Arch).
|
||||
- **Rust / Cargo** (required for building from source only) — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
|
||||
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or via your package manager (e.g., `sudo apt install rustc cargo`).
|
||||
If you use rustup, reopen the shell or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
|
||||
- **GTK/WebKit deps** for the Tauri shell:
|
||||
|
||||
@@ -58,12 +77,15 @@ No FUSE? Use `--appimage-extract-and-run`:
|
||||
./OmniVoice.Studio_*.AppImage --appimage-extract-and-run
|
||||
```
|
||||
|
||||
## Install (.deb)
|
||||
## .deb package
|
||||
|
||||
```bash
|
||||
sudo apt install ./OmniVoice.Studio_*.amd64.deb
|
||||
omnivoice-studio
|
||||
```
|
||||
Not currently published: `.deb` bundling is disabled in the release pipeline
|
||||
because of a `tauri-cli` bug (`Failed to create control scripts`) — see the
|
||||
comment in `.github/workflows/release.yml` for the tracking note. The
|
||||
AppImage above is the supported Linux install path until a `tauri-cli`
|
||||
version resolves it. `apt install`-able `.deb`s shipped before v0.3 (see
|
||||
[.deb ffprobe conflict](#deb-ffprobe-conflict) below) if you're upgrading
|
||||
from one of those.
|
||||
|
||||
The desktop app uses these canonical paths (kept in sync with
|
||||
`scripts/desktop-prod.sh` by the docs-drift CI gate):
|
||||
@@ -74,24 +96,49 @@ APP_ID="com.debpalash.omnivoice-studio"
|
||||
APP_NAME="OmniVoice Studio"
|
||||
```
|
||||
|
||||
## AppImage white-screen on Fedora 44 / Ubuntu 24.04
|
||||
## AppImage white screen / EGL errors (Fedora 44, Ubuntu 24.04+, 26.04)
|
||||
|
||||
<a id="appimage-white-screen-on-fedora-44--ubuntu-2404"></a>
|
||||
|
||||
Newer distros ship WebKitGTK 2.44 / 2.46, which has a compositing-mode
|
||||
regression that lands the Tauri window as a fully-white frame with no UI.
|
||||
Two separate WebKitGTK rendering issues land the Tauri window as a
|
||||
fully-white frame with no UI. Which one you have depends on your WebKitGTK
|
||||
version (`pkg-config --modversion webkit2gtk-4.1` prints it).
|
||||
|
||||
**Workaround:** set `WEBKIT_DISABLE_COMPOSITING_MODE=1` before launching:
|
||||
**Modern WebKitGTK (2.48+ — Ubuntu 24.04 and newer, incl. 26.04): try this
|
||||
first.** WebKit's DMA-BUF renderer fails against some GPU drivers; the
|
||||
terminal typically shows:
|
||||
|
||||
```
|
||||
Could not create default EGL display: EGL_BAD_PARAMETER
|
||||
```
|
||||
|
||||
Disable the DMA-BUF renderer before launching:
|
||||
|
||||
```bash
|
||||
WEBKIT_DISABLE_DMABUF_RENDERER=1 ./OmniVoice.Studio_*.AppImage
|
||||
```
|
||||
|
||||
**WebKitGTK 2.44 / 2.46 (Fedora 44, Ubuntu 24.04 at release):** a
|
||||
compositing-mode regression blanks the surface on first paint. Disable
|
||||
compositing mode instead:
|
||||
|
||||
```bash
|
||||
WEBKIT_DISABLE_COMPOSITING_MODE=1 ./OmniVoice.Studio_*.AppImage
|
||||
```
|
||||
|
||||
OmniVoice's AppRun launcher autodetects the broken WebKitGTK range and sets
|
||||
this for you (shipped in v0.3+). The manual env-var path remains the documented
|
||||
fallback when running from a checked-out source tree.
|
||||
OmniVoice's AppRun launcher autodetects the broken 2.44/2.46 range and sets
|
||||
this second variable for you (shipped in v0.3+). The manual env-var path
|
||||
remains the documented fallback when running from a checked-out source tree.
|
||||
|
||||
Tracking issue: [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62).
|
||||
**Last resort** — if neither variable alone helps, force software rendering
|
||||
(slower, but always paints):
|
||||
|
||||
```bash
|
||||
WEBKIT_DISABLE_DMABUF_RENDERER=1 LIBGL_ALWAYS_SOFTWARE=1 ./OmniVoice.Studio_*.AppImage
|
||||
```
|
||||
|
||||
Tracking issues: [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62),
|
||||
[#961](https://github.com/debpalash/OmniVoice-Studio/issues/961).
|
||||
|
||||
## .deb ffprobe conflict
|
||||
|
||||
@@ -134,36 +181,67 @@ that picks these defaults automatically; for v0.3 set them by hand.
|
||||
|
||||
<a id="amd-gpu-rocm"></a>
|
||||
|
||||
OmniVoice **auto-detects AMD GPUs** — `get_best_device()` returns the GPU when a
|
||||
ROCm build of PyTorch is installed (ROCm-built PyTorch reports through
|
||||
`torch.cuda.is_available()`), and OmniVoice auto-sets `HSA_OVERRIDE_GFX_VERSION`
|
||||
for consumer cards whose GFX ID isn't in the official ROCm support matrix. No
|
||||
code changes or flags are needed.
|
||||
ROCm support is **Linux-only and opt-in**. The **default install ships the
|
||||
CUDA build** of PyTorch (the `pytorch-cuda` index in `pyproject.toml`), so on
|
||||
an AMD-only machine `torch.cuda.is_available()` is `False` and OmniVoice runs
|
||||
on CPU until you opt into the ROCm variant. (On Windows there is no ROCm path
|
||||
at all — PyTorch publishes no Windows ROCm wheels; see
|
||||
[windows.md](windows.md#gpu-support).)
|
||||
|
||||
The catch: the **default install ships the CUDA build** of PyTorch (the
|
||||
`pytorch-cuda` index in `pyproject.toml`), so on an AMD-only machine
|
||||
`torch.cuda.is_available()` is `False` and OmniVoice falls back to CPU. To use
|
||||
your AMD GPU, replace torch with the ROCm wheel **after** the first-run install
|
||||
populates the venv:
|
||||
Three ways to opt in, in order of preference:
|
||||
|
||||
**1. First-run setup screen (recommended).** On Linux the setup screen's
|
||||
**Compute** card offers **"AMD GPU (ROCm, Linux)"** next to the default
|
||||
**Auto**. When OmniVoice detects an AMD GPU *and* the ROCm userspace
|
||||
(`/opt/rocm` present, or `rocminfo` on PATH), the ROCm option is pre-selected;
|
||||
with an AMD GPU but no ROCm runtime it stays offered-but-unselected — install
|
||||
ROCm first (or continue on CPU). Choosing ROCm makes the bootstrap reinstall
|
||||
`torch`/`torchaudio` from the ROCm wheel index
|
||||
(`https://download.pytorch.org/whl/rocm6.4` by default) right after the
|
||||
dependency sync — matched to the app's pinned `torch==2.8.0` (the rocm6.2
|
||||
index only ever published up to torch 2.5.1, so it silently failed the
|
||||
reinstall and left the CPU-only CUDA build in place).
|
||||
|
||||
**2. Environment variable (existing installs / headless).** Set
|
||||
`OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
|
||||
the same ROCm reinstall. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
|
||||
index when you need a different ROCm version — e.g. AMD publishes newer
|
||||
driver-matched builds (7.2.x) at `repo.radeon.com` as a `--find-links` page
|
||||
rather than a PyPI-style index:
|
||||
```bash
|
||||
uv pip install --reinstall torch==2.8.0 torchaudio==2.8.0 \
|
||||
--find-links https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2.4/
|
||||
```
|
||||
run that manually if you want a specific ROCm point release; the
|
||||
`OMNIVOICE_TORCH_INDEX` env var only accepts a PEP 503 index URL, not a
|
||||
find-links page. If the reinstall fails (network, unsupported card), OmniVoice
|
||||
keeps the default torch build and warns instead of breaking the install.
|
||||
|
||||
**3. Manual wheel swap (fallback).** Replace torch with the ROCm wheel
|
||||
**after** the first-run install populates the venv:
|
||||
|
||||
```bash
|
||||
# From the project directory (source install), into OmniVoice's uv venv.
|
||||
# Current stable is ROCm 6.2 — match your installed ROCm/driver version
|
||||
# (https://pytorch.org/get-started/locally/ lists available wheels).
|
||||
# Matches the app's torch==2.8.0 pin — a different ROCm point release
|
||||
# (e.g. rocm6.2, rocm7.x) may not carry that exact torch build.
|
||||
uv pip install --reinstall torch torchaudio \
|
||||
--index-url https://download.pytorch.org/whl/rocm6.2
|
||||
--index-url https://download.pytorch.org/whl/rocm6.4
|
||||
```
|
||||
|
||||
Then relaunch — the Settings → System panel should now report the GPU device
|
||||
instead of `cpu`. Verify the wheel sees your card:
|
||||
Once a ROCm build of PyTorch is in the venv, detection is automatic —
|
||||
`get_best_device()` returns the GPU (ROCm-built PyTorch reports through
|
||||
`torch.cuda.is_available()`), and OmniVoice auto-sets
|
||||
`HSA_OVERRIDE_GFX_VERSION` for consumer cards whose GFX ID isn't in the
|
||||
official ROCm support matrix. Relaunch and the Settings → System panel should
|
||||
report the GPU device instead of `cpu`. Verify the wheel sees your card:
|
||||
|
||||
```bash
|
||||
uv run python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
|
||||
```
|
||||
|
||||
Notes:
|
||||
- ROCm is **Linux-only** and **opt-in** — the default cross-platform behavior
|
||||
(CUDA on NVIDIA, MPS on Apple, CPU elsewhere) is unchanged.
|
||||
- ROCm is exercised far less than the default CUDA/MPS/CPU paths — it works,
|
||||
but expect rough edges on consumer cards and report what you hit.
|
||||
- Unsupported GFX (e.g. some consumer RDNA cards): if it still won't run, set
|
||||
`HSA_OVERRIDE_GFX_VERSION` yourself (e.g. `export HSA_OVERRIDE_GFX_VERSION=11.0.0`)
|
||||
to the nearest supported architecture before launching.
|
||||
@@ -171,8 +249,6 @@ Notes:
|
||||
native ROCm wheel.
|
||||
|
||||
Tracking issue: [#124](https://github.com/debpalash/OmniVoice-Studio/issues/124).
|
||||
An installer-integrated, env-var-driven ROCm wheel selection is a planned
|
||||
follow-up; until then this manual step is the supported path.
|
||||
|
||||
## Hugging Face token (optional but recommended)
|
||||
|
||||
|
||||
+15
-2
@@ -17,13 +17,26 @@ working OmniVoice Studio install on macOS (Apple Silicon).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Using the DMG
|
||||
|
||||
- **macOS 12 (Monterey) or newer** — Apple Silicon (Intel: UI only, see the
|
||||
note above).
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
|
||||
That's it — GPU acceleration (Apple MPS) is automatic on Apple Silicon, and
|
||||
Python, FFmpeg, and the model weights are bundled or bootstrapped by the app
|
||||
itself on first launch. No toolchain needed.
|
||||
|
||||
### Building from source
|
||||
|
||||
Everything above, plus the toolchain:
|
||||
|
||||
- **Xcode Command Line Tools** — `xcode-select --install` (includes **git**
|
||||
and the C toolchain; `curl` ships with macOS).
|
||||
- **Python 3.11+** — `brew install python@3.11` (or use `pyenv` / the system Python if you already have ≥3.11).
|
||||
- **Bun** — `curl -fsSL https://bun.sh/install | bash`.
|
||||
- **Xcode Command Line Tools** — `xcode-select --install`.
|
||||
- **FFmpeg** (used by the dubbing + capture pipelines) — `brew install ffmpeg`.
|
||||
- **Rust / Cargo** (required for building from source only) — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
|
||||
- **Rust / Cargo** — `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh` or `brew install rust`.
|
||||
If you use rustup, reopen the terminal or source `"$HOME/.cargo/env"` before running `bun run desktop-prod`.
|
||||
|
||||
Optional but recommended:
|
||||
|
||||
@@ -115,13 +115,22 @@ quarantines every download.
|
||||
|
||||
**Fix:** see [macos.md#gatekeeper-quarantine](macos.md#gatekeeper-quarantine).
|
||||
|
||||
## 4. AppImage white screen on Fedora 44 / Ubuntu 24.04
|
||||
## 4. AppImage white screen / EGL errors (Fedora 44, Ubuntu 24.04+, 26.04)
|
||||
|
||||
**Symptom:** the AppImage window opens fully white. No UI ever appears.
|
||||
**Symptom:** the AppImage window opens fully white. No UI ever appears. On
|
||||
newer distros (Ubuntu 24.04 and later, incl. 26.04) the terminal often shows
|
||||
`Could not create default EGL display: EGL_BAD_PARAMETER`.
|
||||
|
||||
**Cause:** WebKitGTK 2.44 / 2.46 compositing-mode regression.
|
||||
**Cause:** WebKitGTK rendering regressions — the DMA-BUF renderer on modern
|
||||
WebKitGTK (2.48+), or the 2.44 / 2.46 compositing mode.
|
||||
|
||||
**Fix:** see [linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404](linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404).
|
||||
**Fix:** try `WEBKIT_DISABLE_DMABUF_RENDERER=1` first (modern WebKitGTK / the
|
||||
EGL error), then `WEBKIT_DISABLE_COMPOSITING_MODE=1` — full walkthrough incl.
|
||||
the software-rendering last resort:
|
||||
[linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404](linux.md#appimage-white-screen-on-fedora-44--ubuntu-2404).
|
||||
|
||||
**Linked issues:** [#62](https://github.com/debpalash/OmniVoice-Studio/issues/62),
|
||||
[#961](https://github.com/debpalash/OmniVoice-Studio/issues/961)
|
||||
|
||||
## 5. Windows Triton / torch.compile OOM
|
||||
|
||||
@@ -233,6 +242,15 @@ for the dedicated CosyVoice path.
|
||||
|
||||
**Linked issue:** [#55](https://github.com/debpalash/OmniVoice-Studio/issues/55)
|
||||
|
||||
**Same class, ASR side:** the `nemo-parakeet` ASR engine has the identical
|
||||
problem and currently has **no safe install path** at all — `nemo_toolkit[asr]`
|
||||
hard-pins `transformers>=4.57,<4.58`, which is unsatisfiable alongside
|
||||
OmniVoice's own `transformers>=5.3` requirement. Installing it into the
|
||||
shared venv breaks the backend outright. Do not `pip install nemo_toolkit`
|
||||
into OmniVoice's environment; if you want to try it, use a separate Python
|
||||
environment. Isolated-venv support for this engine (matching CosyVoice/
|
||||
dots-tts) is tracked in [#974](https://github.com/debpalash/OmniVoice-Studio/issues/974).
|
||||
|
||||
## 12. CUDA PyTorch wheel download fails on first run
|
||||
|
||||
**Symptom:** first-run setup stops at **Installing dependencies** with a failure
|
||||
@@ -305,8 +323,10 @@ order:
|
||||
files are a common false-positive quarantine), then re-enable.
|
||||
- **Connection** — use a stable, direct connection; pause any VPN; avoid
|
||||
corporate/school networks.
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, set a
|
||||
mirror **before** launching and relaunch:
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, pick a
|
||||
mirror in-app (**Settings → Models → Hugging Face mirror**, or the quick-pick
|
||||
the first-run system check offers when the endpoint is unreachable), or set
|
||||
it as an env var before launching and relaunch:
|
||||
- macOS/Linux: `export HF_ENDPOINT=https://hf-mirror.com`
|
||||
- Windows (PowerShell): `[Environment]::SetEnvironmentVariable("HF_ENDPOINT","https://hf-mirror.com","User")`
|
||||
|
||||
@@ -382,6 +402,16 @@ overhead. It reuses your existing faster-whisper install (nothing extra to
|
||||
download). OmniVoice never switches engines automatically — this stays your
|
||||
call.
|
||||
|
||||
> **Seeing "The backend crashed (exit code …)" instead?** That's the other
|
||||
> failure mode: the backend **process died** (native CUDA abort, out-of-memory
|
||||
> kill, DLL crash) rather than hanging. Newer desktop builds detect the death,
|
||||
> restart the backend automatically (giving up after 3 crashes in 10 minutes),
|
||||
> and show a crash notice with a **View crash details** button (exit code +
|
||||
> the last error output). Use **Report this bug** from that notice — the crash
|
||||
> evidence is attached to the prefilled GitHub issue automatically, with home
|
||||
> paths scrubbed. The raw markers live next to the backend logs in
|
||||
> `backend_crash_markers.json`.
|
||||
|
||||
## 15. Stuck at "preparing" forever after a crash / BSOD (Windows)
|
||||
|
||||
**Symptom:** after an unclean shutdown (Windows BSOD, forced power-off), every
|
||||
@@ -415,6 +445,41 @@ quit OmniVoice Studio, delete the folder below, then start the app again.
|
||||
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\com.debpalash.omnivoice-studio\EBWebView"
|
||||
```
|
||||
|
||||
## 16. macOS: microphone permission never prompts, OmniVoice never appears in System Settings
|
||||
|
||||
**Symptom:** clicking record shows "Microphone access denied. macOS: open
|
||||
System Settings → Privacy & Security → Microphone and enable OmniVoice" —
|
||||
but OmniVoice never appears in that list, so there's nothing to enable.
|
||||
`NSMicrophoneUsageDescription` is present in the app's `Info.plist`, and
|
||||
resetting the permission (`tccutil reset Microphone
|
||||
com.debpalash.omnivoice-studio`) followed by a relaunch changes nothing — no
|
||||
system prompt ever appears.
|
||||
|
||||
**Cause:** the app bundle was missing the Hardened Runtime *entitlement* for
|
||||
microphone access. An earlier revision of this section blamed an upstream
|
||||
Tauri/WebKit limitation — that was wrong (a community contributor,
|
||||
[@MahdiHedhli](https://github.com/MahdiHedhli), read the sources more
|
||||
carefully and found the real gap). wry's `WKUIDelegate` already grants the
|
||||
WebKit-layer media-capture request; but Tauri's macOS bundler enables
|
||||
Hardened Runtime by default, and Hardened Runtime blocks microphone hardware
|
||||
access unless `com.apple.security.device.audio-input` is present in the
|
||||
signed binary's entitlements — regardless of `Info.plist`'s
|
||||
`NSMicrophoneUsageDescription` (that only supplies the prompt *text*).
|
||||
Without the entitlement, macOS's TCC layer never registers a request, which
|
||||
is exactly why the app never appears in the System Settings list.
|
||||
|
||||
**Fix:** ships in the release after v0.3.12 (the bundle now carries
|
||||
`src-tauri/entitlements.plist` — [#1016](https://github.com/debpalash/OmniVoice-Studio/pull/1016),
|
||||
contributed by the same person who diagnosed it). Update and live recording
|
||||
works, with a normal macOS permission prompt on first use.
|
||||
|
||||
**Workaround on older builds (≤ v0.3.12):** record your voice sample in any
|
||||
other app (Voice Memos, QuickTime, etc.) and upload the resulting file in
|
||||
OmniVoice instead of using live recording — upload-based cloning is
|
||||
unaffected and works normally.
|
||||
|
||||
**Linked issue:** [#1013](https://github.com/debpalash/OmniVoice-Studio/issues/1013)
|
||||
|
||||
## Dub: "translation engine needs the optional … package"
|
||||
|
||||
**Symptom:** in the Dub tab, translating fails with e.g. *"The 'google'
|
||||
|
||||
+55
-5
@@ -5,7 +5,24 @@ working OmniVoice Studio install on Windows 10 / 11 (x64).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Using the MSI installer
|
||||
|
||||
- **Windows 10 (21H2 or newer) or Windows 11**, x64.
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
- Optional: an **NVIDIA GPU + driver** for CUDA acceleration — see
|
||||
[GPU support on Windows](#gpu-support). AMD GPUs run CPU-only on Windows.
|
||||
|
||||
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
|
||||
by the app itself on first launch. No toolchain needed.
|
||||
|
||||
### Building from source
|
||||
|
||||
Everything above, plus the toolchain:
|
||||
|
||||
- **Git for Windows** — `winget install --id Git.Git -e`. Needed for
|
||||
`git clone`, and it includes **Git Bash**, which `bun run desktop-prod`
|
||||
uses to run its build-and-launch script. Without it, `desktop-prod` stops
|
||||
with an error telling you to install it.
|
||||
- **Python 3.11+** — `winget install Python.Python.3.11` (or download from
|
||||
[python.org](https://www.python.org/downloads/windows/)).
|
||||
- **Microsoft C++ Build Tools** — required by some PyPI source distributions
|
||||
@@ -14,13 +31,24 @@ working OmniVoice Studio install on Windows 10 / 11 (x64).
|
||||
with the **"Desktop development with C++"** workload checked.
|
||||
- **Bun** — `powershell -c "irm bun.sh/install.ps1 | iex"`.
|
||||
- **FFmpeg** — `winget install Gyan.FFmpeg`.
|
||||
- **Git for Windows** (from-source installs only) — `winget install --id Git.Git -e`.
|
||||
You need it for `git clone` anyway, and it includes **Git Bash**, which
|
||||
`bun run desktop-prod` uses to run its build-and-launch script. Without it,
|
||||
`desktop-prod` stops with an error telling you to install it.
|
||||
- **Rust / Cargo** (required for building from source only) — `winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
|
||||
- **Rust / Cargo** — `winget install Rust.Rustup` or download `rustup-init.exe` from [rustup.rs](https://rustup.rs/).
|
||||
After installing Rustup, close and reopen PowerShell before running `bun run desktop-prod`.
|
||||
|
||||
## GPU support on Windows
|
||||
|
||||
<a id="gpu-support"></a>
|
||||
|
||||
**GPU acceleration on Windows is NVIDIA/CUDA-only.** The Windows install
|
||||
ships the CUDA build of PyTorch; with an NVIDIA GPU and a regular NVIDIA
|
||||
driver it's picked up automatically (no CUDA Toolkit install needed).
|
||||
|
||||
**AMD GPUs — including Ryzen / Ryzen AI integrated Radeon graphics — run
|
||||
CPU-only on Windows.** ROCm is not supported on Windows: PyTorch publishes no
|
||||
Windows ROCm wheels, and OmniVoice's ROCm option is Linux-only. (The Ryzen AI
|
||||
NPU is likewise not used.) Everything still works on CPU, just slower. If you
|
||||
have an AMD GPU and want GPU acceleration, run OmniVoice on Linux instead —
|
||||
see [linux.md — AMD GPU (ROCm)](linux.md#amd-gpu-rocm).
|
||||
|
||||
## Install (from source)
|
||||
|
||||
Run from a regular (non-admin) PowerShell:
|
||||
@@ -49,6 +77,28 @@ Download the latest MSI from the
|
||||
run it, follow the wizard. The shortcut lands in the Start menu as
|
||||
**OmniVoice Studio**.
|
||||
|
||||
### Installing to a different drive
|
||||
|
||||
<a id="install-other-drive"></a>
|
||||
|
||||
The wizard's **directory picker** lets you install the app to any **local**
|
||||
drive (D:, E:, …). Two caveats:
|
||||
|
||||
- **Mapped network drives (Z: → a share) are not supported** — this is a
|
||||
Windows Installer limitation, not an OmniVoice bug: MSI custom actions run
|
||||
as a service account that doesn't see per-user drive mappings, so the
|
||||
install fails or rolls back. Install to a local drive instead.
|
||||
- The install location only moves the ~200 MB app itself. The big data
|
||||
(models, voices, projects — tens of GB) lives in the **data directory**,
|
||||
which you move independently: **Settings → Storage → Models directory**
|
||||
in-app, or `OMNIVOICE_DATA_DIR` / [Portable mode](#portable-install) for
|
||||
the whole data tree.
|
||||
|
||||
If an install to a local non-C: drive fails anyway, capture a log with
|
||||
`msiexec /i OmniVoice*.msi /L*V install.log` and
|
||||
[open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) with it
|
||||
— that log shows exactly which step rolled back.
|
||||
|
||||
## Portable install (Windows)
|
||||
|
||||
<a id="portable-install"></a>
|
||||
|
||||
@@ -6,7 +6,7 @@ Today the longform renderer (Audiobook + Stories) applies a **single-pass** `lou
|
||||
|
||||
Upgrade to **two-pass** `loudnorm`: a first **measure** pass (`print_format=json`, output to `-f null -`) parses the clip's `input_i / input_tp / input_lra / input_thresh / target_offset`, then a second **apply** pass feeds those measured values back as `measured_*` + `offset` + `linear=true`. This lands the output accurately on the preset target. The change is a **runner enhancement** layered over the existing pure builders — the pure `build_loudnorm_filter()` and `LOUDNESS_PRESETS` stay; we add a measure-filter builder, a measured-apply-filter builder, a JSON parser, a measure-cmd argv builder, and an async two-pass orchestrator that runs in `_render_longform_sse` (`backend/api/routers/audiobook.py:345`) between the chapter renders and the final mux. Loudness stays **opt-in** (`loudness: None` default on both `AudiobookRequest` `:151` and `LongformRenderRequest` `:510`), so default cross-platform behavior is unchanged.
|
||||
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard EQ/Compressor/Reverb chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard highpass/Compressor chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omnivoice-studio",
|
||||
"version": "0.3.10",
|
||||
"version": "0.3.14",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"type": "module",
|
||||
|
||||
Generated
+2
-1
@@ -2941,7 +2941,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.10"
|
||||
version = "0.3.14"
|
||||
dependencies = [
|
||||
"arboard",
|
||||
"dirs-next",
|
||||
@@ -2969,6 +2969,7 @@ dependencies = [
|
||||
"walkdir",
|
||||
"webkit2gtk",
|
||||
"webview2-com",
|
||||
"windows 0.61.3",
|
||||
"windows-core 0.61.2",
|
||||
"zip 2.4.2",
|
||||
]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.10"
|
||||
version = "0.3.14"
|
||||
description = "OmniVoice Studio – AI voice cloning & dubbing desktop app"
|
||||
authors = ["Debpalash"]
|
||||
license = "AGPL-3.0-only"
|
||||
@@ -71,6 +71,13 @@ zip = { version = "2", default-features = false, features = ["deflate"] }
|
||||
# Versions match what wry already locks — no new native code is pulled in.
|
||||
webview2-com = "0.38"
|
||||
windows-core = "0.61"
|
||||
# Raw HWND access (GetWindowLongPtrW/SetWindowLongPtrW/ShowWindow) to mark the
|
||||
# dictation pill WS_EX_NOACTIVATE so showing it doesn't steal Win32 foreground
|
||||
# activation (#982 — Windows counterpart of #287's macOS focus-steal fix).
|
||||
# Version pinned to match what `tauri` itself already resolves to (0.61.x) so
|
||||
# `WebviewWindow::hwnd()`'s return type and our syscalls share the exact same
|
||||
# `HWND` type — no second copy of the crate enters the dependency graph.
|
||||
windows = { version = "0.61", features = ["Win32_Foundation", "Win32_UI_WindowsAndMessaging"] }
|
||||
|
||||
[target.'cfg(unix)'.dependencies]
|
||||
libc = "0.2"
|
||||
|
||||
@@ -22,9 +22,27 @@ HERE="$(dirname -- "$(readlink -f -- "$0")")"
|
||||
# Sourced by AppRun.test.sh — keep this function pure so unit tests can stub
|
||||
# `pkg-config`, source the file, call _detect_webkit_workaround, and inspect
|
||||
# the resulting environment without exec'ing the binary.
|
||||
#
|
||||
# Version source (#961 follow-up): the WebKitGTK that actually RUNS is the
|
||||
# BUNDLED copy (LD_LIBRARY_PATH below puts $HERE/usr/lib first) — NOT the
|
||||
# host's. Asking the host's pkg-config therefore reads the wrong number
|
||||
# whenever host and bundle diverge (e.g. a user who builds from source has
|
||||
# dev packages installed, so pkg-config answers with their system's healthy
|
||||
# 2.48 while the bundle runs an older lib — skipping a workaround the running
|
||||
# library needs). inject-apprun.sh stamps the bundled version into
|
||||
# .bundled-webkitgtk-version at build time, where it is knowable by
|
||||
# construction; the host pkg-config path survives only as a fallback for
|
||||
# bundles predating the stamp. OMNIVOICE_APPRUN_WK_MARKER exists for the
|
||||
# unit tests to point at a fixture marker.
|
||||
_detect_webkit_workaround() {
|
||||
local wk_version="0.0"
|
||||
if command -v pkg-config >/dev/null 2>&1; then
|
||||
local marker="${OMNIVOICE_APPRUN_WK_MARKER:-$HERE/.bundled-webkitgtk-version}"
|
||||
if [ -r "$marker" ]; then
|
||||
# Empty/unreadable marker content → "0.0" (unknown) → fail-safe workaround,
|
||||
# same philosophy as the missing-pkg-config branch below.
|
||||
wk_version="$(cat "$marker" 2>/dev/null | tr -d '[:space:]')"
|
||||
[ -n "$wk_version" ] || wk_version="0.0"
|
||||
elif command -v pkg-config >/dev/null 2>&1; then
|
||||
wk_version="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|
||||
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|
||||
|| echo "0.0")"
|
||||
|
||||
@@ -72,6 +72,56 @@ run_case "2.46 (broken)" "2.46.1" "1"
|
||||
run_case "2.48 (healthy)" "2.48.0" "unset"
|
||||
run_case "pkg-config absent" "0.0" "1" "no"
|
||||
|
||||
# ── Bundled-version marker cases (#961 follow-up) ───────────────────────────
|
||||
# inject-apprun.sh stamps the bundle's actual WebKitGTK version into
|
||||
# .bundled-webkitgtk-version at build time; AppRun must prefer that marker
|
||||
# over the host's pkg-config (which reports the SYSTEM version — wrong
|
||||
# whenever it diverges from the bundled copy, e.g. on a machine with newer
|
||||
# dev packages installed).
|
||||
|
||||
run_marker_case() {
|
||||
local label="$1" marker_content="$2" pkg_output="$3" expected="$4"
|
||||
local marker_file
|
||||
marker_file="$(mktemp)"
|
||||
printf '%s\n' "$marker_content" > "$marker_file"
|
||||
|
||||
local actual
|
||||
actual=$(
|
||||
bash -c '
|
||||
set +e
|
||||
pkg_output="'"$pkg_output"'"
|
||||
export OMNIVOICE_APPRUN_WK_MARKER="'"$marker_file"'"
|
||||
|
||||
pkg-config() { echo "$pkg_output"; }
|
||||
export -f pkg-config
|
||||
|
||||
exec() { :; }
|
||||
export -f exec
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
source "'"$THIS_DIR"'/AppRun" >/dev/null 2>&1 || true
|
||||
echo "${WEBKIT_DISABLE_COMPOSITING_MODE:-unset}"
|
||||
'
|
||||
)
|
||||
rm -f "$marker_file"
|
||||
|
||||
if [[ "$actual" == "$expected" ]]; then
|
||||
echo "PASS [$label]"
|
||||
PASS_COUNT=$((PASS_COUNT + 1))
|
||||
else
|
||||
echo "FAIL [$label]: expected '$expected' got '$actual'" >&2
|
||||
FAIL_COUNT=$((FAIL_COUNT + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
# Marker says broken → workaround applies, even though host pkg-config says healthy.
|
||||
run_marker_case "marker 2.46 beats host 2.48" "2.46.1" "2.48.0" "1"
|
||||
# Marker says healthy → no workaround, even though host pkg-config says broken
|
||||
# (the exact #961 inversion: from-source user with old system lib, new bundle).
|
||||
run_marker_case "marker 2.48 beats host 2.44" "2.48.0" "2.44.3" "unset"
|
||||
# Empty marker → treated as unknown → fail-safe workaround.
|
||||
run_marker_case "empty marker fails safe" "" "2.48.0" "1"
|
||||
|
||||
echo
|
||||
echo "─── AppRun test summary: $PASS_COUNT pass / $FAIL_COUNT fail ───"
|
||||
if [[ $FAIL_COUNT -ne 0 ]]; then
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<!--
|
||||
Tauri's macOS bundle defaults `hardenedRuntime` to true. Hardened
|
||||
Runtime blocks camera/microphone hardware access unless the matching
|
||||
entitlement is present here — regardless of Info.plist's
|
||||
NSMicrophoneUsageDescription and regardless of wry's own WKUIDelegate
|
||||
already granting the request at the WebKit/JS layer
|
||||
(WryWebViewUIDelegate::request_media_capture_permission unconditionally
|
||||
calls WKPermissionDecision::Grant). Without this entitlement, TCC
|
||||
never even registers a request for the app — nothing shows up in
|
||||
System Settings → Privacy & Security → Microphone to enable, because
|
||||
the OS never saw a legitimately-entitled process ask.
|
||||
-->
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<true/>
|
||||
|
||||
<!--
|
||||
Matches Info.plist's forward-looking NSCameraUsageDescription — no
|
||||
current feature uses the camera, but ship the entitlement now so a
|
||||
future getUserMedia({video: true}) call doesn't hit this same bug.
|
||||
-->
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -12,6 +12,7 @@ use serde::Serialize;
|
||||
use tauri::{Emitter, Manager};
|
||||
|
||||
use crate::config::get_effective_region;
|
||||
use crate::crash::BackendExit;
|
||||
use crate::tools::resolve_uv;
|
||||
use crate::{AppFlags, BackendState, backend_port};
|
||||
|
||||
@@ -164,6 +165,7 @@ pub fn retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_, BootstrapS
|
||||
if v.is_empty() { "<unknown>" } else { v.as_str() },
|
||||
env!("CARGO_PKG_VERSION"),
|
||||
);
|
||||
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
|
||||
crate::backend::kill_orphan_on_port(backend_port());
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
@@ -171,6 +173,7 @@ pub fn retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_, BootstrapS
|
||||
}
|
||||
if crate::backend::port_in_use(backend_port()) {
|
||||
log::warn!("Port {} in use — taking ownership", backend_port());
|
||||
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
|
||||
crate::backend::kill_orphan_on_port(backend_port());
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
@@ -192,9 +195,7 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
|
||||
let mut venv_heal_attempted = false;
|
||||
'bootstrap: loop {
|
||||
let child = crate::backend::spawn_backend(app, Some(stage_handle));
|
||||
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
|
||||
*guard = child;
|
||||
}
|
||||
track_backend_child(app, child);
|
||||
let start = std::time::Instant::now();
|
||||
while start.elapsed() < Duration::from_secs(300) {
|
||||
if crate::backend::backend_healthy(backend_port()) {
|
||||
@@ -213,20 +214,41 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
|
||||
}
|
||||
return;
|
||||
}
|
||||
let process_dead = if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
|
||||
match guard.as_mut() {
|
||||
Some(child) => match child.try_wait() {
|
||||
Ok(Some(status)) => Some(status.to_string()),
|
||||
Ok(None) => None,
|
||||
Err(_) => Some("unknown".to_string()),
|
||||
},
|
||||
None => Some("never started".to_string()),
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
if let Some(exit_info) = process_dead {
|
||||
let process_dead: Option<(String, Option<BackendExit>)> =
|
||||
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
|
||||
match guard.as_mut() {
|
||||
Some(child) => match child.try_wait() {
|
||||
Ok(Some(status)) => {
|
||||
let exit = BackendExit::from_status(status);
|
||||
Some((exit.description.clone(), Some(exit)))
|
||||
}
|
||||
Ok(None) => None,
|
||||
// try_wait errored — the death is real but its
|
||||
// shape is unknown; no exit code for the marker.
|
||||
Err(_) => Some(("unknown".to_string(), None)),
|
||||
},
|
||||
// Spawn itself failed — no process ever ran, so this
|
||||
// is a spawn failure (spawn_failure_diagnostic owns
|
||||
// it), NOT a crash: no marker.
|
||||
None => Some(("never started".to_string(), None)),
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
if let Some((exit_info, real_exit)) = process_dead {
|
||||
let err_tail = crate::backend::read_error_log_tail(30);
|
||||
// #941: persist the forensics for every true process death —
|
||||
// startup crashes included — unless the app is shutting down
|
||||
// or a retry flow deliberately killed the child.
|
||||
if let Some(ref exit) = real_exit {
|
||||
if !app_is_quitting(app) && !backend_kill_intended() {
|
||||
crate::crash::record_crash(crate::crash::marker_now(
|
||||
exit,
|
||||
backend_uptime_s(app),
|
||||
crate::backend::read_error_log_tail(CRASH_STDERR_TAIL_LINES),
|
||||
));
|
||||
}
|
||||
}
|
||||
// #314: a backend that dies because the venv itself is broken
|
||||
// can only be healed by rebuilding the venv — do that once
|
||||
// instead of failing into an unwinnable retry loop.
|
||||
@@ -308,12 +330,34 @@ pub fn spawn_backend_and_wait(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<B
|
||||
/// first to reach Ready claims this and the rest fall through.
|
||||
static SUPERVISOR_ACTIVE: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
/// Give up (surface Failed) if the backend dies this many times within
|
||||
/// `RESTART_WINDOW` — a deterministic startup crash must not become a
|
||||
/// fork-bomb. The #314 broken-venv self-heal stays the venv-failure path; the
|
||||
/// #941: set while a retry/clean-retry flow deliberately kills the backend to
|
||||
/// replace it, so the death watchers (startup poll + supervisor) never write a
|
||||
/// crash marker for — or respawn against — an *intentional* kill. Cleared the
|
||||
/// moment a fresh child is spawned and tracked (`track_backend_child`).
|
||||
static BACKEND_KILL_INTENDED: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
pub fn set_backend_kill_intended(value: bool) {
|
||||
BACKEND_KILL_INTENDED.store(value, Ordering::SeqCst);
|
||||
}
|
||||
|
||||
fn backend_kill_intended() -> bool {
|
||||
BACKEND_KILL_INTENDED.load(Ordering::SeqCst)
|
||||
}
|
||||
|
||||
/// How much of backend_err.log rides inside a crash marker (#941). ~40 lines
|
||||
/// is enough for a Python traceback or a native abort banner without bloating
|
||||
/// the marker file or the bug-report URL (the frontend truncates further).
|
||||
const CRASH_STDERR_TAIL_LINES: usize = 40;
|
||||
|
||||
/// Crash-loop escalation guard (#941, supersedes the #567 5-in-60s budget):
|
||||
/// give up (surface Failed with the crash details) once the backend has died
|
||||
/// `MAX_RESTARTS` times inside `RESTART_WINDOW`. The longer 10-minute window
|
||||
/// catches *slow* crash loops (e.g. an engine that OOMs a couple of minutes
|
||||
/// into every generation) that the old 60-second window let spin silently
|
||||
/// forever. The #314 broken-venv self-heal stays the venv-failure path; the
|
||||
/// supervisor only handles post-Ready deaths.
|
||||
const MAX_RESTARTS: usize = 5;
|
||||
const RESTART_WINDOW: Duration = Duration::from_secs(60);
|
||||
const MAX_RESTARTS: usize = 3;
|
||||
const RESTART_WINDOW: Duration = Duration::from_secs(600);
|
||||
|
||||
fn app_is_quitting(app: &tauri::AppHandle) -> bool {
|
||||
app.try_state::<AppFlags>()
|
||||
@@ -321,17 +365,39 @@ fn app_is_quitting(app: &tauri::AppHandle) -> bool {
|
||||
.unwrap_or(false)
|
||||
}
|
||||
|
||||
/// Returns `Some(exit description)` if the tracked backend child has exited,
|
||||
/// Store the freshly spawned backend child (and its spawn time, for the crash
|
||||
/// marker's `uptime_s`), and re-arm the death watchers: any deliberate-kill
|
||||
/// window ends the moment a new child is tracked.
|
||||
fn track_backend_child(app: &tauri::AppHandle, child: Option<std::process::Child>) {
|
||||
let state = app.state::<BackendState>();
|
||||
if let Ok(mut guard) = state.process.lock() {
|
||||
*guard = child;
|
||||
}
|
||||
if let Ok(mut spawned) = state.spawned_at.lock() {
|
||||
*spawned = Some(Instant::now());
|
||||
}
|
||||
set_backend_kill_intended(false);
|
||||
}
|
||||
|
||||
/// Seconds since the tracked backend child was spawned (0 when unknown).
|
||||
fn backend_uptime_s(app: &tauri::AppHandle) -> u64 {
|
||||
app.try_state::<BackendState>()
|
||||
.and_then(|s| s.spawned_at.lock().ok().and_then(|g| *g))
|
||||
.map(|t| t.elapsed().as_secs())
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Returns `Some(BackendExit)` if the tracked backend child has exited,
|
||||
/// `None` if it is still running (or none is tracked — which we never treat as
|
||||
/// a death to respawn, to avoid fighting a deliberate teardown).
|
||||
fn backend_child_exit(app: &tauri::AppHandle) -> Option<String> {
|
||||
fn backend_child_exit(app: &tauri::AppHandle) -> Option<BackendExit> {
|
||||
let state = app.try_state::<BackendState>()?;
|
||||
let mut guard = state.process.lock().ok()?;
|
||||
match guard.as_mut() {
|
||||
Some(child) => match child.try_wait() {
|
||||
Ok(Some(status)) => Some(status.to_string()),
|
||||
Ok(Some(status)) => Some(BackendExit::from_status(status)),
|
||||
Ok(None) => None,
|
||||
Err(e) => Some(format!("try_wait error: {e}")),
|
||||
Err(e) => Some(BackendExit::unknown(&format!("try_wait error: {e}"))),
|
||||
},
|
||||
None => None,
|
||||
}
|
||||
@@ -358,21 +424,39 @@ fn supervise_backend(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<BootstrapS
|
||||
if app_is_quitting(app) {
|
||||
return;
|
||||
}
|
||||
let exit_info = match backend_child_exit(app) {
|
||||
Some(info) => info,
|
||||
let exit = match backend_child_exit(app) {
|
||||
Some(exit) => exit,
|
||||
None => continue, // still running
|
||||
};
|
||||
// The exit may have raced with a shutdown that killed the child.
|
||||
if app_is_quitting(app) {
|
||||
return;
|
||||
}
|
||||
// A retry/clean-retry flow killed the child on purpose and owns the
|
||||
// respawn — no crash marker, and step aside so the retry's own
|
||||
// spawn_backend_and_wait claims the supervisor slot at Ready (#941).
|
||||
if backend_kill_intended() {
|
||||
log::info!("Backend exit was a deliberate replace — supervisor yielding to the retry flow");
|
||||
return;
|
||||
}
|
||||
let exit_info = exit.description.clone();
|
||||
// #941: make the death self-documenting BEFORE any restart attempt —
|
||||
// the marker (exit code/signal + stderr tail + uptime) is what turns
|
||||
// the next "Can't reach the backend" report into a diagnosable one.
|
||||
let uptime_s = backend_uptime_s(app);
|
||||
crate::crash::record_crash(crate::crash::marker_now(
|
||||
&exit,
|
||||
uptime_s,
|
||||
crate::backend::read_error_log_tail(CRASH_STDERR_TAIL_LINES),
|
||||
));
|
||||
if restart_budget_exhausted(&mut restart_times, Instant::now()) {
|
||||
let tail = crate::backend::read_error_log_tail(30);
|
||||
let msg = format!(
|
||||
"The backend kept crashing ({} times in {}s) and couldn't be kept running. \
|
||||
Use Clean & Retry, or check Settings → Logs → Backend.{}",
|
||||
"The backend kept crashing ({} times in {} min; last death: {}) and couldn't \
|
||||
be kept running. Use Clean & Retry, or check Settings → Logs → Backend.{}",
|
||||
MAX_RESTARTS,
|
||||
RESTART_WINDOW.as_secs(),
|
||||
RESTART_WINDOW.as_secs() / 60,
|
||||
exit.label(),
|
||||
if tail.is_empty() { String::new() } else { format!("\n\nLast output:\n{tail}") },
|
||||
);
|
||||
log::error!("Backend supervisor giving up: {msg}");
|
||||
@@ -393,9 +477,7 @@ fn supervise_backend(app: &tauri::AppHandle, stage_handle: &Arc<Mutex<BootstrapS
|
||||
std::thread::sleep(Duration::from_millis(300));
|
||||
}
|
||||
let child = crate::backend::spawn_backend(app, Some(stage_handle));
|
||||
if let Ok(mut guard) = app.state::<BackendState>().process.lock() {
|
||||
*guard = child;
|
||||
}
|
||||
track_backend_child(app, child);
|
||||
// Wait (bounded) for the respawn to become healthy. If it dies again
|
||||
// immediately, bail early so the next loop counts it toward the cap.
|
||||
let start = Instant::now();
|
||||
@@ -430,6 +512,7 @@ pub fn clean_and_retry_bootstrap(app: tauri::AppHandle, state: tauri::State<'_,
|
||||
// project dir, otherwise bootstrap will "attach" to the stale process.
|
||||
if crate::backend::port_in_use(backend_port()) {
|
||||
log::warn!("Clean retry: killing stale backend on port {}", backend_port());
|
||||
set_backend_kill_intended(true); // deliberate kill, not a crash (#941)
|
||||
crate::backend::kill_orphan_on_port(backend_port());
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
@@ -621,9 +704,14 @@ fn sync_failure_is_torch_download(tail: &str) -> bool {
|
||||
|| (low.contains("torch") && (low.contains("failed to download") || low.contains("failed to fetch")))
|
||||
}
|
||||
|
||||
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124). ROCm 6.2 is
|
||||
/// the current stable wheel set; overridable via OMNIVOICE_TORCH_INDEX.
|
||||
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.2";
|
||||
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124).
|
||||
/// ROCm 6.4, not 6.2: the app's pinned `torch==2.8.0` (pyproject.toml) has no
|
||||
/// build on the rocm6.2 index (it tops out at 2.5.1), so that index silently
|
||||
/// failed the reinstall and left the default CUDA build in place — which runs
|
||||
/// on CPU on an AMD GPU (#972). rocm6.4 carries a matching 2.8.0 build.
|
||||
/// Overridable via OMNIVOICE_TORCH_INDEX (e.g. a `--find-links` URL for
|
||||
/// distro-matched ROCm builds torch's own index doesn't carry).
|
||||
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.4";
|
||||
|
||||
/// `uv pip install` args that replace the default CUDA torch build with the AMD
|
||||
/// ROCm wheel (#124). Opt-in (gated on OMNIVOICE_TORCH_VARIANT=rocm by the
|
||||
@@ -1668,6 +1756,15 @@ mod tests {
|
||||
assert_eq!(envs.get("UV_HTTP_RETRIES").map(String::as_str), Some("5"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn crash_loop_policy_is_three_deaths_in_ten_minutes() {
|
||||
// #941 escalation guard: ≥3 crashes inside 10 min must stop the
|
||||
// respawn loop and land on the Failed screen with the crash details —
|
||||
// the old 5-in-60s budget let slow crash loops spin silently forever.
|
||||
assert_eq!(MAX_RESTARTS, 3);
|
||||
assert_eq!(RESTART_WINDOW, Duration::from_secs(600));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn restart_budget_caps_respawns_and_prunes_old_ones() {
|
||||
// Supervisor backoff policy (#567): fewer than MAX_RESTARTS deaths
|
||||
@@ -1725,7 +1822,10 @@ mod tests {
|
||||
assert!(args.iter().any(|a| a == "torch"));
|
||||
assert!(args.iter().any(|a| a == "torchaudio"));
|
||||
let i = args.iter().position(|a| a == "--index-url").expect("has --index-url");
|
||||
assert!(args[i + 1].contains("rocm6.2"), "default index is the rocm6.2 wheel set");
|
||||
// rocm6.4, not rocm6.2: rocm6.2's index tops out at torch 2.5.1 and
|
||||
// can't satisfy the app's torch==2.8.0 pin (#972) — a regression to
|
||||
// rocm6.2 here would silently resurrect the CPU-fallback bug.
|
||||
assert!(args[i + 1].contains("rocm6.4"), "default index is the rocm6.4 wheel set (matches torch==2.8.0)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
//! Backend crash forensics (#941).
|
||||
//!
|
||||
//! When the backend PROCESS dies (native CUDA abort, OOM kill, DLL crash),
|
||||
//! the user used to see only "Can't reach the local OmniVoice backend" — and
|
||||
//! the evidence (exit code, stderr tail) evaporated with the process. Every
|
||||
//! such report was undiagnosable without asking for logs nobody sends.
|
||||
//!
|
||||
//! This module makes every backend death self-documenting: the death watchers
|
||||
//! in `bootstrap.rs` (the startup health poll and the post-Ready supervisor)
|
||||
//! call [`record_crash`] with the exit status and captured stderr tail, which
|
||||
//! persists a small JSON **crash marker** next to the backend logs. The
|
||||
//! frontend reads the newest marker via the `get_last_backend_crash` command
|
||||
//! to replace the vague unreachable-toast with the honest story ("the backend
|
||||
//! crashed (exit code X)…"), and the bug-report prefill attaches it so the
|
||||
//! next #941-class GitHub issue arrives WITH the evidence.
|
||||
//!
|
||||
//! Only the last [`MAX_MARKERS`] crashes are kept. Acknowledgment is a
|
||||
//! persisted timestamp (not deletion!) so viewing the crash details doesn't
|
||||
//! destroy the evidence a subsequent bug report needs.
|
||||
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::ExitStatus;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// How many crash markers to retain (newest first).
|
||||
pub const MAX_MARKERS: usize = 3;
|
||||
|
||||
// ── Exit-status decomposition ──────────────────────────────────────────────
|
||||
|
||||
/// Structured view of how the backend child ended: the numeric exit code (or
|
||||
/// Unix signal) for the marker, plus the human-readable `ExitStatus` display
|
||||
/// for logs and bootstrap messages.
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub struct BackendExit {
|
||||
pub code: Option<i32>,
|
||||
pub signal: Option<i32>,
|
||||
pub description: String,
|
||||
}
|
||||
|
||||
impl BackendExit {
|
||||
pub fn from_status(status: ExitStatus) -> Self {
|
||||
#[cfg(unix)]
|
||||
let signal = {
|
||||
use std::os::unix::process::ExitStatusExt;
|
||||
status.signal()
|
||||
};
|
||||
#[cfg(not(unix))]
|
||||
let signal = None;
|
||||
BackendExit { code: status.code(), signal, description: status.to_string() }
|
||||
}
|
||||
|
||||
/// For deaths we can't decompose (`try_wait` errored).
|
||||
pub fn unknown(description: &str) -> Self {
|
||||
BackendExit { code: None, signal: None, description: description.to_string() }
|
||||
}
|
||||
|
||||
/// Short human label — "exit code 3221226505" / "signal 6" — for messages.
|
||||
pub fn label(&self) -> String {
|
||||
match (self.code, self.signal) {
|
||||
(Some(c), _) => format!("exit code {}", c),
|
||||
(None, Some(s)) => format!("signal {}", s),
|
||||
(None, None) => self.description.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Marker model ───────────────────────────────────────────────────────────
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CrashMarker {
|
||||
/// Unix seconds when the death was detected.
|
||||
pub ts: u64,
|
||||
/// Process exit code, when the OS reported one.
|
||||
pub exit_code: Option<i32>,
|
||||
/// Unix signal that killed the process (None on Windows / normal exits).
|
||||
pub signal: Option<i32>,
|
||||
/// Human-readable `ExitStatus` display ("exit status: 134", …).
|
||||
pub exit_desc: String,
|
||||
/// App/backend version (lockstep per the versioning rule).
|
||||
pub backend_version: String,
|
||||
/// Seconds the backend had been running when it died.
|
||||
pub uptime_s: u64,
|
||||
/// Tail of backend_err.log captured at death time.
|
||||
pub last_stderr: String,
|
||||
}
|
||||
|
||||
/// The single on-disk store: newest-first markers plus the acknowledgment
|
||||
/// watermark. One file keeps rotation + ack updates atomic-ish and avoids
|
||||
/// filename collisions for same-second crashes.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct CrashStore {
|
||||
/// `ts` of the newest marker the user has acknowledged (seen). Markers
|
||||
/// with `ts <= acked_ts` are "old news" for UI purposes but are retained
|
||||
/// for bug-report attachment.
|
||||
#[serde(default)]
|
||||
pub acked_ts: u64,
|
||||
/// Newest first, capped at [`MAX_MARKERS`].
|
||||
#[serde(default)]
|
||||
pub markers: Vec<CrashMarker>,
|
||||
}
|
||||
|
||||
/// Prepend `marker` and keep only the newest [`MAX_MARKERS`]. Pure so the
|
||||
/// rotation policy is unit-tested without touching the filesystem.
|
||||
pub fn push_marker(store: &mut CrashStore, marker: CrashMarker) {
|
||||
store.markers.insert(0, marker);
|
||||
store.markers.truncate(MAX_MARKERS);
|
||||
}
|
||||
|
||||
/// Newest marker + whether the user has already acknowledged it.
|
||||
pub fn newest_with_ack(store: &CrashStore) -> Option<(CrashMarker, bool)> {
|
||||
store.markers.first().map(|m| (m.clone(), m.ts <= store.acked_ts))
|
||||
}
|
||||
|
||||
// ── Persistence ────────────────────────────────────────────────────────────
|
||||
|
||||
/// The marker store lives next to the backend logs (same rationale: it's
|
||||
/// forensic output of the backend process, discoverable alongside
|
||||
/// backend.log / backend_err.log).
|
||||
pub fn markers_path() -> PathBuf {
|
||||
crate::backend::backend_log_path().with_file_name("backend_crash_markers.json")
|
||||
}
|
||||
|
||||
pub fn load_store_from(path: &Path) -> CrashStore {
|
||||
fs::read_to_string(path)
|
||||
.ok()
|
||||
.and_then(|s| serde_json::from_str(&s).ok())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn save_store_to(path: &Path, store: &CrashStore) {
|
||||
match serde_json::to_string_pretty(store) {
|
||||
Ok(json) => {
|
||||
if let Err(e) = fs::write(path, json) {
|
||||
log::warn!("Could not persist crash marker to {}: {}", path.display(), e);
|
||||
}
|
||||
}
|
||||
Err(e) => log::warn!("Could not serialize crash marker: {}", e),
|
||||
}
|
||||
}
|
||||
|
||||
fn now_unix_s() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
/// Build a marker for a death detected right now.
|
||||
pub fn marker_now(exit: &BackendExit, uptime_s: u64, last_stderr: String) -> CrashMarker {
|
||||
CrashMarker {
|
||||
ts: now_unix_s(),
|
||||
exit_code: exit.code,
|
||||
signal: exit.signal,
|
||||
exit_desc: exit.description.clone(),
|
||||
backend_version: env!("CARGO_PKG_VERSION").to_string(),
|
||||
uptime_s,
|
||||
last_stderr,
|
||||
}
|
||||
}
|
||||
|
||||
/// Persist an unexpected backend death. Called by the death watchers in
|
||||
/// `bootstrap.rs` AFTER they have ruled out intentional shutdowns (app quit,
|
||||
/// deliberate retry/clean-retry kills).
|
||||
pub fn record_crash(marker: CrashMarker) {
|
||||
log::error!(
|
||||
"Backend process died unexpectedly ({}, uptime {} s). Crash marker written. Stderr tail:\n{}",
|
||||
marker.exit_desc,
|
||||
marker.uptime_s,
|
||||
if marker.last_stderr.is_empty() { "<none captured>" } else { &marker.last_stderr },
|
||||
);
|
||||
let path = markers_path();
|
||||
let mut store = load_store_from(&path);
|
||||
push_marker(&mut store, marker);
|
||||
save_store_to(&path, &store);
|
||||
}
|
||||
|
||||
// ── Tauri commands ─────────────────────────────────────────────────────────
|
||||
|
||||
/// Newest crash marker + its acknowledgment state, as returned to the
|
||||
/// frontend (`get_last_backend_crash`).
|
||||
#[derive(Clone, Debug, Serialize)]
|
||||
pub struct CrashNotice {
|
||||
#[serde(flatten)]
|
||||
pub marker: CrashMarker,
|
||||
pub acknowledged: bool,
|
||||
}
|
||||
|
||||
/// Newest backend crash marker, or null when the backend has never crashed.
|
||||
/// `acknowledged` tells the UI whether the user already viewed/dismissed it.
|
||||
#[tauri::command]
|
||||
pub fn get_last_backend_crash() -> Option<CrashNotice> {
|
||||
let store = load_store_from(&markers_path());
|
||||
newest_with_ack(&store).map(|(marker, acknowledged)| CrashNotice { marker, acknowledged })
|
||||
}
|
||||
|
||||
/// Mark the newest crash as seen. Deliberately does NOT delete the marker —
|
||||
/// the bug-report prefill still needs the evidence after the user viewed it.
|
||||
#[tauri::command]
|
||||
pub fn acknowledge_backend_crash() {
|
||||
let path = markers_path();
|
||||
let mut store = load_store_from(&path);
|
||||
if let Some(newest_ts) = store.markers.first().map(|m| m.ts) {
|
||||
if store.acked_ts < newest_ts {
|
||||
store.acked_ts = newest_ts;
|
||||
save_store_to(&path, &store);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn marker(ts: u64) -> CrashMarker {
|
||||
CrashMarker {
|
||||
ts,
|
||||
exit_code: Some(1),
|
||||
signal: None,
|
||||
exit_desc: format!("exit status: 1 (#{ts})"),
|
||||
backend_version: "0.0.0-test".into(),
|
||||
uptime_s: 42,
|
||||
last_stderr: "Traceback…".into(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rotation_keeps_only_the_last_three_newest_first() {
|
||||
// #941: write 4 markers → only the newest MAX_MARKERS survive.
|
||||
let mut store = CrashStore::default();
|
||||
for ts in [1, 2, 3, 4] {
|
||||
push_marker(&mut store, marker(ts));
|
||||
}
|
||||
assert_eq!(store.markers.len(), MAX_MARKERS);
|
||||
let kept: Vec<u64> = store.markers.iter().map(|m| m.ts).collect();
|
||||
assert_eq!(kept, vec![4, 3, 2], "newest first, oldest dropped");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ack_semantics_survive_newer_crashes() {
|
||||
let mut store = CrashStore::default();
|
||||
push_marker(&mut store, marker(100));
|
||||
// Fresh crash → unacknowledged.
|
||||
let (m, acked) = newest_with_ack(&store).expect("has a marker");
|
||||
assert_eq!(m.ts, 100);
|
||||
assert!(!acked, "a fresh crash must be unacknowledged");
|
||||
// Viewing acks the newest…
|
||||
store.acked_ts = 100;
|
||||
assert!(newest_with_ack(&store).unwrap().1, "viewed crash is acknowledged");
|
||||
// …but a NEWER crash re-arms the notice, and the marker itself is
|
||||
// retained (evidence survives the ack — bug reports still attach it).
|
||||
push_marker(&mut store, marker(200));
|
||||
let (m2, acked2) = newest_with_ack(&store).unwrap();
|
||||
assert_eq!(m2.ts, 200);
|
||||
assert!(!acked2, "a newer crash must surface again");
|
||||
assert_eq!(store.markers.len(), 2, "ack never deletes markers");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn store_roundtrips_through_json_and_defaults_when_missing() {
|
||||
let dir = std::env::temp_dir().join(format!("omnivoice-test-941-{}", std::process::id()));
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("backend_crash_markers.json");
|
||||
|
||||
// Missing file → default store, never an error (first run).
|
||||
assert_eq!(load_store_from(&path), CrashStore::default());
|
||||
// Corrupt file → default store (a truncated write must not wedge the
|
||||
// whole forensics path).
|
||||
fs::write(&path, "{not json").unwrap();
|
||||
assert_eq!(load_store_from(&path), CrashStore::default());
|
||||
|
||||
let mut store = CrashStore::default();
|
||||
push_marker(
|
||||
&mut store,
|
||||
CrashMarker {
|
||||
ts: 1,
|
||||
exit_code: None,
|
||||
signal: Some(6), // SIGABRT — the native-CUDA-abort shape
|
||||
exit_desc: "signal: 6 (SIGABRT)".into(),
|
||||
backend_version: "0.3.10".into(),
|
||||
uptime_s: 7,
|
||||
last_stderr: "CUDA error: an illegal memory access".into(),
|
||||
},
|
||||
);
|
||||
store.acked_ts = 0;
|
||||
save_store_to(&path, &store);
|
||||
let loaded = load_store_from(&path);
|
||||
assert_eq!(loaded, store, "Option fields (code=None, signal=Some) must roundtrip");
|
||||
let _ = fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn backend_exit_labels_code_signal_and_unknown() {
|
||||
let coded = BackendExit { code: Some(-1073740791), signal: None, description: "x".into() };
|
||||
assert_eq!(coded.label(), "exit code -1073740791");
|
||||
let signaled = BackendExit { code: None, signal: Some(9), description: "x".into() };
|
||||
assert_eq!(signaled.label(), "signal 9");
|
||||
let unknown = BackendExit::unknown("try_wait error: gone");
|
||||
assert_eq!(unknown.label(), "try_wait error: gone");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn backend_exit_decomposes_real_exit_statuses() {
|
||||
use std::os::unix::process::ExitStatusExt;
|
||||
// Normal exit with code 3.
|
||||
let e = BackendExit::from_status(ExitStatus::from_raw(3 << 8));
|
||||
assert_eq!(e.code, Some(3));
|
||||
assert_eq!(e.signal, None);
|
||||
// Killed by SIGABRT (6) — code is None, signal carries the story.
|
||||
let k = BackendExit::from_status(ExitStatus::from_raw(6));
|
||||
assert_eq!(k.code, None);
|
||||
assert_eq!(k.signal, Some(6));
|
||||
assert_eq!(k.label(), "signal 6");
|
||||
}
|
||||
}
|
||||
+168
-12
@@ -13,6 +13,7 @@ pub mod bootstrap;
|
||||
pub mod tools;
|
||||
pub mod backend;
|
||||
pub mod commands;
|
||||
pub mod crash;
|
||||
pub mod updater_channel;
|
||||
|
||||
use std::process::Child;
|
||||
@@ -41,6 +42,9 @@ pub fn backend_port() -> u16 {
|
||||
|
||||
pub struct BackendState {
|
||||
pub process: Mutex<Option<Child>>,
|
||||
/// When the tracked child was spawned — feeds the crash marker's
|
||||
/// `uptime_s` (#941). Set alongside `process` in bootstrap.rs.
|
||||
pub spawned_at: Mutex<Option<std::time::Instant>>,
|
||||
}
|
||||
|
||||
pub struct AppFlags {
|
||||
@@ -75,9 +79,18 @@ pub const TRAY_ICON_RECORDING: &[u8] = include_bytes!("../icons/tray-recording.p
|
||||
// applies on top.
|
||||
// - Linux (WebKitGTK): media-stream must be enabled per-WebView and the
|
||||
// permission request answered programmatically.
|
||||
// - macOS (WKWebView): nothing to do here — wry grants media-capture to the
|
||||
// app origin and the user-visible consent is the system TCC prompt driven
|
||||
// by NSMicrophoneUsageDescription in src-tauri/Info.plist.
|
||||
// - macOS (WKWebView): nothing to do here in code — wry's own WKUIDelegate
|
||||
// (WryWebViewUIDelegate::request_media_capture_permission) already grants
|
||||
// every media-capture request unconditionally at the WebKit/JS layer. But
|
||||
// that alone isn't sufficient (#1013): Tauri's macOS bundle defaults
|
||||
// `hardenedRuntime` to true, and Hardened Runtime blocks camera/microphone
|
||||
// hardware access unless the matching entitlement is present — without it,
|
||||
// TCC never even registers a request, so the app never appears in System
|
||||
// Settings → Privacy & Security → Microphone for the user to enable. See
|
||||
// src-tauri/entitlements.plist (wired in via tauri.conf.json's
|
||||
// bundle.macOS.entitlements) for the actual grant; NSMicrophoneUsageDescription
|
||||
// in Info.plist only supplies the *prompt text* TCC shows, it doesn't
|
||||
// substitute for the entitlement.
|
||||
|
||||
/// True for origins the app itself serves: the Tauri custom-protocol origin
|
||||
/// in production and the Vite dev server / loopback in `tauri dev`.
|
||||
@@ -201,6 +214,120 @@ mod media_permission_tests {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Windows: dictation pill must never take foreground focus (#982) ────────
|
||||
//
|
||||
// Windows counterpart of #287 (macOS auto-paste — don't steal focus). The
|
||||
// pill is `.always_on_top(true).skip_taskbar(true)` and is documented above
|
||||
// (see `grant_webview_media_permissions`) as "deliberately unfocused so the
|
||||
// auto-paste lands in the target app" — true on macOS, but on Windows,
|
||||
// showing an always-on-top top-level window gives it Win32 foreground
|
||||
// activation by default (ordinary Windows window-manager behavior; macOS
|
||||
// doesn't force-activate a shown window the same way). Nothing marked the
|
||||
// pill non-activating, so on Windows it stole foreground on every show —
|
||||
// the synthesized Ctrl+V from `simulate_paste` landed back in the pill
|
||||
// instead of the app the user was dictating into, and because the pill
|
||||
// wrongly held focus for the whole session the target app never got it back
|
||||
// until the pill's auto-dismiss timer eventually hid it.
|
||||
//
|
||||
// Two pieces, both required (verified by reading how `.show()` is used at
|
||||
// the call sites below — several are followed by an explicit `set_focus()`
|
||||
// that would fight the style bit on its own):
|
||||
// 1. WS_EX_NOACTIVATE on the HWND, applied once right after creation, so
|
||||
// the OS never grants this window foreground activation implicitly.
|
||||
// 2. `ShowWindow(SW_SHOWNOACTIVATE)` in place of `WebviewWindow::show()` at
|
||||
// the pill's dictation-trigger call sites, and the explicit
|
||||
// `set_focus()` calls at those same sites are skipped on Windows (the
|
||||
// same way they already are on macOS below).
|
||||
//
|
||||
// The flag math (`with_noactivate_style`) is a plain function so it's
|
||||
// unit-testable on every platform — the actual Win32 syscalls that use it
|
||||
// are Windows-only and can't run under `cargo test` on a non-Windows runner.
|
||||
|
||||
/// `WS_EX_NOACTIVATE` (winuser.h: `#define WS_EX_NOACTIVATE 0x08000000L`).
|
||||
/// Hardcoded rather than imported from the `windows` crate so `with_noactivate_style`
|
||||
/// below stays free of the Windows-only dependency and is testable everywhere.
|
||||
/// Only consumed by Windows-only code (or the platform-agnostic test module
|
||||
/// below) — `#[allow(dead_code)]` elsewhere, same as `is_app_origin` above.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
const WS_EX_NOACTIVATE_BIT: isize = 0x0800_0000;
|
||||
|
||||
/// OR `WS_EX_NOACTIVATE` into an existing extended window style, preserving
|
||||
/// every other bit already set (topmost, layered, etc. — the pill's
|
||||
/// `always_on_top(true)` sets one of these). Pure so it's unit-testable
|
||||
/// without a real HWND. See module comment above for why this exists.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
fn with_noactivate_style(current_ex_style: isize) -> isize {
|
||||
current_ex_style | WS_EX_NOACTIVATE_BIT
|
||||
}
|
||||
|
||||
/// Mark the pill's HWND `WS_EX_NOACTIVATE`, once, right after creation — this
|
||||
/// holds for every later `.show()` regardless of call site (belt-and-braces
|
||||
/// alongside `show_pill_noactivate` below, which some call sites also need
|
||||
/// because they pair `.show()` with an explicit `set_focus()`).
|
||||
#[cfg(target_os = "windows")]
|
||||
fn mark_pill_noactivate(win: &tauri::WebviewWindow) {
|
||||
use windows::Win32::UI::WindowsAndMessaging::{
|
||||
GetWindowLongPtrW, SetWindowLongPtrW, GWL_EXSTYLE,
|
||||
};
|
||||
let Ok(hwnd) = win.hwnd() else {
|
||||
log::warn!("pill: could not resolve HWND to apply WS_EX_NOACTIVATE (#982)");
|
||||
return;
|
||||
};
|
||||
unsafe {
|
||||
let current = GetWindowLongPtrW(hwnd, GWL_EXSTYLE);
|
||||
SetWindowLongPtrW(hwnd, GWL_EXSTYLE, with_noactivate_style(current));
|
||||
}
|
||||
}
|
||||
|
||||
/// Show the pill without granting it foreground activation. Used instead of
|
||||
/// `WebviewWindow::show()` at the pill's dictation-trigger call sites on
|
||||
/// Windows — `.show()` maps to plain `ShowWindow(SW_SHOW)`, which relies on
|
||||
/// the NOACTIVATE style alone to suppress activation; `SW_SHOWNOACTIVATE` is
|
||||
/// the explicit, documented way to show a window without activating it and
|
||||
/// costs nothing extra now that the style bit is also set (#982).
|
||||
#[cfg(target_os = "windows")]
|
||||
fn show_pill_noactivate(win: &tauri::WebviewWindow) {
|
||||
use windows::Win32::UI::WindowsAndMessaging::{ShowWindow, SW_SHOWNOACTIVATE};
|
||||
let Ok(hwnd) = win.hwnd() else {
|
||||
log::warn!("pill: could not resolve HWND for non-activating show (#982)");
|
||||
return;
|
||||
};
|
||||
unsafe {
|
||||
let _ = ShowWindow(hwnd, SW_SHOWNOACTIVATE);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod pill_noactivate_tests {
|
||||
use super::{with_noactivate_style, WS_EX_NOACTIVATE_BIT};
|
||||
|
||||
#[test]
|
||||
fn adds_noactivate_bit_without_clobbering_existing_style() {
|
||||
// Stand-in for whatever bits the pill's always_on_top/skip_taskbar
|
||||
// window already carries (e.g. WS_EX_TOPMOST = 0x00000008) —
|
||||
// NOACTIVATE must be added on top, never replace them.
|
||||
let topmost = 0x0000_0008isize;
|
||||
let updated = with_noactivate_style(topmost);
|
||||
assert_eq!(
|
||||
updated & WS_EX_NOACTIVATE_BIT,
|
||||
WS_EX_NOACTIVATE_BIT,
|
||||
"NOACTIVATE bit must be set"
|
||||
);
|
||||
assert_eq!(updated & topmost, topmost, "pre-existing style bits must survive");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn idempotent_if_already_noactivate() {
|
||||
assert_eq!(with_noactivate_style(WS_EX_NOACTIVATE_BIT), WS_EX_NOACTIVATE_BIT);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn matches_documented_win32_value() {
|
||||
// winuser.h: #define WS_EX_NOACTIVATE 0x08000000L
|
||||
assert_eq!(WS_EX_NOACTIVATE_BIT, 0x0800_0000);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Tauri entry ───────────────────────────────────────────────────────────
|
||||
|
||||
#[cfg_attr(mobile, tauri::mobile_entry_point)]
|
||||
@@ -271,6 +398,8 @@ pub fn run() {
|
||||
commands::get_launch_as_widget,
|
||||
commands::set_launch_as_widget,
|
||||
commands::clear_webview_cache_and_relaunch,
|
||||
crash::get_last_backend_crash,
|
||||
crash::acknowledge_backend_crash,
|
||||
])
|
||||
.setup(move |app| {
|
||||
app.handle().plugin(tauri_plugin_dialog::init())?;
|
||||
@@ -328,9 +457,15 @@ pub fn run() {
|
||||
.skip_taskbar(true)
|
||||
.center()
|
||||
.build();
|
||||
if let Err(e) = result {
|
||||
if let Err(e) = &result {
|
||||
log::error!("Failed to create widget window: {e:?}");
|
||||
}
|
||||
// Windows: mark the pill non-activating right away so it holds
|
||||
// for every later `.show()` regardless of call site (#982).
|
||||
#[cfg(target_os = "windows")]
|
||||
if let Ok(win) = &result {
|
||||
mark_pill_noactivate(win);
|
||||
}
|
||||
}
|
||||
|
||||
app.manage(AppFlags {
|
||||
@@ -364,12 +499,17 @@ pub fn run() {
|
||||
if win.move_window(Position::BottomCenter).is_err() {
|
||||
let _ = win.center();
|
||||
}
|
||||
// Windows: show without granting foreground activation
|
||||
// (#982) — `.show()` on other platforms is unaffected.
|
||||
#[cfg(target_os = "windows")]
|
||||
show_pill_noactivate(&win);
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
let _ = win.show();
|
||||
// Don't steal focus on macOS: the simulated ⌘V from
|
||||
// simulate_paste() must land in the app the user is
|
||||
// dictating into — focusing the widget would swallow
|
||||
// it (#287).
|
||||
#[cfg(not(target_os = "macos"))]
|
||||
// Don't steal focus on macOS or Windows: the simulated
|
||||
// ⌘V/Ctrl+V from simulate_paste() must land in the app
|
||||
// the user is dictating into — focusing the widget would
|
||||
// swallow it (#287 macOS, #982 Windows).
|
||||
#[cfg(not(any(target_os = "macos", target_os = "windows")))]
|
||||
let _ = win.set_focus();
|
||||
}
|
||||
let _ = app_handle.emit("tray-dictate", ());
|
||||
@@ -520,7 +660,9 @@ pub fn run() {
|
||||
// + focus the widget BEFORE emitting tray-dictate so
|
||||
// the user sees the pill instead of silent recording.
|
||||
// Positioning mirrors the global-shortcut handler:
|
||||
// bottom-center (WhisperFlow style).
|
||||
// bottom-center (WhisperFlow style). Windows skips the
|
||||
// focus (and uses a non-activating show) for the same
|
||||
// reason the global-shortcut handler does — see #982.
|
||||
if let Some(win) = app.get_webview_window("widget") {
|
||||
if win.is_visible().unwrap_or(false) {
|
||||
let _ = app.emit("tray-dictate-stop", ());
|
||||
@@ -528,8 +670,13 @@ pub fn run() {
|
||||
if win.move_window(Position::BottomCenter).is_err() {
|
||||
let _ = win.center();
|
||||
}
|
||||
let _ = win.show();
|
||||
let _ = win.set_focus();
|
||||
#[cfg(target_os = "windows")]
|
||||
show_pill_noactivate(&win);
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
{
|
||||
let _ = win.show();
|
||||
let _ = win.set_focus();
|
||||
}
|
||||
let _ = app.emit("tray-dictate", ());
|
||||
}
|
||||
} else {
|
||||
@@ -639,6 +786,7 @@ pub fn run() {
|
||||
app.manage(bootstrap_state);
|
||||
app.manage(BackendState {
|
||||
process: Mutex::new(None),
|
||||
spawned_at: Mutex::new(None),
|
||||
});
|
||||
|
||||
let app_handle = app.handle().clone();
|
||||
@@ -737,6 +885,14 @@ pub fn run() {
|
||||
|
||||
app.run(|app_handle, event| {
|
||||
if let tauri::RunEvent::ExitRequested { .. } = event {
|
||||
// Raise the quitting flag FIRST: exits that don't pass through the
|
||||
// tray Quit item (macOS ⌘Q, OS session end) would otherwise let a
|
||||
// death watcher observe our own SIGTERM below and record a false
|
||||
// "backend crashed" marker (#941).
|
||||
app_handle
|
||||
.state::<AppFlags>()
|
||||
.quitting
|
||||
.store(true, Ordering::SeqCst);
|
||||
if let Ok(mut lock) = app_handle.state::<BackendState>().process.lock() {
|
||||
if let Some(ref mut child) = *lock {
|
||||
let pid = child.id();
|
||||
|
||||
@@ -85,7 +85,8 @@
|
||||
],
|
||||
"macOS": {
|
||||
"minimumSystemVersion": "12.0",
|
||||
"signingIdentity": "-"
|
||||
"signingIdentity": "-",
|
||||
"entitlements": "entitlements.plist"
|
||||
}
|
||||
},
|
||||
"plugins": {
|
||||
|
||||
+63
-6
@@ -42,6 +42,7 @@ import WorkspaceVoices from './components/WorkspaceVoices';
|
||||
import WorkspaceProjects from './components/WorkspaceProjects';
|
||||
import ErrorBoundary from './components/ErrorBoundary';
|
||||
import FloatingPill from './components/FloatingPill';
|
||||
import BackendCrashNotice from './components/BackendCrashNotice';
|
||||
// RemoteAuthGate is mounted at the true outermost provider in main-app.jsx so
|
||||
// it covers all app states (setup check / wizard / bootstrap), not just the
|
||||
// main studio return below. Do not re-wrap here — double-gating renders two
|
||||
@@ -72,6 +73,7 @@ import {
|
||||
CLONE_MAX_SECONDS,
|
||||
} from './utils/constants';
|
||||
import { LANG_CODES } from './utils/languages';
|
||||
import { restoreProjectExtras } from './utils/projectState';
|
||||
import { API, apiFetch } from './api/client';
|
||||
import { flushMemory as apiFlushMemory } from './api/system';
|
||||
import {
|
||||
@@ -416,9 +418,15 @@ function App() {
|
||||
const defaultTrack = useAppStore((s) => s.defaultTrack);
|
||||
const setDefaultTrack = useAppStore((s) => s.setDefaultTrack);
|
||||
const exportTracks = useAppStore((s) => s.exportTracks);
|
||||
const setExportTracks = useAppStore((s) => s.setExportTracks);
|
||||
const previewSegIds = useAppStore((s) => s.previewSegIds);
|
||||
const speakerClones = useAppStore((s) => s.speakerClones);
|
||||
const setSpeakerClones = useAppStore((s) => s.setSpeakerClones);
|
||||
// Multi-language batch picks (P1.4) — saved with the project payload.
|
||||
const multiLangMode = useAppStore((s) => s.multiLangMode);
|
||||
const setMultiLangMode = useAppStore((s) => s.setMultiLangMode);
|
||||
const multiLangs = useAppStore((s) => s.multiLangs);
|
||||
const setMultiLangs = useAppStore((s) => s.setMultiLangs);
|
||||
|
||||
const setGlossaryTerms = useAppStore((s) => s.setGlossaryTerms);
|
||||
const dualSubs = useAppStore((s) => s.dualSubs);
|
||||
@@ -450,6 +458,8 @@ function App() {
|
||||
closeDirection,
|
||||
saveDirection,
|
||||
setLastGenFingerprints,
|
||||
fingerprintsByLang,
|
||||
setFingerprintsByLang,
|
||||
incrementalPlan,
|
||||
recomputeIncremental,
|
||||
} = useSegmentEditing();
|
||||
@@ -952,6 +962,17 @@ function App() {
|
||||
preserveBg,
|
||||
defaultTrack,
|
||||
speakerClones,
|
||||
// P1.4 — multi-language batch setup + export-track prefs travel with
|
||||
// the project. Additive: loaders default them when absent (see
|
||||
// utils/projectState.js).
|
||||
multiLangMode,
|
||||
multiLangs,
|
||||
exportTracks,
|
||||
// P1.3 — per-language segment fingerprints, so reopening a project
|
||||
// keeps every track's "Regen N changed" plan. Additive: legacy
|
||||
// loaders ignore the key; segments' `translations` maps ride along
|
||||
// inside dubSegments above.
|
||||
segHashesByLang: fingerprintsByLang,
|
||||
},
|
||||
};
|
||||
try {
|
||||
@@ -993,9 +1014,24 @@ function App() {
|
||||
setDubStep(s.dubStep === 'done' ? 'done' : s.dubSegments?.length ? 'editing' : 'idle');
|
||||
// Phase 4.5 — rehydrate per-segment fingerprints. The incremental plan
|
||||
// immediately shows "N segments changed" for any segments edited after
|
||||
// the last generate.
|
||||
setLastGenFingerprints(s.segHashes || {});
|
||||
// the last generate. P1.3: prefer the per-language map; a legacy flat
|
||||
// `segHashes` can only describe the project's saved target language.
|
||||
if (
|
||||
s.segHashesByLang &&
|
||||
typeof s.segHashesByLang === 'object' &&
|
||||
!Array.isArray(s.segHashesByLang)
|
||||
) {
|
||||
setFingerprintsByLang(s.segHashesByLang);
|
||||
} else {
|
||||
setLastGenFingerprints(s.segHashes || {}, s.dubLangCode || 'en');
|
||||
}
|
||||
setSpeakerClones(s.speakerClones || {});
|
||||
// P1.4 — restore multi-lang picks; legacy payloads default to off/empty
|
||||
// and leave the in-session exportTracks untouched (null sentinel).
|
||||
const extras = restoreProjectExtras(s);
|
||||
setMultiLangMode(extras.multiLangMode);
|
||||
setMultiLangs(extras.multiLangs);
|
||||
if (extras.exportTracks) setExportTracks(extras.exportTracks);
|
||||
toast.success(i18n.t('app.toast_opened', { name: data.name }));
|
||||
} catch (err) {
|
||||
toast.error(err.message);
|
||||
@@ -1045,14 +1081,31 @@ function App() {
|
||||
})),
|
||||
);
|
||||
setDubTranscript(job.full_transcript || '');
|
||||
setDubLang(item.language || 'Auto');
|
||||
setDubLangCode(item.language_code || 'und');
|
||||
// Older DBs froze the language/language_code COLUMNS at the ingest-time
|
||||
// "" (the UPSERT didn't update them until #P0 fixed it), but the job_data
|
||||
// JSON always carried the value generation set. Falling back to job_data
|
||||
// restores existing rows correctly without a migration.
|
||||
setDubLang(item.language || job.language || 'Auto');
|
||||
setDubLangCode(item.language_code || job.language_code || 'und');
|
||||
setDubTracks(Object.keys(job.dubbed_tracks || {}));
|
||||
setDubStep(Object.keys(job.dubbed_tracks || {}).length > 0 ? 'done' : 'editing');
|
||||
// Phase 4.5 — seg_hashes are written per successful segment by
|
||||
// dub_generate.py. Reloading a half-generated dub lets the "Regen N
|
||||
// changed" button resume right where the crash happened.
|
||||
setLastGenFingerprints(job.seg_hashes || {});
|
||||
// changed" button resume right where the crash happened. P1.3: prefer
|
||||
// the per-language map (multi-track jobs); a legacy flat map belongs to
|
||||
// the job's last-generated language — the code restored just above.
|
||||
if (
|
||||
job.seg_hashes_by_lang &&
|
||||
typeof job.seg_hashes_by_lang === 'object' &&
|
||||
!Array.isArray(job.seg_hashes_by_lang)
|
||||
) {
|
||||
setFingerprintsByLang(job.seg_hashes_by_lang);
|
||||
} else {
|
||||
setLastGenFingerprints(
|
||||
job.seg_hashes || {},
|
||||
item.language_code || job.language_code || 'und',
|
||||
);
|
||||
}
|
||||
// Rehydrate the auto-extracted speaker clones so the CAST dropdown's
|
||||
// "🎤 From video" option reappears after a reload. Projects that
|
||||
// predate the speaker-clone feature have an empty map; the Extract
|
||||
@@ -1217,6 +1270,10 @@ function App() {
|
||||
|
||||
<FloatingPill />
|
||||
|
||||
{/* #941: honest surfacing of backend process crashes (exit code +
|
||||
stderr tail from the shell's crash marker), with ack-on-view. */}
|
||||
<BackendCrashNotice />
|
||||
|
||||
<Header
|
||||
mode={mode}
|
||||
setMode={setMode}
|
||||
|
||||
@@ -8,6 +8,16 @@
|
||||
// and the API, so a remote device on http://<host>:<share-port> must hit
|
||||
// that same origin — NOT a hardcoded :3900, which is cross-origin (CORS)
|
||||
// and loopback-only/unreachable from another machine.
|
||||
// Explicit .ts extension: tests/frontend/apiClient.test.mjs loads this module
|
||||
// under `node --experimental-strip-types`, whose ESM resolver requires real
|
||||
// file extensions (tsconfig has allowImportingTsExtensions for tsc).
|
||||
import {
|
||||
getUnacknowledgedBackendCrash,
|
||||
describeCrashExit,
|
||||
crashAge,
|
||||
type BackendCrashMarker,
|
||||
} from '../utils/backendCrash.ts';
|
||||
|
||||
const viteEnv = import.meta.env ?? {};
|
||||
// Remote-backend settings (Wave 2.3): user-configured in Settings → Sharing.
|
||||
// localStorage so the choice survives restarts; read once at module load —
|
||||
@@ -143,6 +153,28 @@ export async function apiFetch(path: string, opts: RequestInit = {}): Promise<Re
|
||||
await new Promise((r) => setTimeout(r, TRANSPORT_RETRY_BACKOFF_MS[attempt]));
|
||||
continue;
|
||||
}
|
||||
// #941: if the desktop shell recorded an unacknowledged backend crash,
|
||||
// tell the honest story instead of the vague "can't reach" — and let
|
||||
// BackendCrashNotice raise its "View crash details" affordance.
|
||||
let crash: BackendCrashMarker | null = null;
|
||||
try {
|
||||
crash = await getUnacknowledgedBackendCrash();
|
||||
} catch {
|
||||
/* forensics unavailable — fall through to the generic message */
|
||||
}
|
||||
if (crash) {
|
||||
try {
|
||||
window.dispatchEvent(new CustomEvent('ov:backend-crashed', { detail: crash }));
|
||||
} catch {
|
||||
/* no window (tests) — the ApiError below still tells the story */
|
||||
}
|
||||
throw new ApiError(
|
||||
`The local OmniVoice backend crashed (${describeCrashExit(crash)}) ${crashAge(crash)} ago ` +
|
||||
'and is being restarted — this request could not reach it. ' +
|
||||
'Open the crash notice for the error output, or check Settings → Logs → Backend.',
|
||||
{ status: 0, detail: lastDetail },
|
||||
);
|
||||
}
|
||||
throw new ApiError(
|
||||
"Can't reach the local OmniVoice backend — it may still be starting up, or it stopped. " +
|
||||
'Wait a few seconds and try again; if it persists, restart the app (or check Settings → Logs → Backend).',
|
||||
|
||||
@@ -111,6 +111,23 @@ export async function clearDubHistory(): Promise<Response> {
|
||||
return apiFetch('/dub/history', { method: 'DELETE' });
|
||||
}
|
||||
|
||||
export interface DubTrackInfo {
|
||||
path?: string;
|
||||
language?: string;
|
||||
language_code?: string;
|
||||
duration?: number;
|
||||
timing_strategy?: string;
|
||||
}
|
||||
|
||||
/** Per-track metadata (duration, timing strategy, …) keyed by language code.
|
||||
* Backs the track-pill tooltips; the store only carries the track codes. */
|
||||
export async function dubListTracks(jobId: string): Promise<Record<string, DubTrackInfo>> {
|
||||
const res = await apiJson<{ tracks?: Record<string, DubTrackInfo> }>(
|
||||
`/dub/tracks/${encodeURIComponent(jobId)}`,
|
||||
);
|
||||
return res?.tracks || {};
|
||||
}
|
||||
|
||||
export interface DubQCResponse {
|
||||
engine: string;
|
||||
total: number;
|
||||
|
||||
@@ -46,8 +46,15 @@ export async function listEngines(): Promise<AllEnginesResponse> {
|
||||
export async function selectEngine(
|
||||
family: EngineFamily,
|
||||
backendId: string,
|
||||
modelId?: string,
|
||||
): Promise<SelectEngineResponse> {
|
||||
return apiPost<SelectEngineResponse>('/engines/select', { family, backend_id: backendId });
|
||||
return apiPost<SelectEngineResponse>('/engines/select', {
|
||||
family,
|
||||
backend_id: backendId,
|
||||
// Only mlx-audio's curated-model picker (#981) sets this — omit
|
||||
// entirely rather than send `undefined`/null for every other call site.
|
||||
...(modelId ? { model_id: modelId } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -40,6 +40,19 @@ interface EngineBackend {
|
||||
effective_device?: EffectiveDevice;
|
||||
routing_status?: RoutingStatus;
|
||||
routing_reason?: string | null;
|
||||
// #981 — mlx-audio ONLY: it multiplexes 7+ curated models behind one
|
||||
// backend id, so its entry also carries the roster + current pick so
|
||||
// Settings can render a model picker. Absent on every other backend.
|
||||
curated_models?: CuratedModel[];
|
||||
active_model_id?: string;
|
||||
}
|
||||
|
||||
// #981 — one of mlx-audio's curated models (see backend
|
||||
// MLXAudioBackend.CURATED_MODELS / _MLX_AUDIO_MODEL_LABELS).
|
||||
export interface CuratedModel {
|
||||
key: string;
|
||||
label: string;
|
||||
repo_id: string;
|
||||
}
|
||||
|
||||
interface EngineFamilyResponse {
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
import { useCallback, useEffect, useState } from 'react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { AlertTriangle, X } from 'lucide-react';
|
||||
import { Button, Dialog } from '../ui';
|
||||
import {
|
||||
acknowledgeBackendCrash,
|
||||
crashAge,
|
||||
describeCrashExit,
|
||||
getUnacknowledgedBackendCrash,
|
||||
} from '../utils/backendCrash';
|
||||
import { openExternal } from '../api/external';
|
||||
import { buildBugReportUrl } from '../utils/bugReport';
|
||||
|
||||
/**
|
||||
* BackendCrashNotice — the honest half of #941.
|
||||
*
|
||||
* When the backend PROCESS dies, the desktop shell records a crash marker
|
||||
* (src-tauri/src/crash.rs). This component surfaces it: a banner naming the
|
||||
* exit code and when it happened, with a "View crash details" affordance that
|
||||
* shows the captured stderr tail and a report path. Sources:
|
||||
* - `ov:backend-crashed` window events, dispatched by api/client.ts when a
|
||||
* request fails against a freshly crashed backend, and
|
||||
* - a mount-time check, so a crash that happened with no request in flight
|
||||
* (or a crash-loop that forced an app restart) still gets told.
|
||||
*
|
||||
* Viewing or dismissing acknowledges the marker (it is retained on disk so
|
||||
* bug reports can still attach the evidence). Outside the Tauri shell the
|
||||
* marker getters resolve null and this renders nothing.
|
||||
*/
|
||||
export default function BackendCrashNotice() {
|
||||
const { t } = useTranslation();
|
||||
const [marker, setMarker] = useState(null);
|
||||
const [showDetails, setShowDetails] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
getUnacknowledgedBackendCrash()
|
||||
.then((m) => {
|
||||
if (!cancelled && m) setMarker(m);
|
||||
})
|
||||
.catch(() => {});
|
||||
const onCrash = (e) => {
|
||||
if (e?.detail) setMarker(e.detail);
|
||||
};
|
||||
window.addEventListener('ov:backend-crashed', onCrash);
|
||||
return () => {
|
||||
cancelled = true;
|
||||
window.removeEventListener('ov:backend-crashed', onCrash);
|
||||
};
|
||||
}, []);
|
||||
|
||||
const view = useCallback(() => {
|
||||
setShowDetails(true);
|
||||
// Ack on view — the user has seen the honest story; the marker itself
|
||||
// stays on disk for bug-report attachment.
|
||||
acknowledgeBackendCrash().catch(() => {});
|
||||
}, []);
|
||||
|
||||
const dismiss = useCallback(() => {
|
||||
acknowledgeBackendCrash().catch(() => {});
|
||||
setShowDetails(false);
|
||||
setMarker(null);
|
||||
}, []);
|
||||
|
||||
if (!marker) return null;
|
||||
|
||||
const exit = describeCrashExit(marker);
|
||||
const ago = crashAge(marker);
|
||||
|
||||
return (
|
||||
<>
|
||||
<div
|
||||
role="alert"
|
||||
className="fixed left-1/2 top-[var(--space-4)] z-[70] flex w-[min(600px,92vw)] -translate-x-1/2 items-center gap-[var(--space-3)] rounded-lg border border-border bg-bg-elev-1 px-[var(--space-4)] py-[var(--space-3)] shadow-lg backdrop-blur-md"
|
||||
>
|
||||
<AlertTriangle size={16} className="shrink-0 text-danger" aria-hidden />
|
||||
<span className="flex-1 text-[length:var(--text-sm)] text-fg">
|
||||
{t('crash.notice', { exit, ago })}
|
||||
</span>
|
||||
<Button variant="subtle" size="sm" onClick={view}>
|
||||
{t('crash.view')}
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
iconSize="sm"
|
||||
onClick={dismiss}
|
||||
title={t('crash.dismiss')}
|
||||
>
|
||||
<X size={12} />
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<Dialog
|
||||
open={showDetails}
|
||||
onClose={() => {
|
||||
setShowDetails(false);
|
||||
setMarker(null);
|
||||
}}
|
||||
title={t('crash.details_title')}
|
||||
size="lg"
|
||||
footer={
|
||||
<>
|
||||
<Button
|
||||
variant="subtle"
|
||||
onClick={async () => {
|
||||
try {
|
||||
// buildBugReportUrl attaches the crash marker (exit code +
|
||||
// scrubbed stderr tail) automatically — the report arrives
|
||||
// WITH the evidence.
|
||||
await openExternal(
|
||||
await buildBugReportUrl({ title: `[Crash] Backend died (${exit})` }),
|
||||
);
|
||||
} catch (e) {
|
||||
console.warn('[BackendCrashNotice] report action failed', e);
|
||||
}
|
||||
}}
|
||||
>
|
||||
{t('errors.report')}
|
||||
</Button>
|
||||
<Button variant="primary" onClick={dismiss}>
|
||||
{t('common.close')}
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-[var(--space-4)]">
|
||||
<p className="m-0 text-[length:var(--text-sm)] text-fg-muted">
|
||||
{t('crash.details_intro', { exit, ago })}
|
||||
</p>
|
||||
<dl className="m-0 grid grid-cols-[max-content_1fr] gap-x-[var(--space-5)] gap-y-[var(--space-2)] text-[length:var(--text-sm)]">
|
||||
<dt className="text-fg-subtle">{t('crash.field_exit')}</dt>
|
||||
<dd className="m-0 font-mono text-fg">{exit}</dd>
|
||||
<dt className="text-fg-subtle">{t('crash.field_when')}</dt>
|
||||
<dd className="m-0 text-fg">{new Date(marker.ts * 1000).toLocaleString()}</dd>
|
||||
<dt className="text-fg-subtle">{t('crash.field_uptime')}</dt>
|
||||
<dd className="m-0 text-fg">{t('crash.uptime_value', { count: marker.uptime_s })}</dd>
|
||||
<dt className="text-fg-subtle">{t('crash.field_version')}</dt>
|
||||
<dd className="m-0 text-fg">{marker.backend_version}</dd>
|
||||
</dl>
|
||||
<div>
|
||||
<div className="mb-[var(--space-2)] text-[length:var(--text-sm)] text-fg-subtle">
|
||||
{t('crash.stderr_title')}
|
||||
</div>
|
||||
<pre className="m-0 max-h-[40vh] overflow-auto rounded-md border border-border bg-bg-elev-2 p-[var(--space-3)] font-mono text-[length:var(--text-xs)] leading-relaxed text-fg whitespace-pre-wrap">
|
||||
{marker.last_stderr || t('crash.no_stderr')}
|
||||
</pre>
|
||||
</div>
|
||||
</div>
|
||||
</Dialog>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
|
||||
import BackendCrashNotice from './BackendCrashNotice';
|
||||
import { acknowledgeBackendCrash, getUnacknowledgedBackendCrash } from '../utils/backendCrash';
|
||||
|
||||
// #941: the crash-notice branch — a recorded backend death must surface the
|
||||
// honest message (exit code + age) with a "View crash details" affordance,
|
||||
// and viewing/dismissing must acknowledge the marker.
|
||||
vi.mock('../utils/backendCrash', async (importOriginal) => {
|
||||
const actual = await importOriginal();
|
||||
return {
|
||||
...actual,
|
||||
getUnacknowledgedBackendCrash: vi.fn().mockResolvedValue(null),
|
||||
acknowledgeBackendCrash: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
});
|
||||
vi.mock('../utils/bugReport', () => ({
|
||||
buildBugReportUrl: vi.fn().mockResolvedValue('https://example.test/issues/new'),
|
||||
}));
|
||||
vi.mock('../api/external', () => ({
|
||||
openExternal: vi.fn().mockResolvedValue(undefined),
|
||||
}));
|
||||
|
||||
const MARKER = {
|
||||
ts: Math.floor(Date.now() / 1000) - 12,
|
||||
exit_code: 134,
|
||||
signal: null,
|
||||
exit_desc: 'exit status: 134',
|
||||
backend_version: '0.3.10',
|
||||
uptime_s: 87,
|
||||
last_stderr: 'CUDA error: an illegal memory access was encountered',
|
||||
acknowledged: false,
|
||||
};
|
||||
|
||||
describe('BackendCrashNotice', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
getUnacknowledgedBackendCrash.mockResolvedValue(null);
|
||||
});
|
||||
|
||||
it('renders nothing when the shell reports no crash', async () => {
|
||||
const { container } = render(<BackendCrashNotice />);
|
||||
await waitFor(() => expect(getUnacknowledgedBackendCrash).toHaveBeenCalled());
|
||||
expect(container).toBeEmptyDOMElement();
|
||||
});
|
||||
|
||||
it('shows the honest message and the details affordance for a fresh marker', async () => {
|
||||
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
|
||||
render(<BackendCrashNotice />);
|
||||
const alert = await screen.findByRole('alert');
|
||||
// Honest: names the exit code instead of a vague "can't reach".
|
||||
expect(alert.textContent).toContain('crashed');
|
||||
expect(alert.textContent).toContain('exit code 134');
|
||||
expect(screen.getByRole('button', { name: /view crash details/i })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('surfaces a crash pushed via the ov:backend-crashed event', async () => {
|
||||
render(<BackendCrashNotice />);
|
||||
await waitFor(() => expect(getUnacknowledgedBackendCrash).toHaveBeenCalled());
|
||||
window.dispatchEvent(new CustomEvent('ov:backend-crashed', { detail: MARKER }));
|
||||
const alert = await screen.findByRole('alert');
|
||||
expect(alert.textContent).toContain('exit code 134');
|
||||
});
|
||||
|
||||
it('acks on view and shows the stderr tail in the details dialog', async () => {
|
||||
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
|
||||
render(<BackendCrashNotice />);
|
||||
fireEvent.click(await screen.findByRole('button', { name: /view crash details/i }));
|
||||
expect(acknowledgeBackendCrash).toHaveBeenCalledTimes(1);
|
||||
expect(await screen.findByText(/illegal memory access/)).toBeInTheDocument();
|
||||
expect(screen.getByText('Backend crash details')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('ack + clear on dismiss', async () => {
|
||||
getUnacknowledgedBackendCrash.mockResolvedValue(MARKER);
|
||||
render(<BackendCrashNotice />);
|
||||
await screen.findByRole('alert');
|
||||
fireEvent.click(screen.getByRole('button', { name: /dismiss/i }));
|
||||
expect(acknowledgeBackendCrash).toHaveBeenCalledTimes(1);
|
||||
await waitFor(() => expect(screen.queryByRole('alert')).not.toBeInTheDocument());
|
||||
});
|
||||
});
|
||||
@@ -17,7 +17,7 @@ import { useTranslation } from 'react-i18next';
|
||||
import { listEngines, getEngineHealth, selfTestEngine } from '../api/engines';
|
||||
import { copyText } from '../utils/copyText';
|
||||
import { ChevronRight } from 'lucide-react';
|
||||
import { Badge, Button, Segmented, Table } from '../ui';
|
||||
import { Badge, Button, Segmented, Select, Table } from '../ui';
|
||||
import { cn } from '@/lib/utils';
|
||||
import SupertonicLicenseDialog from './SupertonicLicenseDialog';
|
||||
|
||||
@@ -63,12 +63,18 @@ function reasonMentionsLicense(reason) {
|
||||
*
|
||||
* Props:
|
||||
* - family: 'tts' | 'asr' | 'llm' default 'tts'
|
||||
* - onSelect?: (family, backendId) => Promise<void> optional — when
|
||||
* provided, a "Use" button appears next to "Test engine" for
|
||||
* - onSelect?: (family, backendId, modelId?) => Promise<void> optional —
|
||||
* when provided, a "Use" button appears next to "Test engine" for
|
||||
* available, non-active rows. Lets the matrix double as an engine
|
||||
* picker so Settings doesn't need a parallel table.
|
||||
* picker so Settings doesn't need a parallel table. The optional third
|
||||
* arg is set only by mlx-audio's curated-model picker (#981).
|
||||
* - activeId?: string the currently-active backend id for this
|
||||
* family. Used to render the "active" badge.
|
||||
* - showFamilyTabs?: boolean default true. When false, the matrix is
|
||||
* pinned to `family` — no TTS/ASR/LLM switcher, and the header names
|
||||
* the family ("ASR Engines") instead of the generic matrix title.
|
||||
* Settings → Engines stacks one pinned matrix per family so the ASR
|
||||
* and LLM pickers are visible instead of tucked behind a tab.
|
||||
*/
|
||||
const FAMILY_META = {
|
||||
tts: { label: 'TTS', icon: Cpu },
|
||||
@@ -139,6 +145,10 @@ function normalizeEntry(entry) {
|
||||
effective_device: entry.effective_device || null,
|
||||
routing_status: entry.routing_status || null,
|
||||
routing_reason: entry.routing_reason || null,
|
||||
// #981 — mlx-audio ONLY: the curated-model roster + current pick.
|
||||
// null/absent on every other backend, which never renders a picker.
|
||||
curated_models: Array.isArray(entry.curated_models) ? entry.curated_models : null,
|
||||
active_model_id: entry.active_model_id || null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -153,8 +163,10 @@ export default function EngineCompatibilityMatrix({
|
||||
family = 'tts',
|
||||
onSelect = null,
|
||||
activeId = null,
|
||||
// Test-friendly overrides — let the RTL suite mock the API layer
|
||||
// without resorting to module-level vi.mock incantations.
|
||||
showFamilyTabs = true,
|
||||
// Injectable API layer — lets the RTL suite mock it without module-level
|
||||
// vi.mock incantations, and lets EnginesTab share one in-flight
|
||||
// GET /engines across its stacked per-family matrices.
|
||||
apiListEngines = listEngines,
|
||||
apiGetEngineHealth = getEngineHealth,
|
||||
apiSelfTestEngine = selfTestEngine,
|
||||
@@ -291,6 +303,18 @@ export default function EngineCompatibilityMatrix({
|
||||
setTimeout(() => setCopiedId((c) => (c === id ? null : c)), 1500);
|
||||
}, []);
|
||||
|
||||
// #981 — mlx-audio's curated-model picker. Reuses the same onSelect the
|
||||
// "Use" button calls, with the curated model key as the optional third
|
||||
// arg, then reloads so active_model_id reflects the new pick immediately.
|
||||
const changeModel = useCallback(
|
||||
async (id, modelId) => {
|
||||
if (!onSelect || !modelId) return;
|
||||
await onSelect(activeFamily, id, modelId);
|
||||
reload();
|
||||
},
|
||||
[onSelect, activeFamily, reload],
|
||||
);
|
||||
|
||||
const COLUMNS = [
|
||||
{ key: 'name', label: t('engines.matrixTitle').split(' ')[0] || 'Engine', flex: 3 },
|
||||
{ key: 'status', label: t('engines.status'), width: 130, align: 'center' },
|
||||
@@ -330,12 +354,19 @@ export default function EngineCompatibilityMatrix({
|
||||
// TTS-05: the license dialog registered for the engine awaiting acceptance
|
||||
// (or null). Capitalized so JSX renders it as a component below.
|
||||
const LicenseDialog = licenseDialogFor ? LICENSE_DIALOGS[licenseDialogFor] : null;
|
||||
// Pinned mode: the header names the family (with its icon) since there is
|
||||
// no switcher to say which family this table is.
|
||||
const familyMeta = FAMILY_META[activeFamily] || FAMILY_META.tts;
|
||||
const TitleIcon = showFamilyTabs ? Layers : familyMeta.icon;
|
||||
|
||||
return (
|
||||
<section className="engine-matrix flex flex-col gap-[var(--space-3,8px)]">
|
||||
<header className="engine-matrix__head flex items-center justify-between gap-[12px]">
|
||||
<h3 className="engine-matrix__title inline-flex items-center gap-[6px] m-0 text-[13px] font-semibold text-[color:var(--chrome-fg,currentColor)]">
|
||||
<Layers size={14} /> {t('engines.matrixTitle')}
|
||||
<TitleIcon size={14} />{' '}
|
||||
{showFamilyTabs
|
||||
? t('engines.matrixTitle')
|
||||
: t('engines.familyMatrixTitle', { family: familyMeta.label })}
|
||||
</h3>
|
||||
<Button
|
||||
size="sm"
|
||||
@@ -348,7 +379,7 @@ export default function EngineCompatibilityMatrix({
|
||||
</Button>
|
||||
</header>
|
||||
|
||||
{families.length > 1 && (
|
||||
{showFamilyTabs && families.length > 1 && (
|
||||
<Segmented
|
||||
size="sm"
|
||||
value={activeFamily}
|
||||
@@ -413,6 +444,35 @@ export default function EngineCompatibilityMatrix({
|
||||
<code className="engine-matrix__id font-mono text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
|
||||
{b.id}
|
||||
</code>
|
||||
{/* #981 — mlx-audio multiplexes 7+ curated models behind this
|
||||
one backend id (Kokoro, CSM, OuteTTS, …); without this
|
||||
picker there's no way to load anything but the default
|
||||
(Kokoro) even after downloading a different model's
|
||||
weights in Settings → Models. Disabled while the row
|
||||
itself isn't available/selectable, matching the "Use"
|
||||
button's gating. */}
|
||||
{b.curated_models && b.curated_models.length > 0 && (
|
||||
<div className="engine-matrix__model-picker flex items-center gap-[6px] mt-[2px]">
|
||||
<span className="text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
|
||||
{t('engines.curatedModelLabel')}
|
||||
</span>
|
||||
<Select
|
||||
size="sm"
|
||||
className="w-auto min-w-[150px]"
|
||||
value={b.active_model_id || ''}
|
||||
disabled={!onSelect || !b.available}
|
||||
onChange={(e) => changeModel(b.id, e.target.value)}
|
||||
aria-label={t('engines.curatedModelAria', { engine: b.display_name })}
|
||||
data-testid={`curated-model-select-${b.id}`}
|
||||
>
|
||||
{b.curated_models.map((m) => (
|
||||
<option key={m.key} value={m.key}>
|
||||
{m.label}
|
||||
</option>
|
||||
))}
|
||||
</Select>
|
||||
</div>
|
||||
)}
|
||||
{/* For available rows, show install_hint inline (one line — usually
|
||||
a parenthetical like "(bundled — no extra install needed)").
|
||||
For unavailable rows, collapse reason + install_hint + last_error
|
||||
|
||||
@@ -259,7 +259,11 @@ export default function ExportModal({
|
||||
onClose?.();
|
||||
};
|
||||
const runClips = () => {
|
||||
handleAudioExport?.(`${API}/dub/export-segments/${jobId}`, 'segments.zip');
|
||||
// Ask for the ACTIVE track's per-segment clips (P1.3 — the cache is
|
||||
// language-keyed now); omitted lang falls back to the last-generated
|
||||
// track server-side, which is all a legacy single-track job has.
|
||||
const langQ = dubLangCode ? `?lang=${encodeURIComponent(dubLangCode)}` : '';
|
||||
handleAudioExport?.(`${API}/dub/export-segments/${jobId}${langQ}`, 'segments.zip');
|
||||
onClose?.();
|
||||
};
|
||||
|
||||
|
||||
@@ -376,6 +376,13 @@ export default function Header({
|
||||
<div className="flex flex-col gap-[1px] min-w-0">
|
||||
<span className="text-[12px] text-[var(--color-fg)] font-medium">
|
||||
{m.name}
|
||||
{/* Resident-but-not-routed engine (e.g. OmniVoice still in
|
||||
VRAM after switching to another backend) — say so. */}
|
||||
{m.is_active_engine === false && (
|
||||
<span className="ml-[6px] text-[10px] font-normal text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
|
||||
{t('header.model_not_active')}
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<span className="text-[10px] text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
|
||||
{m.device} {m.vram_mb > 0 ? `· ${m.vram_mb.toFixed(0)} MB` : ''}
|
||||
|
||||
@@ -73,9 +73,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
[t],
|
||||
);
|
||||
|
||||
const donateLabel = t('donate.pill', { defaultValue: 'Support OmniVoice' });
|
||||
const donateActive = mode === 'donate';
|
||||
|
||||
// `nav-rail` is retained purely as the layout hook the (out-of-scope)
|
||||
// `.app-container > .nav-rail` grid rules position by; all visual styling now
|
||||
// lives in the utilities below. Border flips to the inner edge when on the right.
|
||||
@@ -84,17 +81,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
? '[border-left:1px_solid_var(--chrome-border)]'
|
||||
: '[border-right:1px_solid_var(--chrome-border)]';
|
||||
|
||||
// Quiet "Support" pill (was `.rail-btn.donate-pill`): neutral at rest, warms to
|
||||
// the accent on hover/active.
|
||||
const donateState = donateActive
|
||||
? 'text-[var(--chrome-accent)] bg-[var(--chrome-accent-bg)] [border:1px_solid_var(--chrome-accent-border)]'
|
||||
: 'bg-transparent text-[var(--chrome-fg-dim)] [border:1px_solid_transparent] hover:bg-[color-mix(in_srgb,var(--chrome-accent)_10%,transparent)] hover:text-[var(--chrome-accent)]';
|
||||
const heartBase =
|
||||
'text-[16px] leading-none [transition:filter_0.16s,opacity_0.16s,transform_0.16s] group-hover:[transform:scale(1.1)] motion-reduce:[transition:none] motion-reduce:group-hover:[transform:none]';
|
||||
const heartState = donateActive
|
||||
? 'opacity-100 [filter:grayscale(0)]'
|
||||
: 'opacity-75 [filter:grayscale(0.55)] group-hover:opacity-100 group-hover:[filter:grayscale(0)]';
|
||||
|
||||
return (
|
||||
<aside
|
||||
className={`nav-rail z-50 flex select-none flex-col items-center gap-[6px] bg-[var(--chrome-bg)] py-[8px] ${asideBorder}`}
|
||||
@@ -111,19 +97,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
))}
|
||||
</div>
|
||||
<div className="flex flex-col items-center gap-[4px]">
|
||||
{/* Quiet "Support" pill — warms to the accent on hover, opens the
|
||||
donate page. Sits with the footer nav (Settings / flip). (#007) */}
|
||||
<button
|
||||
onClick={() => setMode('donate')}
|
||||
title={donateLabel}
|
||||
aria-label={donateLabel}
|
||||
className={`${RAIL_BTN_BASE} ${donateState}`}
|
||||
>
|
||||
<span className={`${heartBase} ${heartState}`} aria-hidden="true">
|
||||
🩷
|
||||
</span>
|
||||
<span className={railLabelCls(side)}>{donateLabel}</span>
|
||||
</button>
|
||||
{footerItems.map((it) => (
|
||||
<RailBtn
|
||||
key={it.id}
|
||||
|
||||
@@ -1,8 +1,17 @@
|
||||
import React, { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from 'react';
|
||||
import React, {
|
||||
useCallback,
|
||||
useEffect,
|
||||
useLayoutEffect,
|
||||
useMemo,
|
||||
useRef,
|
||||
useState,
|
||||
useSyncExternalStore,
|
||||
} from 'react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { Play, Headphones } from 'lucide-react';
|
||||
import {
|
||||
REGION_COLORS,
|
||||
getRegionColors,
|
||||
subscribeRegionColors,
|
||||
SNAP_PX,
|
||||
visibleSegmentRange,
|
||||
snapTime,
|
||||
@@ -135,13 +144,14 @@ export default function SegmentTrack({
|
||||
[effSegments, viewStart, viewEnd],
|
||||
);
|
||||
|
||||
// Palette snapshot re-blends against the new --chrome-bg on theme change
|
||||
// (#963) — new array identity per re-blend, so the memo below recolors.
|
||||
const regionColors = useSyncExternalStore(subscribeRegionColors, getRegionColors);
|
||||
const speakerColor = useMemo(() => {
|
||||
const speakers = [...new Set(segments.map((s) => s.speaker_id).filter(Boolean))];
|
||||
const bySpeaker = new Map(
|
||||
speakers.map((sp, i) => [sp, REGION_COLORS[i % REGION_COLORS.length]]),
|
||||
);
|
||||
return (seg, idx) => bySpeaker.get(seg.speaker_id) || REGION_COLORS[idx % REGION_COLORS.length];
|
||||
}, [segments]);
|
||||
const bySpeaker = new Map(speakers.map((sp, i) => [sp, regionColors[i % regionColors.length]]));
|
||||
return (seg, idx) => bySpeaker.get(seg.speaker_id) || regionColors[idx % regionColors.length];
|
||||
}, [segments, regionColors]);
|
||||
|
||||
// ── Onset tick strip (one viewport-sized canvas, non-interactive) ───────
|
||||
useEffect(() => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render, screen, fireEvent } from '@testing-library/react';
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { render, screen, fireEvent, act } from '@testing-library/react';
|
||||
import SegmentTrack from './SegmentTrack';
|
||||
|
||||
// Mocked transport: fixed pxPerSec/scrollLeft, no WaveSurfer. jsdom has no
|
||||
@@ -211,6 +211,46 @@ describe('SegmentTrack — compositor-safe positioning (#373)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('SegmentTrack — engine-independent box paint (#963)', () => {
|
||||
const root = document.documentElement;
|
||||
|
||||
afterEach(async () => {
|
||||
// Restore the default theme and let the palette observer settle so the
|
||||
// module-level cache can't leak into other tests in this file.
|
||||
await act(async () => {
|
||||
root.style.removeProperty('--chrome-bg');
|
||||
root.removeAttribute('data-theme');
|
||||
await new Promise((resolve) => setTimeout(resolve, 0));
|
||||
});
|
||||
});
|
||||
|
||||
it('inline background is a literal opaque rgb() — no color-mix/var() the CSSOM could reject', () => {
|
||||
// WebView2/Chromium < 111 rejects a color-mix() inline-style assignment
|
||||
// wholesale, and .seg-track__box declares no fallback background → the
|
||||
// boxes rendered fully transparent (#963). The inline value must be
|
||||
// plain rgb() so every engine parses it.
|
||||
setup();
|
||||
for (const el of screen.getAllByRole('option')) {
|
||||
expect(el.style.background).toMatch(/^rgb\(\d{1,3}, \d{1,3}, \d{1,3}\)$/);
|
||||
}
|
||||
// Default theme, first palette slot: 0.45·rgb(211,134,155) over #0f1011.
|
||||
expect(box(0).style.background).toBe('rgb(103, 69, 79)');
|
||||
});
|
||||
|
||||
it('boxes re-blend live when the theme changes ([data-theme] on <html>)', async () => {
|
||||
setup();
|
||||
expect(box(0).style.background).toBe('rgb(103, 69, 79)');
|
||||
await act(async () => {
|
||||
// Same seam App.jsx uses: swap --chrome-bg and flag the theme.
|
||||
root.style.setProperty('--chrome-bg', '#1e293b');
|
||||
root.setAttribute('data-theme', 'slate');
|
||||
await new Promise((resolve) => setTimeout(resolve, 0)); // flush MutationObserver
|
||||
});
|
||||
// round(0.45·[211,134,155] + 0.55·[30,41,59])
|
||||
expect(box(0).style.background).toBe('rgb(111, 83, 102)');
|
||||
});
|
||||
});
|
||||
|
||||
describe('SegmentTrack — pointer + selection', () => {
|
||||
it('pointerdown selects the segment (table sync)', () => {
|
||||
const { onSelectSeg } = setup();
|
||||
|
||||
@@ -351,7 +351,14 @@ function WaveformTimeline(
|
||||
console.warn('WebKit audio decode not supported, using media element directly');
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
// Don't rely solely on the 'ready' event firing again for this
|
||||
// recovery load — the play button stayed permanently disabled
|
||||
// when it didn't (the waveform still rendered from the peaks, so
|
||||
// there was no visible sign anything was wrong). Confirm
|
||||
// readiness explicitly once this load settles either way.
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setReady(true);
|
||||
}
|
||||
@@ -372,7 +379,12 @@ function WaveformTimeline(
|
||||
})
|
||||
.then((audioBuffer) => {
|
||||
const channelData = audioBuffer.getChannelData(0);
|
||||
ws.load(undefined, [channelData], audioBuffer.duration);
|
||||
// Same explicit-readiness guard as the NotSupportedError branch
|
||||
// above — don't depend on the 'ready' event re-firing for this
|
||||
// manually-decoded recovery load.
|
||||
Promise.resolve(ws.load(undefined, [channelData], audioBuffer.duration))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
})
|
||||
.catch((decodeErr) => {
|
||||
// HTTP 404 on the companion audio means the source file is
|
||||
@@ -391,7 +403,9 @@ function WaveformTimeline(
|
||||
console.warn('Audio decode fallback failed, loading with empty peaks:', decodeErr);
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setLoadError(true);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
// Regression guard: the dub editor's play button stayed permanently disabled
|
||||
// (disabled={!ready}) whenever the initial WaveSurfer decode failed and the
|
||||
// component fell back to a peaks-only ws.load(undefined, [peaks], duration)
|
||||
// call — the waveform still rendered from those peaks (so nothing looked
|
||||
// visibly broken), but `ready` was only ever set from the 'ready' event
|
||||
// re-firing on that recovery load, which this component's own error-handling
|
||||
// code never actually confirmed. Each fallback load must now explicitly
|
||||
// confirm readiness once it settles, instead of assuming the event fires.
|
||||
//
|
||||
// Driving WaveSurfer + a real decode-failure/recovery sequence through jsdom
|
||||
// is brittle (see WaveformTimeline.unlock.test.js), so this is a
|
||||
// source-level contract guard, same house pattern: every `ws.load(undefined,
|
||||
// ...)` recovery call inside the `ws.on('error', ...)` handler must be
|
||||
// followed by an explicit setReady(true) confirmation.
|
||||
|
||||
const src = readFileSync(
|
||||
path.resolve(process.cwd(), 'src/components/WaveformTimeline.jsx'),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
describe('WaveformTimeline error-recovery ready confirmation', () => {
|
||||
it("confirms readiness explicitly after every fallback ws.load() call, not just via the 'ready' event", () => {
|
||||
const errorHandler = /ws\.on\('error', \(err\) => \{([\s\S]*?)\n \}\);/.exec(src)?.[1];
|
||||
expect(errorHandler, "ws.on('error', ...) handler not found").toBeTruthy();
|
||||
|
||||
// Every recovery load in this handler passes peaks explicitly
|
||||
// (`ws.load(undefined, [...], ...)`) — each occurrence must be
|
||||
// immediately confirmed ready via a .then()/.catch() pair (or an
|
||||
// unconditional setReady in a synchronous catch), not left to hope the
|
||||
// 'ready' event re-fires on its own.
|
||||
const loadCalls = [...errorHandler.matchAll(/ws\.load\(undefined, \[[^\]]*\][^)]*\)/g)];
|
||||
expect(loadCalls.length).toBeGreaterThanOrEqual(3);
|
||||
|
||||
for (const match of loadCalls) {
|
||||
const tail = errorHandler.slice(match.index, match.index + 220);
|
||||
expect(tail, `no readiness confirmation after: ${match[0]}`).toMatch(/setReady\(true\)/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -147,6 +147,13 @@ export default function DesignMethodPanel({
|
||||
{Object.entries(CATEGORIES).map(([key, options]) => {
|
||||
const many = options.length > 6;
|
||||
const optLabel = (val) => {
|
||||
// #983: a profile/localStorage-restored vdStates can carry a
|
||||
// partial shape (missing category keys) — val is then undefined
|
||||
// here even though the 'Auto' check above only catches the
|
||||
// literal sentinel. Guard before .replace() rather than crash;
|
||||
// 'Auto' matches how the rest of the component (the ternary
|
||||
// above, the chip/select fallbacks) treats an unset category.
|
||||
if (typeof val !== 'string' || !val) return 'Auto';
|
||||
const tKey = `clone.opt_${val.replace(/[ -]/g, '_')}`;
|
||||
const tl = t(tKey);
|
||||
return tl !== tKey ? tl : val;
|
||||
@@ -180,9 +187,13 @@ export default function DesignMethodPanel({
|
||||
aria-label={t(`clone.cat_${key}`)}
|
||||
>
|
||||
{options.map((opt, i) => {
|
||||
const optTKey = `clone.opt_${opt.replace(/[ -]/g, '_')}`;
|
||||
// `opt` is always a hardcoded CATEGORIES string today,
|
||||
// never undefined — guarded anyway for consistency with
|
||||
// the identical pattern above (#983).
|
||||
const safeOpt = typeof opt === 'string' && opt ? opt : 'Auto';
|
||||
const optTKey = `clone.opt_${safeOpt.replace(/[ -]/g, '_')}`;
|
||||
const optTl = t(optTKey);
|
||||
const optLabel = optTl !== optTKey ? optTl : opt;
|
||||
const optLabel = optTl !== optTKey ? optTl : safeOpt;
|
||||
const checked = vdStates[key] === opt;
|
||||
// Roving tabindex: the checked chip is the group's
|
||||
// single tab stop (first chip if nothing matches).
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import DesignMethodPanel from './DesignMethodPanel';
|
||||
|
||||
// #983: "Cannot read properties of undefined (reading 'replace')" — the
|
||||
// identity panel crashed whenever vdStates was missing one of the 6
|
||||
// CATEGORIES keys (a design profile saved by an older/foreign client, or a
|
||||
// stale localStorage shape). The label helper called `val.replace(...)` on
|
||||
// an undefined category value. This regression-tests the render guard added
|
||||
// to DesignMethodPanel.jsx directly, independent of the upstream data-shape
|
||||
// fixes in useProfiles.js / useAppData.js / profiles.py.
|
||||
|
||||
// A minimal i18next-compatible mock: returns the defaultValue if given, else
|
||||
// echoes the key back (mirrors i18next's behavior for a missing translation,
|
||||
// which is what `optLabel`'s `tl !== tKey` check relies on).
|
||||
const t = (key, opts) => opts?.defaultValue ?? key;
|
||||
|
||||
function setup(vdStates, props = {}) {
|
||||
return render(
|
||||
<DesignMethodPanel
|
||||
t={t}
|
||||
describeText=""
|
||||
onDescribeChange={vi.fn()}
|
||||
describeMatchedAny={false}
|
||||
describeUnmatched={[]}
|
||||
chipPersonalities={[]}
|
||||
activePersonality={null}
|
||||
applyPersonality={vi.fn()}
|
||||
applyPreset={vi.fn()}
|
||||
identityOpen={true}
|
||||
setIdentityOpen={vi.fn()}
|
||||
identityRecipe="test recipe"
|
||||
vdStates={vdStates}
|
||||
setVdStates={vi.fn()}
|
||||
onChipKeyDown={vi.fn()}
|
||||
showSaveProfile={false}
|
||||
setShowSaveProfile={vi.fn()}
|
||||
profileName=""
|
||||
setProfileName={vi.fn()}
|
||||
handleSaveDesignProfile={vi.fn()}
|
||||
instruct=""
|
||||
language="Auto"
|
||||
{...props}
|
||||
/>,
|
||||
);
|
||||
}
|
||||
|
||||
describe('DesignMethodPanel — #983 partial vdStates crash', () => {
|
||||
it('does not throw when vdStates is missing 5 of the 6 CATEGORIES keys', () => {
|
||||
// Only Gender is set — Age, Pitch, Style, EnglishAccent, ChineseDialect
|
||||
// are all undefined, exercising both the chip-based and <select>-based
|
||||
// ("many" options) render paths.
|
||||
expect(() => setup({ Gender: 'male' })).not.toThrow();
|
||||
});
|
||||
|
||||
it('does not throw when vdStates is a fully empty object', () => {
|
||||
expect(() => setup({})).not.toThrow();
|
||||
});
|
||||
|
||||
it('still renders category labels and the identity recipe with a partial shape', () => {
|
||||
const { container } = setup({ Gender: 'male' });
|
||||
expect(screen.getByText('test recipe')).toBeInTheDocument();
|
||||
// The label text sits alongside a sibling <span> kicker, so assert via
|
||||
// textContent rather than getByText's exact-node matching.
|
||||
expect(container.textContent).toContain('clone.cat_Gender');
|
||||
});
|
||||
});
|
||||
@@ -25,6 +25,7 @@ export default function DubHeader({
|
||||
handleDubStop,
|
||||
dubProgress,
|
||||
onGenerateClick,
|
||||
isTranslating,
|
||||
multiLangMode,
|
||||
multiLangs,
|
||||
incrementalPlan,
|
||||
@@ -34,121 +35,128 @@ export default function DubHeader({
|
||||
setExportOpen,
|
||||
}) {
|
||||
return (
|
||||
<div className="flex flex-wrap justify-between items-center gap-x-[var(--space-2)] gap-y-[4px] min-w-0 px-[10px] py-[4px] shrink-0 bg-[var(--color-bg-elev-1)] rounded-md mb-[2px]">
|
||||
{/* Pipeline spine, inlined onto the header row (Upload → … → Export). */}
|
||||
<DubPipelineStepper dubStep={dubStep} inline />
|
||||
<div className="label-row dub-head__title !gap-[6px]">
|
||||
<FileText className="label-icon" size={11} />
|
||||
<span className="font-medium text-[0.78rem] min-w-0 overflow-hidden text-ellipsis whitespace-nowrap text-fg normal-case">
|
||||
{dubFilename}
|
||||
</span>
|
||||
<span className="text-fg-muted font-normal whitespace-nowrap text-[0.68rem] normal-case shrink-0">
|
||||
· {formatTime(dubDuration)} · {dubSegments.length} {t('dub.segs')}
|
||||
</span>
|
||||
{activeProjectName && activeProjectName !== dubFilename && (
|
||||
<span className="text-[#b8bb26] ml-[var(--space-2)] whitespace-nowrap text-[0.68rem] normal-case overflow-hidden text-ellipsis min-w-0">
|
||||
— {activeProjectName}
|
||||
<div className="flex flex-col gap-[2px] min-w-0 px-[10px] py-[4px] shrink-0 bg-[var(--color-bg-elev-1)] rounded-md mb-[2px]">
|
||||
{/* Row 1: project title (left) + actions (right). Row 2: the pipeline
|
||||
spine (Upload → … → Export) sits directly under the title with a
|
||||
tight 2px gap — title-first, owner-requested order. */}
|
||||
<div className="flex flex-wrap justify-between items-center gap-x-[var(--space-2)] gap-y-[4px] min-w-0">
|
||||
<div className="label-row dub-head__title !gap-[6px]">
|
||||
<FileText className="label-icon" size={11} />
|
||||
<span className="font-medium text-[0.78rem] min-w-0 overflow-hidden text-ellipsis whitespace-nowrap text-fg normal-case">
|
||||
{dubFilename}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex gap-[6px] items-center shrink-0">
|
||||
{/* Icon-only secondary actions (tooltips carry the labels);
|
||||
<span className="text-fg-muted font-normal whitespace-nowrap text-[0.68rem] normal-case shrink-0">
|
||||
· {formatTime(dubDuration)} · {dubSegments.length} {t('dub.segs')}
|
||||
</span>
|
||||
{activeProjectName && activeProjectName !== dubFilename && (
|
||||
<span className="text-[#b8bb26] ml-[var(--space-2)] whitespace-nowrap text-[0.68rem] normal-case overflow-hidden text-ellipsis min-w-0">
|
||||
— {activeProjectName}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex gap-[6px] items-center shrink-0">
|
||||
{/* Icon-only secondary actions (tooltips carry the labels);
|
||||
Generate Dub keeps its label as the primary verb. */}
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
onClick={saveProject}
|
||||
title={t('dub.save')}
|
||||
aria-label={t('dub.save')}
|
||||
>
|
||||
<Save size={12} />
|
||||
</Button>
|
||||
<Button
|
||||
variant="danger"
|
||||
size="sm"
|
||||
onClick={resetDub}
|
||||
title={t('dub.reset')}
|
||||
aria-label={t('dub.reset')}
|
||||
>
|
||||
<RotateCcw size={12} />
|
||||
</Button>
|
||||
{/* Primary actions live on the header bar (compact) — moved up from the footer. */}
|
||||
<div className="flex gap-[6px] items-center pl-[var(--space-2)] ml-[2px]">
|
||||
{dubStep === 'stopping' ? (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="stopping"
|
||||
disabled
|
||||
icon={<Loader className="spinner" size={9} />}
|
||||
label={t('dub.stopping')}
|
||||
/>
|
||||
) : dubStep === 'generating' ? (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="danger"
|
||||
onClick={handleDubStop}
|
||||
icon={<Square size={9} />}
|
||||
label={t('dub.stop_progress', {
|
||||
current: dubProgress.current,
|
||||
total: dubProgress.total,
|
||||
})}
|
||||
/>
|
||||
) : (
|
||||
<>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
onClick={saveProject}
|
||||
title={t('dub.save')}
|
||||
aria-label={t('dub.save')}
|
||||
>
|
||||
<Save size={12} />
|
||||
</Button>
|
||||
<Button
|
||||
variant="danger"
|
||||
size="sm"
|
||||
onClick={resetDub}
|
||||
title={t('dub.reset')}
|
||||
aria-label={t('dub.reset')}
|
||||
>
|
||||
<RotateCcw size={12} />
|
||||
</Button>
|
||||
{/* Primary actions live on the header bar (compact) — moved up from the footer. */}
|
||||
<div className="flex gap-[6px] items-center pl-[var(--space-2)] ml-[2px]">
|
||||
{dubStep === 'stopping' ? (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone={dubSegments.length ? 'pink' : 'idle'}
|
||||
onClick={onGenerateClick}
|
||||
disabled={!dubSegments.length}
|
||||
icon={<Play size={11} />}
|
||||
label={
|
||||
multiLangMode && multiLangs.length > 1
|
||||
? t('dub.generate_dub_multi', {
|
||||
count: multiLangs.length,
|
||||
defaultValue: 'Generate {{count}} dubs',
|
||||
})
|
||||
: t('dub.generate_dub')
|
||||
}
|
||||
tone="stopping"
|
||||
disabled
|
||||
icon={<Loader className="spinner" size={9} />}
|
||||
label={t('dub.stopping')}
|
||||
/>
|
||||
{dubStep === 'done' && incrementalPlan && incrementalPlan.stale?.length > 0 && (
|
||||
) : dubStep === 'generating' ? (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="danger"
|
||||
onClick={handleDubStop}
|
||||
icon={<Square size={9} />}
|
||||
label={t('dub.stop_progress', {
|
||||
current: dubProgress.current,
|
||||
total: dubProgress.total,
|
||||
})}
|
||||
/>
|
||||
) : (
|
||||
<>
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="pink"
|
||||
onClick={() =>
|
||||
handleDubGenerate({ regenOnly: incrementalPlan.stale, preview: true })
|
||||
}
|
||||
tone={dubSegments.length && !isTranslating ? 'pink' : 'idle'}
|
||||
onClick={onGenerateClick}
|
||||
// The multi-language batch translates between generates while
|
||||
// dubStep briefly sits back at 'editing' — keep the CTA inert
|
||||
// during that phase so a re-click can't start a second batch.
|
||||
disabled={!dubSegments.length || isTranslating}
|
||||
icon={<Play size={11} />}
|
||||
label={t('dub.regen_changed', { count: incrementalPlan.stale.length })}
|
||||
label={
|
||||
multiLangMode && multiLangs.length > 1
|
||||
? t('dub.generate_dub_multi', {
|
||||
count: multiLangs.length,
|
||||
defaultValue: 'Generate {{count}} dubs',
|
||||
})
|
||||
: t('dub.generate_dub')
|
||||
}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
{dubStep === 'done' && (
|
||||
{dubStep === 'done' && incrementalPlan && incrementalPlan.stale?.length > 0 && (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="pink"
|
||||
onClick={() =>
|
||||
handleDubGenerate({ regenOnly: incrementalPlan.stale, preview: true })
|
||||
}
|
||||
icon={<Play size={11} />}
|
||||
label={t('dub.regen_changed', { count: incrementalPlan.stale.length })}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
{dubStep === 'done' && (
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="idle"
|
||||
disabled={qcRunning || !dubSegments.length}
|
||||
onClick={handleDubQc}
|
||||
icon={
|
||||
qcRunning ? <Loader className="spinner" size={11} /> : <ShieldCheck size={11} />
|
||||
}
|
||||
title={t('dub.qc_btn', { defaultValue: 'Verify dub timing (second-pass check)' })}
|
||||
aria-label={t('dub.qc_btn', {
|
||||
defaultValue: 'Verify dub timing (second-pass check)',
|
||||
})}
|
||||
/>
|
||||
)}
|
||||
<FooterBtn
|
||||
sm
|
||||
tone="idle"
|
||||
disabled={qcRunning || !dubSegments.length}
|
||||
onClick={handleDubQc}
|
||||
icon={
|
||||
qcRunning ? <Loader className="spinner" size={11} /> : <ShieldCheck size={11} />
|
||||
}
|
||||
title={t('dub.qc_btn', { defaultValue: 'Verify dub timing (second-pass check)' })}
|
||||
aria-label={t('dub.qc_btn', {
|
||||
defaultValue: 'Verify dub timing (second-pass check)',
|
||||
})}
|
||||
tone={dubStep === 'done' ? 'green' : 'idle'}
|
||||
disabled={dubStep !== 'done' && !dubSegments.length}
|
||||
onClick={() => setExportOpen(true)}
|
||||
icon={<Download size={12} />}
|
||||
title={t('dub.export_btn')}
|
||||
aria-label={t('dub.export_btn')}
|
||||
/>
|
||||
)}
|
||||
<FooterBtn
|
||||
sm
|
||||
tone={dubStep === 'done' ? 'green' : 'idle'}
|
||||
disabled={dubStep !== 'done' && !dubSegments.length}
|
||||
onClick={() => setExportOpen(true)}
|
||||
icon={<Download size={12} />}
|
||||
title={t('dub.export_btn')}
|
||||
aria-label={t('dub.export_btn')}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<DubPipelineStepper dubStep={dubStep} inline />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -18,6 +18,7 @@ import { useAppStore } from '../../store';
|
||||
import WaveformTimeline from '../WaveformTimeline';
|
||||
import MultiLangPicker from '../MultiLangPicker';
|
||||
import { API } from '../../api/client';
|
||||
import { dubListTracks } from '../../api/dub';
|
||||
import { LANG_CODES } from '../../utils/languages';
|
||||
import ALL_LANGUAGES from '../../languages.json';
|
||||
import { POPULAR_LANGS, PRESETS } from '../../utils/constants';
|
||||
@@ -143,6 +144,49 @@ export default function DubLeftColumn({
|
||||
else toast.error(t('dub.copy_failed'));
|
||||
};
|
||||
|
||||
// Per-track metadata (duration + timing strategy) for the pill tooltips.
|
||||
// The store only carries the track codes, so hydrate lazily from the
|
||||
// existing GET /dub/tracks/{job_id} once the editor shows tracks (re-runs
|
||||
// when a new language finishes and dubTracks changes). Failure-silent:
|
||||
// the pills render fine without tooltips.
|
||||
const [trackInfo, setTrackInfo] = useState({});
|
||||
useEffect(() => {
|
||||
if (!hasDubbedTrack || !dubJobId) return undefined;
|
||||
let cancelled = false;
|
||||
dubListTracks(dubJobId)
|
||||
.then((tracks) => {
|
||||
if (!cancelled) setTrackInfo(tracks || {});
|
||||
})
|
||||
.catch(() => {
|
||||
/* tooltip enrichment only — never block or toast */
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [hasDubbedTrack, dubJobId, dubTracks]);
|
||||
const trackTooltip = (code) => {
|
||||
const info = trackInfo[code];
|
||||
if (!info) return undefined;
|
||||
const parts = [];
|
||||
if (Number.isFinite(info.duration) && info.duration > 0) {
|
||||
parts.push(
|
||||
t('dub.track_tip_duration', {
|
||||
duration: fmtDur(Math.round(info.duration)),
|
||||
defaultValue: 'Duration {{duration}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (info.timing_strategy) {
|
||||
// Reuse the timing-strategy display names where they exist
|
||||
// (dub.timing_<id>); unknown/future strategies fall back to the raw id.
|
||||
const strategy = t(`dub.timing_${info.timing_strategy}`, {
|
||||
defaultValue: info.timing_strategy,
|
||||
});
|
||||
parts.push(t('dub.track_tip_timing', { strategy, defaultValue: 'Timing {{strategy}}' }));
|
||||
}
|
||||
return parts.length ? parts.join(' · ') : undefined;
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="studio-panel dub-panel-col">
|
||||
{hasDubbedTrack && (
|
||||
@@ -170,6 +214,7 @@ export default function DubLeftColumn({
|
||||
aria-checked={previewMode === code}
|
||||
className={`dub-lang-pill ${previewMode === code ? 'is-active' : ''}`}
|
||||
onClick={() => setPreviewMode(code)}
|
||||
title={trackTooltip(code)}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
|
||||
@@ -91,8 +91,13 @@ export default function CommunityZone({
|
||||
name: r.name,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
flash(t('gallery.use_failed', { defaultValue: 'Could not add that voice.' }));
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
}}
|
||||
onDesign={(item) =>
|
||||
|
||||
@@ -110,7 +110,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const r = await searchYoutube(q, 'import', 10);
|
||||
setResults(r.results || []);
|
||||
} catch (e) {
|
||||
flash(t('gallery.search_failed', { defaultValue: 'Search failed.' }));
|
||||
flash(
|
||||
t('gallery.search_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Search failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
setIsSearching(false);
|
||||
}
|
||||
@@ -149,7 +154,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
await uploadVoiceClip(fd);
|
||||
reload();
|
||||
} catch (err) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: err?.message || String(err),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
if (fileRef.current) fileRef.current.value = '';
|
||||
}
|
||||
@@ -165,7 +175,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
}),
|
||||
);
|
||||
} catch (e) {
|
||||
flash(t('gallery.save_failed', { defaultValue: 'Could not save profile.' }));
|
||||
flash(
|
||||
t('gallery.save_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not save profile: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -179,8 +194,13 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
try {
|
||||
await deleteGalleryVoice(v.id);
|
||||
reload();
|
||||
} catch {
|
||||
/* noop */
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.delete_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not delete: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -191,7 +211,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const file = new File([blob], `${v.name}.wav`, { type: 'audio/wav' });
|
||||
setTrimming({ voice: v, file });
|
||||
} catch (e) {
|
||||
flash(t('gallery.trim_load_failed', { defaultValue: 'Could not load audio for trimming.' }));
|
||||
flash(
|
||||
t('gallery.trim_load_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not load audio for trimming: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -209,7 +234,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
reload();
|
||||
setTrimming(null);
|
||||
} catch (e) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* Settings → Models tab → OpenAI-compatible remote ASR panel (#877).
|
||||
*
|
||||
* A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's
|
||||
* own Whisper API — today, without waiting on transformers to ship a direct
|
||||
* Qwen3-ASR integration. Configures the `openai-compat-asr` backend's
|
||||
* base_url/model/api_key; activating it as the active ASR engine still needs
|
||||
* `OMNIVOICE_ASR_BACKEND=openai-compat-asr` (no in-app ASR engine picker
|
||||
* exists yet for any ASR backend — this panel only configures this one).
|
||||
*
|
||||
* Endpoints (loopback-only):
|
||||
* GET /api/settings/asr-openai-compat → {base_url, model, has_key}
|
||||
* PUT /api/settings/asr-openai-compat body {base_url?, model?, api_key?}
|
||||
* ('' clears api_key; omitted/null leaves it unchanged — never returned)
|
||||
*/
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
import { Mic } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { apiJson, apiFetch } from '../../api/client';
|
||||
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
|
||||
import { Button } from '../../ui';
|
||||
|
||||
export default function AsrOpenAICompatPanel() {
|
||||
const { t } = useTranslation();
|
||||
const [baseUrl, setBaseUrl] = useState('');
|
||||
const [model, setModel] = useState('');
|
||||
const [apiKey, setApiKey] = useState('');
|
||||
const [hasKey, setHasKey] = useState(false);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
setError(null);
|
||||
try {
|
||||
const d = await apiJson('/api/settings/asr-openai-compat');
|
||||
setBaseUrl(d?.base_url || '');
|
||||
setModel(d?.model || '');
|
||||
setHasKey(Boolean(d?.has_key));
|
||||
setApiKey(''); // the key is never returned — the field always starts blank
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatLoadError'));
|
||||
}
|
||||
}, [t]);
|
||||
|
||||
useEffect(() => {
|
||||
refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const save = async () => {
|
||||
setSaving(true);
|
||||
setError(null);
|
||||
try {
|
||||
const res = await apiFetch('/api/settings/asr-openai-compat', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
base_url: baseUrl,
|
||||
model,
|
||||
// Only send api_key when the user actually typed something —
|
||||
// an untouched field must leave the stored key unchanged, not
|
||||
// clear it (the field is always blank on load, so "unchanged"
|
||||
// and "empty" would otherwise be indistinguishable).
|
||||
...(apiKey ? { api_key: apiKey } : {}),
|
||||
}),
|
||||
});
|
||||
const d = await res.json();
|
||||
setBaseUrl(d.base_url || '');
|
||||
setModel(d.model || '');
|
||||
setHasKey(Boolean(d.has_key));
|
||||
setApiKey('');
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatSaveError'));
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<SettingsSection
|
||||
icon={Mic}
|
||||
title={t('models.asrOpenAICompatTitle')}
|
||||
description={t('models.asrOpenAICompatDescription')}
|
||||
>
|
||||
{error && (
|
||||
<div className="perfpanel__error" role="alert">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatBaseUrlTitle')}
|
||||
hint={t('models.asrOpenAICompatBaseUrlHint')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
placeholder="http://localhost:8000/v1"
|
||||
data-testid="asr-openai-compat-base-url"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatModelTitle')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
placeholder="whisper-1"
|
||||
data-testid="asr-openai-compat-model"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatApiKeyTitle')}
|
||||
hint={
|
||||
hasKey ? t('models.asrOpenAICompatKeyConfigured') : t('models.asrOpenAICompatApiKeyHint')
|
||||
}
|
||||
control={
|
||||
<>
|
||||
<SettingsInput
|
||||
mono
|
||||
type="password"
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
placeholder={hasKey ? '••••••••' : t('models.asrOpenAICompatApiKeyOptional')}
|
||||
data-testid="asr-openai-compat-api-key"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
onClick={save}
|
||||
loading={saving}
|
||||
disabled={saving}
|
||||
data-testid="asr-openai-compat-save"
|
||||
>
|
||||
{t('common.save')}
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
</SettingsSection>
|
||||
);
|
||||
}
|
||||
@@ -1,12 +1,19 @@
|
||||
import React, { useCallback } from 'react';
|
||||
import React, { useCallback, useRef } from 'react';
|
||||
import { toast } from 'react-hot-toast';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { addBreadcrumb } from '../../utils/breadcrumbs';
|
||||
import { selectEngine } from '../../api/engines';
|
||||
import { listEngines, selectEngine } from '../../api/engines';
|
||||
import { notifyEngineSelected } from '../../utils/engineSelectToast';
|
||||
import EngineCompatibilityMatrix from '../EngineCompatibilityMatrix';
|
||||
import { SETTINGS_SECTION_SURFACE } from './primitives';
|
||||
|
||||
/** One pinned matrix per family, stacked in this order. ASR used to be
|
||||
* reachable only through the matrix's family tabs, which read as a
|
||||
* TTS-only table — README even promised a Settings ASR picker that
|
||||
* didn't exist (UX gap found during #877). Every family now gets a
|
||||
* visible picker; `OMNIVOICE_*_BACKEND` env vars still win over any pick. */
|
||||
const FAMILIES = ['tts', 'asr', 'llm'];
|
||||
|
||||
export default function EnginesTab() {
|
||||
const { t } = useTranslation();
|
||||
|
||||
@@ -17,10 +24,12 @@ export default function EnginesTab() {
|
||||
//
|
||||
// Review mode (the staged-checkpoint nudges) moved to Settings → General.
|
||||
const onSelect = useCallback(
|
||||
async (family, backendId) => {
|
||||
// modelId is only ever set by mlx-audio's curated-model picker (#981) —
|
||||
// every other call site (the "Use" button) omits it.
|
||||
async (family, backendId, modelId) => {
|
||||
try {
|
||||
addBreadcrumb(`engine:${family}=${backendId}`);
|
||||
const r = await selectEngine(family, backendId);
|
||||
const r = await selectEngine(family, backendId, modelId);
|
||||
// Consume the routing echo: warn (not a bare success) when the pick
|
||||
// lands on a CPU fallback on this host. See notifyEngineSelected.
|
||||
notifyEngineSelected(r, t, family);
|
||||
@@ -31,9 +40,32 @@ export default function EnginesTab() {
|
||||
[t],
|
||||
);
|
||||
|
||||
// The stacked matrices all consume the same GET /engines payload — share
|
||||
// one in-flight request so opening the tab probes every engine once, not
|
||||
// once per family. A per-matrix Refresh after the shared promise settles
|
||||
// still triggers a fresh fetch.
|
||||
const inflightList = useRef(null);
|
||||
const listEnginesShared = useCallback(() => {
|
||||
if (!inflightList.current) {
|
||||
inflightList.current = listEngines().finally(() => {
|
||||
inflightList.current = null;
|
||||
});
|
||||
}
|
||||
return inflightList.current;
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<section className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
|
||||
<EngineCompatibilityMatrix family="tts" onSelect={onSelect} />
|
||||
</section>
|
||||
<>
|
||||
{FAMILIES.map((family) => (
|
||||
<section key={family} className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
|
||||
<EngineCompatibilityMatrix
|
||||
family={family}
|
||||
showFamilyTabs={false}
|
||||
onSelect={onSelect}
|
||||
apiListEngines={listEnginesShared}
|
||||
/>
|
||||
</section>
|
||||
))}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
|
||||
// Keep toast side-channels out of the test (timers, portals).
|
||||
vi.mock('react-hot-toast', () => ({
|
||||
default: { error: vi.fn(), success: vi.fn() },
|
||||
toast: Object.assign(vi.fn(), { error: vi.fn(), success: vi.fn() }),
|
||||
}));
|
||||
|
||||
vi.mock('../../api/engines', () => ({
|
||||
listEngines: vi.fn(),
|
||||
selectEngine: vi.fn(),
|
||||
getEngineHealth: vi.fn(),
|
||||
selfTestEngine: vi.fn(),
|
||||
}));
|
||||
|
||||
import { listEngines, selectEngine } from '../../api/engines';
|
||||
import EnginesTab from './EnginesTab';
|
||||
|
||||
function entry(id, name) {
|
||||
return {
|
||||
id,
|
||||
display_name: name,
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
};
|
||||
}
|
||||
|
||||
const ENGINES = {
|
||||
tts: { active: 'omnivoice', backends: [entry('omnivoice', 'OmniVoice (test)')] },
|
||||
asr: {
|
||||
active: 'whisperx',
|
||||
backends: [
|
||||
entry('whisperx', 'WhisperX (test)'),
|
||||
entry('openai-compat-asr', 'OpenAI-compatible ASR (test)'),
|
||||
],
|
||||
},
|
||||
llm: { active: 'off', backends: [entry('off', 'Off (test)')] },
|
||||
};
|
||||
|
||||
describe('EnginesTab', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
listEngines.mockResolvedValue(ENGINES);
|
||||
});
|
||||
|
||||
it('renders a pinned picker per family — TTS, ASR and LLM all visible at once', async () => {
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
|
||||
// One named section per family (the ASR picker used to be tucked behind
|
||||
// a family tab inside a single TTS-titled matrix — no picker to find).
|
||||
expect(screen.getByText('TTS Engines')).toBeInTheDocument();
|
||||
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
|
||||
expect(screen.getByText('LLM Engines')).toBeInTheDocument();
|
||||
// Pinned matrices render no family switcher.
|
||||
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
|
||||
});
|
||||
|
||||
it('the stacked matrices share one GET /engines on mount', async () => {
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
expect(listEngines).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('clicking Use on an ASR engine selects it with family="asr"', async () => {
|
||||
selectEngine.mockResolvedValue({
|
||||
family: 'asr',
|
||||
active: 'openai-compat-asr',
|
||||
env_override: false,
|
||||
routing_status: 'cpu_only',
|
||||
effective_device: 'cpu',
|
||||
routing_reason: null,
|
||||
});
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('OpenAI-compatible ASR (test)'));
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /use openai-compatible asr \(test\)/i }));
|
||||
await waitFor(() => {
|
||||
expect(selectEngine).toHaveBeenCalledWith('asr', 'openai-compat-asr', undefined);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -40,6 +40,10 @@ export default function LLMProvidersPanel() {
|
||||
const [modelsTruncated, setModelsTruncated] = useState(false);
|
||||
const [loadingModels, setLoadingModels] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
// True after a save/Test whose provider is still NOT the active one — the
|
||||
// save persisted fine but translation keeps using another provider, so be
|
||||
// honest about it instead of letting a green Test read as "done" (#963).
|
||||
const [savedInactive, setSavedInactive] = useState(false);
|
||||
|
||||
const current = useMemo(
|
||||
() => providers.find((p) => p.id === editing) || null,
|
||||
@@ -76,6 +80,7 @@ export default function LLMProvidersPanel() {
|
||||
setTest(null);
|
||||
setModels(null);
|
||||
setModelsTruncated(false);
|
||||
setSavedInactive(false);
|
||||
}, []);
|
||||
|
||||
const refresh = useCallback(
|
||||
@@ -127,7 +132,12 @@ export default function LLMProvidersPanel() {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
await refresh(current.id);
|
||||
const data = await refresh(current.id);
|
||||
// Saved but another provider stays active → say so (populate() above
|
||||
// cleared the previous notice). Suppress while LLM_DEFAULT_PROVIDER
|
||||
// pins the choice — the env banner already explains and the suggested
|
||||
// button is disabled.
|
||||
setSavedInactive(Boolean(data) && data.active !== current.id && !current.active_from_env);
|
||||
} catch (e) {
|
||||
setError(e?.message || t('settings.llmp_save_failed'));
|
||||
} finally {
|
||||
@@ -411,6 +421,16 @@ export default function LLMProvidersPanel() {
|
||||
</div>
|
||||
}
|
||||
/>
|
||||
|
||||
{savedInactive && (
|
||||
<div
|
||||
role="status"
|
||||
data-testid="llm-not-active-notice"
|
||||
className="text-[length:var(--text-xs)] text-[color:var(--chrome-fg-dim)] leading-[1.5] py-[var(--space-2)]"
|
||||
>
|
||||
{t('settings.llmp_saved_not_active')}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</SettingsSection>
|
||||
|
||||
@@ -181,4 +181,35 @@ describe('LLMProvidersPanel', () => {
|
||||
fireEvent.change(select, { target: { value: 'ollama' } });
|
||||
await waitFor(() => expect(screen.queryByTestId('llm-provider-key')).toBeNull());
|
||||
});
|
||||
|
||||
// #963 honesty: a green Test on a provider that is NOT the active one must
|
||||
// say the provider isn't used for translation yet — pre-fix the panel read
|
||||
// as "done" while translation kept using another provider.
|
||||
it('test-only flow on a non-active provider surfaces the not-yet-active notice', async () => {
|
||||
global.fetch = mockFetchSequence(
|
||||
{ body: PROVIDERS }, // mount GET (active: groq)
|
||||
{ body: {} }, // save PUT (ollama, make_active:false)
|
||||
{ body: PROVIDERS }, // refresh GET — active is still groq
|
||||
{ body: { ok: true, model: 'llama3', reply: 'ok', latency_ms: 9 } }, // test POST
|
||||
);
|
||||
render(<LLMProvidersPanel />);
|
||||
const select = await screen.findByTestId('llm-provider-select');
|
||||
fireEvent.change(select, { target: { value: 'ollama' } });
|
||||
fireEvent.click(screen.getByTestId('llm-provider-test'));
|
||||
await waitFor(() => expect(screen.getByTestId('llm-not-active-notice')).toBeInTheDocument());
|
||||
expect(screen.getByText(/not yet used for translation/)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('no notice when the saved provider IS the active one', async () => {
|
||||
global.fetch = mockFetchSequence(
|
||||
{ body: PROVIDERS }, // mount GET (active: groq)
|
||||
{ body: {} }, // save PUT (groq)
|
||||
{ body: PROVIDERS }, // refresh GET — groq active
|
||||
{ body: { ok: true, model: 'llama-3.3-70b', reply: 'ok', latency_ms: 412 } }, // test POST
|
||||
);
|
||||
render(<LLMProvidersPanel />);
|
||||
fireEvent.click(await screen.findByTestId('llm-provider-test'));
|
||||
await waitFor(() => expect(screen.getByText(/llama-3\.3-70b · 412 ms/)).toBeInTheDocument());
|
||||
expect(screen.queryByTestId('llm-not-active-notice')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,6 +8,7 @@ import { listExportHistory } from '../api/exports';
|
||||
import { modelStatus as apiModelStatus } from '../api/system';
|
||||
import { useModelStatus } from '../api/hooks';
|
||||
import useRealtimeEvents from './useRealtimeEvents';
|
||||
import { mergeDescribedAttrs } from '../utils/voiceInstruct';
|
||||
|
||||
/**
|
||||
* Encapsulates all data-loading effects, localStorage persistence,
|
||||
@@ -174,7 +175,12 @@ export default function useAppData() {
|
||||
setDefineMethod('design');
|
||||
} else if (saved.mode) setMode(saved.mode);
|
||||
if (saved.defineMethod) setDefineMethod(saved.defineMethod);
|
||||
if (saved.vdStates) setVdStates(saved.vdStates);
|
||||
// #983: legacy localStorage state had no shape validation at all — a
|
||||
// partial/corrupt saved.vdStates crashed DesignMethodPanel on restore.
|
||||
// Mirror useProfiles.js's guard: require a plain object, then complete
|
||||
// it to the full CATEGORIES shape (missing/unknown keys → 'Auto').
|
||||
if (saved.vdStates && typeof saved.vdStates === 'object')
|
||||
setVdStates(mergeDescribedAttrs(saved.vdStates));
|
||||
if (saved.language) setLanguage(saved.language);
|
||||
if (saved.isSidebarCollapsed !== undefined) setIsSidebarCollapsed(saved.isSidebarCollapsed);
|
||||
if (saved.sidebarTab) setSidebarTab(saved.sidebarTab);
|
||||
|
||||
@@ -687,109 +687,137 @@ export default function useDubWorkflow({
|
||||
}
|
||||
}, [dubJobId, dubSegments, setDubSegments]);
|
||||
|
||||
const handleTranslateAll = useCallback(async () => {
|
||||
if (!dubSegments.length || !dubLangCode) return;
|
||||
setIsTranslating(true);
|
||||
// Root cause of the "sticky TRANSLATION FAILED banner": a new translate
|
||||
// attempt never cleared the previous failure, so a stale 400 survived even
|
||||
// a successful retry. Clear it up front — the whole class of translate/
|
||||
// pipeline error banners should reset on the next relevant action.
|
||||
setDubError('');
|
||||
try {
|
||||
const data = await dubTranslate({
|
||||
segments: dubSegments.map((s) => ({
|
||||
id: String(s.id),
|
||||
text: s.text_original && s.text_original.trim() ? s.text_original : s.text,
|
||||
target_lang: s.target_lang,
|
||||
direction: s.direction || undefined,
|
||||
slot_seconds: s.end != null && s.start != null ? s.end - s.start : undefined,
|
||||
})),
|
||||
target_lang: dubLangCode,
|
||||
provider: translateProvider,
|
||||
quality: translateQuality,
|
||||
// #280: regional dialect — only sent when it matches the target
|
||||
// language so a stale "es-AR" never rides on a French translate.
|
||||
dialect: dialectMatchesLang(dubDialect, dubLangCode) ? dubDialect : undefined,
|
||||
glossary: glossaryTerms.length
|
||||
? glossaryTerms.map((t) => ({ source: t.source, target: t.target, note: t.note || '' }))
|
||||
: undefined,
|
||||
});
|
||||
const translatedMap = {};
|
||||
const errors = [];
|
||||
(data.translated || []).forEach((t) => {
|
||||
translatedMap[t.id] = t;
|
||||
if (t.error) errors.push({ id: t.id, error: t.error });
|
||||
});
|
||||
setDubSegments(
|
||||
dubSegments.map((s) => {
|
||||
const hit = translatedMap[s.id];
|
||||
if (!hit) return s;
|
||||
return {
|
||||
...s,
|
||||
text: hit.text && hit.text.trim() ? hit.text : s.text,
|
||||
translate_error: hit.error || undefined,
|
||||
translate_literal: hit.literal || undefined,
|
||||
translate_critique: hit.critique || undefined,
|
||||
// Carry over the predicted compression ratio so the per-row
|
||||
// badge + job-level compression warning can light up before
|
||||
// the user clicks Generate Dub.
|
||||
rate_ratio: hit.rate_ratio != null ? hit.rate_ratio : s.rate_ratio,
|
||||
rate_error: hit.rate_error || s.rate_error,
|
||||
};
|
||||
}),
|
||||
);
|
||||
if (data.cinematic_skipped === 'no-llm-configured') {
|
||||
toast(t('dub_workflow.cinematic_no_llm'), { icon: 'ℹ️', duration: 8000 });
|
||||
// #372: the backend fell back to Fast — reflect that in the toggle so
|
||||
// the UI doesn't claim Cinematic while delivering Fast.
|
||||
useAppStore.getState().setTranslateQuality?.('fast');
|
||||
}
|
||||
// #280: the user picked a dialect but the chosen engine can't honor it
|
||||
// (Argos/NLLB/Google in Fast mode). Tell them how to make it count.
|
||||
// #372: skip when the cinematic toast above already fired — both at once
|
||||
// sent users in a circle ("pick Cinematic" ↔ "Cinematic needs an LLM").
|
||||
if (
|
||||
data.dialect &&
|
||||
data.dialect_applied === false &&
|
||||
data.cinematic_skipped !== 'no-llm-configured'
|
||||
) {
|
||||
toast(t('dub_workflow.dialect_not_applied'), { icon: 'ℹ️', duration: 8000 });
|
||||
}
|
||||
if (errors.length) {
|
||||
const unique = [...new Set(errors.map((e) => e.error))];
|
||||
toast.error(
|
||||
t('dub_workflow.translate_errors', {
|
||||
errorCount: errors.length,
|
||||
totalCount: data.translated.length,
|
||||
firstError: unique[0].slice(0, 120),
|
||||
// `langOverride` (optional ISO code string) is the multi-language batch path:
|
||||
// the generate loop translates INTO each pick before dubbing it. No-arg calls
|
||||
// (the Translate All button, the review checkpoint) behave exactly as before
|
||||
// — the guard also shields the direct `onClick={handleTranslateAll}` usages,
|
||||
// where the first argument is a click event, not a language.
|
||||
// Resolves `true` when a translation landed in the segments, `false` when the
|
||||
// request failed or nothing got translated — the batch loop skips generating
|
||||
// that language rather than rendering a wrong-language track.
|
||||
const handleTranslateAll = useCallback(
|
||||
async (langOverride) => {
|
||||
const targetLang =
|
||||
typeof langOverride === 'string' && langOverride ? langOverride : dubLangCode;
|
||||
// Snapshot segments at call time: inside the multi-language loop the
|
||||
// click-time closure is stale after the previous pick's translate pass.
|
||||
const segs = useAppStore.getState().dubSegments;
|
||||
if (!segs.length || !targetLang) return false;
|
||||
setIsTranslating(true);
|
||||
// Root cause of the "sticky TRANSLATION FAILED banner": a new translate
|
||||
// attempt never cleared the previous failure, so a stale 400 survived even
|
||||
// a successful retry. Clear it up front — the whole class of translate/
|
||||
// pipeline error banners should reset on the next relevant action.
|
||||
setDubError('');
|
||||
let ok = false;
|
||||
try {
|
||||
const data = await dubTranslate({
|
||||
segments: segs.map((s) => ({
|
||||
id: String(s.id),
|
||||
text: s.text_original && s.text_original.trim() ? s.text_original : s.text,
|
||||
target_lang: s.target_lang,
|
||||
direction: s.direction || undefined,
|
||||
slot_seconds: s.end != null && s.start != null ? s.end - s.start : undefined,
|
||||
})),
|
||||
target_lang: targetLang,
|
||||
provider: translateProvider,
|
||||
quality: translateQuality,
|
||||
// #280: regional dialect — only sent when it matches the target
|
||||
// language so a stale "es-AR" never rides on a French translate.
|
||||
dialect: dialectMatchesLang(dubDialect, targetLang) ? dubDialect : undefined,
|
||||
glossary: glossaryTerms.length
|
||||
? glossaryTerms.map((t) => ({ source: t.source, target: t.target, note: t.note || '' }))
|
||||
: undefined,
|
||||
});
|
||||
const translatedMap = {};
|
||||
const errors = [];
|
||||
(data.translated || []).forEach((t) => {
|
||||
translatedMap[t.id] = t;
|
||||
if (t.error) errors.push({ id: t.id, error: t.error });
|
||||
});
|
||||
setDubSegments((prev) =>
|
||||
prev.map((s) => {
|
||||
const hit = translatedMap[s.id];
|
||||
if (!hit) return s;
|
||||
const gotText = !!(hit.text && hit.text.trim());
|
||||
return {
|
||||
...s,
|
||||
text: gotText ? hit.text : s.text,
|
||||
// P1.2 — keep every language's translation, keyed by target.
|
||||
// `text` stays the currently-shown language (legacy single-slot
|
||||
// contract); switching the target language swaps from this map
|
||||
// instead of destroying the previous language's work.
|
||||
...(gotText ? { translations: { ...s.translations, [targetLang]: hit.text } } : {}),
|
||||
translate_error: hit.error || undefined,
|
||||
translate_literal: hit.literal || undefined,
|
||||
translate_critique: hit.critique || undefined,
|
||||
// Carry over the predicted compression ratio so the per-row
|
||||
// badge + job-level compression warning can light up before
|
||||
// the user clicks Generate Dub.
|
||||
rate_ratio: hit.rate_ratio != null ? hit.rate_ratio : s.rate_ratio,
|
||||
rate_error: hit.rate_error || s.rate_error,
|
||||
};
|
||||
}),
|
||||
{ duration: 6000 },
|
||||
);
|
||||
} else {
|
||||
const qLabel =
|
||||
data.quality_used === 'cinematic' ? t('dub_workflow.translated_cinematic_suffix') : '';
|
||||
toast.success(
|
||||
t('dub_workflow.translated_segments', {
|
||||
count: data.translated.length,
|
||||
lang: data.target_lang,
|
||||
}) + qLabel,
|
||||
);
|
||||
// "Translated" for the batch loop means at least one segment actually
|
||||
// got new text — an empty result or an all-errors result would make
|
||||
// the follow-up generate render the source language verbatim.
|
||||
const total = (data.translated || []).length;
|
||||
ok = total > 0 && errors.length < total;
|
||||
if (data.cinematic_skipped === 'no-llm-configured') {
|
||||
toast(t('dub_workflow.cinematic_no_llm'), { icon: 'ℹ️', duration: 8000 });
|
||||
// #372: the backend fell back to Fast — reflect that in the toggle so
|
||||
// the UI doesn't claim Cinematic while delivering Fast.
|
||||
useAppStore.getState().setTranslateQuality?.('fast');
|
||||
}
|
||||
// #280: the user picked a dialect but the chosen engine can't honor it
|
||||
// (Argos/NLLB/Google in Fast mode). Tell them how to make it count.
|
||||
// #372: skip when the cinematic toast above already fired — both at once
|
||||
// sent users in a circle ("pick Cinematic" ↔ "Cinematic needs an LLM").
|
||||
if (
|
||||
data.dialect &&
|
||||
data.dialect_applied === false &&
|
||||
data.cinematic_skipped !== 'no-llm-configured'
|
||||
) {
|
||||
toast(t('dub_workflow.dialect_not_applied'), { icon: 'ℹ️', duration: 8000 });
|
||||
}
|
||||
if (errors.length) {
|
||||
const unique = [...new Set(errors.map((e) => e.error))];
|
||||
toast.error(
|
||||
t('dub_workflow.translate_errors', {
|
||||
errorCount: errors.length,
|
||||
totalCount: data.translated.length,
|
||||
firstError: unique[0].slice(0, 120),
|
||||
}),
|
||||
{ duration: 6000 },
|
||||
);
|
||||
} else {
|
||||
const qLabel =
|
||||
data.quality_used === 'cinematic' ? t('dub_workflow.translated_cinematic_suffix') : '';
|
||||
toast.success(
|
||||
t('dub_workflow.translated_segments', {
|
||||
count: data.translated.length,
|
||||
lang: data.target_lang,
|
||||
}) + qLabel,
|
||||
);
|
||||
}
|
||||
} catch (err) {
|
||||
setDubError(t('dub_workflow.translation_failed', { message: err.message }));
|
||||
}
|
||||
} catch (err) {
|
||||
setDubError(t('dub_workflow.translation_failed', { message: err.message }));
|
||||
}
|
||||
setIsTranslating(false);
|
||||
}, [
|
||||
dubSegments,
|
||||
dubLangCode,
|
||||
dubDialect,
|
||||
translateProvider,
|
||||
translateQuality,
|
||||
glossaryTerms,
|
||||
setIsTranslating,
|
||||
setDubSegments,
|
||||
setDubError,
|
||||
]);
|
||||
setIsTranslating(false);
|
||||
return ok;
|
||||
},
|
||||
[
|
||||
dubLangCode,
|
||||
dubDialect,
|
||||
translateProvider,
|
||||
translateQuality,
|
||||
glossaryTerms,
|
||||
setIsTranslating,
|
||||
setDubSegments,
|
||||
setDubError,
|
||||
],
|
||||
);
|
||||
|
||||
const handleDubGenerate = useCallback(
|
||||
async (opts = {}) => {
|
||||
@@ -801,8 +829,12 @@ export default function useDubWorkflow({
|
||||
// here, overriding the store's single selection (which is stale inside the
|
||||
// loop). Each run appends its track to the job's dubbed_tracks.
|
||||
const langOv = opts.langOverride || null;
|
||||
// Snapshot segments at call time, not click time: the multi-language
|
||||
// loop awaits a translate pass right before each generate, and the
|
||||
// click-time closure would still hold the pre-translation text.
|
||||
const segs = useAppStore.getState().dubSegments;
|
||||
setDubStep('generating');
|
||||
setDubProgress({ current: 0, total: dubSegments.length, text: '' });
|
||||
setDubProgress({ current: 0, total: segs.length, text: '' });
|
||||
setDubError('');
|
||||
const genLabel = regenOnly
|
||||
? t('dub_workflow.regenerating', { count: regenOnly.length })
|
||||
@@ -812,12 +844,12 @@ export default function useDubWorkflow({
|
||||
.showPill('generating', genLabel, { cancellable: true, homeMode: 'dub' });
|
||||
try {
|
||||
const body = {
|
||||
segment_ids: dubSegments.map((s) => String(s.id)),
|
||||
segment_ids: segs.map((s) => String(s.id)),
|
||||
regen_only: regenOnly,
|
||||
// Generation inputs come from the shared helper so the stored
|
||||
// fingerprints (seg_hashes) match what /tools/incremental recomputes
|
||||
// later — see utils/segments.js (#281).
|
||||
segments: dubSegments.map((s) => ({
|
||||
segments: segs.map((s) => ({
|
||||
start: s.start,
|
||||
end: s.end,
|
||||
gain: s.gain !== undefined && s.gain !== 1.0 ? s.gain : undefined,
|
||||
@@ -890,17 +922,22 @@ export default function useDubWorkflow({
|
||||
.map(([id]) => id);
|
||||
setPreviewSegIds(previewIds);
|
||||
}
|
||||
// P1.3 — hashes belong to the track that just generated
|
||||
// (the event carries its language), not to whatever the
|
||||
// store's selection is when the stream drains.
|
||||
const genLang = evt.language_code || body.language_code;
|
||||
if (evt.seg_hashes && Object.keys(evt.seg_hashes).length > 0) {
|
||||
setLastGenFingerprints(evt.seg_hashes);
|
||||
setLastGenFingerprints(evt.seg_hashes, genLang);
|
||||
} else {
|
||||
try {
|
||||
const plan = await apiPost('/tools/incremental', {
|
||||
segments: dubSegments.map((s) => ({
|
||||
segments: segs.map((s) => ({
|
||||
id: String(s.id),
|
||||
...segmentGenInputs(s),
|
||||
})),
|
||||
lang: genLang,
|
||||
});
|
||||
setLastGenFingerprints(plan.fingerprints || {});
|
||||
setLastGenFingerprints(plan.fingerprints || {}, genLang);
|
||||
} catch (err) {
|
||||
console.warn('Incremental plan fallback failed:', err);
|
||||
}
|
||||
@@ -941,7 +978,6 @@ export default function useDubWorkflow({
|
||||
},
|
||||
[
|
||||
dubJobId,
|
||||
dubSegments,
|
||||
dubLang,
|
||||
dubLangCode,
|
||||
dubInstruct,
|
||||
|
||||
@@ -11,7 +11,11 @@ import { generateSpeech, audioUrlWithCacheBust } from '../api/generate';
|
||||
import { apiFetch } from '../api/client';
|
||||
import { playBlobAudio } from '../utils/media';
|
||||
import { PRESETS } from '../utils/constants';
|
||||
import { instructToFormValue } from '../utils/voiceInstruct';
|
||||
import {
|
||||
instructToFormValue,
|
||||
mergeDescribedAttrs,
|
||||
buildDesignInstruct,
|
||||
} from '../utils/voiceInstruct';
|
||||
import { askConfirm } from '../utils/dialog';
|
||||
import { toast } from 'react-hot-toast';
|
||||
import { recordValueMoment } from '../utils/donationMoments';
|
||||
@@ -55,7 +59,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
const safeBlob = new Blob([arrBuf], { type: refAudio.type });
|
||||
formData.append('ref_audio', safeBlob, refAudio.name || 'profile.wav');
|
||||
formData.append('ref_text', refText);
|
||||
formData.append('instruct', instruct);
|
||||
// #1010: the backend only sanitizes instruct on save for kind='design'
|
||||
// profiles — a clone profile (this call always creates kind='clone')
|
||||
// would silently persist an unsupported free-text instruct and then
|
||||
// 400 every single time it's used to generate. Filter here too.
|
||||
const { instruct: safeInst } = buildDesignInstruct({}, instruct);
|
||||
formData.append('instruct', safeInst);
|
||||
formData.append('language', language);
|
||||
try {
|
||||
await createProfile(formData);
|
||||
@@ -96,7 +105,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
if (profile.kind === 'design' && profile.vd_states) {
|
||||
try {
|
||||
const parsed = JSON.parse(profile.vd_states);
|
||||
if (parsed && typeof parsed === 'object') setVdStates(parsed);
|
||||
// #983: a profile saved by an older/foreign client (or hand-edited)
|
||||
// can carry a partial shape — mergeDescribedAttrs (already used for
|
||||
// the "describe your voice" restore path) guarantees every
|
||||
// CATEGORIES key is present, defaulting missing/unknown ones to
|
||||
// 'Auto', so DesignMethodPanel never sees an undefined category.
|
||||
if (parsed && typeof parsed === 'object') setVdStates(mergeDescribedAttrs(parsed));
|
||||
} catch {
|
||||
/* malformed stored state — sliders keep their current values */
|
||||
}
|
||||
@@ -199,6 +213,25 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
fin_prof = '';
|
||||
}
|
||||
|
||||
// #1010: this instruct string comes straight from segment/preset data,
|
||||
// never through the validator-safe builder — a preset's raw attrs or a
|
||||
// free-text style field can carry phrases outside the active engine's
|
||||
// supported instruct vocabulary, 400ing instead of previewing. Same
|
||||
// client-side guard useTTS.js already applies to the clone path.
|
||||
if (fin_inst) {
|
||||
const { instruct: safeInst, unsupported, duplicates } = buildDesignInstruct({}, fin_inst);
|
||||
if (unsupported.length) {
|
||||
toast(t('tts_errors.ignored_unsupported', { items: unsupported.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
if (duplicates.length) {
|
||||
toast(t('tts_errors.ignored_duplicate', { items: duplicates.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
fin_inst = safeInst;
|
||||
}
|
||||
if (fin_prof) formData.append('profile_id', fin_prof);
|
||||
if (fin_inst) formData.append('instruct', fin_inst);
|
||||
const fin_lang = seg.target_lang || dubLang;
|
||||
@@ -240,7 +273,10 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
: item.text
|
||||
: '';
|
||||
formData.append('ref_text', extractedText);
|
||||
formData.append('instruct', item.instruct || '');
|
||||
// #1010: same guard as handleSaveProfile — this always creates a
|
||||
// kind='clone' profile, which the backend never sanitizes on save.
|
||||
const { instruct: safeHistInst } = buildDesignInstruct({}, item.instruct || '');
|
||||
formData.append('instruct', safeHistInst);
|
||||
formData.append('language', item.language || 'Auto');
|
||||
if (item.seed !== undefined && item.seed !== null) {
|
||||
formData.append('seed', item.seed);
|
||||
|
||||
@@ -11,6 +11,10 @@ import { apiPost } from '../api/client';
|
||||
import { segmentGenInputs } from '../utils/segments';
|
||||
import { commitMoveResize } from '../utils/timeline';
|
||||
|
||||
// Stable empty map so `lastGenFingerprints` keeps a constant identity for a
|
||||
// language with no stored hashes (avoids effect/callback churn).
|
||||
const EMPTY_FINGERPRINTS = {};
|
||||
|
||||
export default function useSegmentEditing() {
|
||||
const dubSegments = useAppStore((s) => s.dubSegments);
|
||||
const setDubSegments = useAppStore((s) => s.setDubSegments);
|
||||
@@ -50,7 +54,20 @@ export default function useSegmentEditing() {
|
||||
const segmentEditField = useCallback(
|
||||
(id, field, value) => {
|
||||
pushUndo(dubSegments);
|
||||
setDubSegments((prev) => prev.map((s) => (s.id === id ? { ...s, [field]: value } : s)));
|
||||
// P1.2 — a manual text edit is a translation edit for the CURRENT
|
||||
// target language: keep `translations[lang]` in lock-step with `text`
|
||||
// so switching languages and back never loses the edit.
|
||||
const lang = useAppStore.getState().dubLangCode;
|
||||
setDubSegments((prev) =>
|
||||
prev.map((s) => {
|
||||
if (s.id !== id) return s;
|
||||
const next = { ...s, [field]: value };
|
||||
if (field === 'text' && lang) {
|
||||
next.translations = { ...s.translations, [lang]: value };
|
||||
}
|
||||
return next;
|
||||
}),
|
||||
);
|
||||
},
|
||||
[dubSegments],
|
||||
);
|
||||
@@ -87,10 +104,22 @@ export default function useSegmentEditing() {
|
||||
const segmentRestoreOriginal = useCallback(
|
||||
(id) => {
|
||||
pushUndo(dubSegments);
|
||||
// "Use the original text for this row" is a per-language decision like
|
||||
// any other text edit — record it under the current language so a
|
||||
// round-trip through another language doesn't resurrect the discarded
|
||||
// translation (P1.2).
|
||||
const lang = useAppStore.getState().dubLangCode;
|
||||
setDubSegments((prev) =>
|
||||
prev.map((s) =>
|
||||
s.id === id ? { ...s, text: s.text_original || s.text, translate_error: undefined } : s,
|
||||
),
|
||||
prev.map((s) => {
|
||||
if (s.id !== id) return s;
|
||||
const restored = s.text_original || s.text;
|
||||
return {
|
||||
...s,
|
||||
text: restored,
|
||||
...(lang ? { translations: { ...s.translations, [lang]: restored } } : {}),
|
||||
translate_error: undefined,
|
||||
};
|
||||
}),
|
||||
);
|
||||
},
|
||||
[dubSegments],
|
||||
@@ -164,12 +193,16 @@ export default function useSegmentEditing() {
|
||||
const pos = Math.max(1, Math.min(cursorPos, text.length - 1));
|
||||
const ratio = text.length > 0 ? pos / text.length : 0.5;
|
||||
const midT = seg.start + (seg.end - seg.start) * ratio;
|
||||
// Other languages' saved texts (P1.2) can't be split at a sensible
|
||||
// position for the halves — drop them; the halves are new segment ids
|
||||
// that need fresh TTS per language anyway.
|
||||
const left = {
|
||||
...seg,
|
||||
id: `${seg.id}_a`,
|
||||
text: text.slice(0, pos).trim(),
|
||||
end: midT,
|
||||
text_original: text.slice(0, pos).trim(),
|
||||
translations: undefined,
|
||||
};
|
||||
const right = {
|
||||
...seg,
|
||||
@@ -177,6 +210,7 @@ export default function useSegmentEditing() {
|
||||
text: text.slice(pos).trim(),
|
||||
start: midT,
|
||||
text_original: text.slice(pos).trim(),
|
||||
translations: undefined,
|
||||
};
|
||||
return [...prev.slice(0, idx), left, right, ...prev.slice(idx + 1)];
|
||||
});
|
||||
@@ -193,12 +227,24 @@ export default function useSegmentEditing() {
|
||||
if (idx < 0 || idx >= prev.length - 1) return prev;
|
||||
const a = prev[idx];
|
||||
const b = prev[idx + 1];
|
||||
// Merge per-language texts (P1.2) only where BOTH sides carry the
|
||||
// language — a half-known language would otherwise mix two languages
|
||||
// in one entry. Missing entries just mean "translate again".
|
||||
const ta = a.translations || {};
|
||||
const tb = b.translations || {};
|
||||
const mergedTranslations = {};
|
||||
for (const lang of Object.keys(ta)) {
|
||||
if (typeof ta[lang] === 'string' && typeof tb[lang] === 'string') {
|
||||
mergedTranslations[lang] = `${ta[lang]} ${tb[lang]}`.trim();
|
||||
}
|
||||
}
|
||||
const merged = {
|
||||
...a,
|
||||
text: `${a.text || ''} ${b.text || ''}`.trim(),
|
||||
text_original:
|
||||
`${a.text_original || a.text || ''} ${b.text_original || b.text || ''}`.trim(),
|
||||
end: b.end,
|
||||
translations: Object.keys(mergedTranslations).length ? mergedTranslations : undefined,
|
||||
};
|
||||
return [...prev.slice(0, idx), merged, ...prev.slice(idx + 2)];
|
||||
});
|
||||
@@ -221,8 +267,25 @@ export default function useSegmentEditing() {
|
||||
[directionSegId, dubSegments],
|
||||
);
|
||||
|
||||
// Incremental plan — tracks which segments changed since last generate
|
||||
const [lastGenFingerprints, setLastGenFingerprints] = useState({});
|
||||
// Incremental plan — tracks which segments changed since last generate.
|
||||
// P1.3: fingerprints are stored PER LANGUAGE ({ lang: { segId: hash } }),
|
||||
// and `lastGenFingerprints` is the ACTIVE language's map — so "Regen N
|
||||
// changed" is judged against the track you're looking at, never against
|
||||
// whichever language happened to generate last. Switching to a language
|
||||
// that was never generated yields an empty map → no plan (no false
|
||||
// "all fresh" / "all stale" claims).
|
||||
const dubLangCode = useAppStore((s) => s.dubLangCode);
|
||||
const [fingerprintsByLang, setFingerprintsByLang] = useState({});
|
||||
const lastGenFingerprints = fingerprintsByLang[dubLangCode] || EMPTY_FINGERPRINTS;
|
||||
// Same call signature as before for existing single-track callers; the
|
||||
// optional `lang` pins the map to the track that produced the hashes
|
||||
// (e.g. each pick of the multi-language batch loop) instead of whatever
|
||||
// the store's selection is by the time the response lands.
|
||||
const setLastGenFingerprints = useCallback((map, lang) => {
|
||||
const key = lang || useAppStore.getState().dubLangCode;
|
||||
if (!key) return;
|
||||
setFingerprintsByLang((prev) => ({ ...prev, [key]: map || {} }));
|
||||
}, []);
|
||||
const [incrementalPlan, setIncrementalPlan] = useState(null);
|
||||
|
||||
const recomputeIncremental = useCallback(async () => {
|
||||
@@ -232,16 +295,19 @@ export default function useSegmentEditing() {
|
||||
}
|
||||
try {
|
||||
// Same payload shape as the generate request (utils/segments.js) so
|
||||
// stored fingerprints actually match unchanged segments (#281).
|
||||
// stored fingerprints actually match unchanged segments (#281). `lang`
|
||||
// must match the language the generate run hashed with — it's part of
|
||||
// the fingerprint now (P1.3).
|
||||
const res = await apiPost('/tools/incremental', {
|
||||
segments: dubSegments.map((s) => ({ id: String(s.id), ...segmentGenInputs(s) })),
|
||||
stored_hashes: lastGenFingerprints,
|
||||
lang: dubLangCode,
|
||||
});
|
||||
setIncrementalPlan({ stale: res.stale, fresh: res.fresh });
|
||||
} catch (e) {
|
||||
console.warn('incremental plan failed', e);
|
||||
}
|
||||
}, [dubSegments, lastGenFingerprints]);
|
||||
}, [dubSegments, lastGenFingerprints, dubLangCode]);
|
||||
|
||||
return {
|
||||
// Undo/Redo
|
||||
@@ -275,6 +341,10 @@ export default function useSegmentEditing() {
|
||||
// Incremental plan
|
||||
lastGenFingerprints,
|
||||
setLastGenFingerprints,
|
||||
// Per-language fingerprint store (P1.3) — for project save/load and dub
|
||||
// history restore, which persist/rehydrate ALL tracks' hashes at once.
|
||||
fingerprintsByLang,
|
||||
setFingerprintsByLang,
|
||||
incrementalPlan,
|
||||
setIncrementalPlan,
|
||||
recomputeIncremental,
|
||||
|
||||
@@ -348,6 +348,7 @@
|
||||
"llmp_save": "Save",
|
||||
"llmp_save_active": "Save & use for translation",
|
||||
"llmp_save_keep": "Save & keep active",
|
||||
"llmp_saved_not_active": "Saved — not yet used for translation. Click “Save & use for translation” to switch.",
|
||||
"llmp_test": "Test",
|
||||
"llmp_active_badge": "active",
|
||||
"llmp_test_ok": "ok — {{model}} · {{ms}} ms",
|
||||
@@ -892,6 +893,11 @@
|
||||
"burn_subs_title": "Render subtitles directly into the MP4 video stream (hardsubs). Uses the dual-subtitle format when Dual subtitles is on.",
|
||||
"timing_smart_fit": "Smart Fit",
|
||||
"timing_smart_fit_title": "Splits the difference: slightly speeds up the audio (pitch preserved, up to 1.5×) and slightly slows down that segment of the video (up to 2×) so natural-rate speech fits. Anything beyond the caps is trimmed and flagged. Export re-encodes the video.",
|
||||
"timing_concise": "Concise",
|
||||
"timing_stretch_video": "Stretch Video",
|
||||
"timing_strict_slot": "Strict slot",
|
||||
"track_tip_duration": "Duration {{duration}}",
|
||||
"track_tip_timing": "Timing {{strategy}}",
|
||||
"default_track": "Default Track:",
|
||||
"original_track": "Original",
|
||||
"selected_dub": "{{code}} (Selected Dub)",
|
||||
@@ -922,6 +928,8 @@
|
||||
"hq_needs_llm_hint": "High-quality translation fits each line to its segment time using a local or cloud LLM. Set one up to enable it.",
|
||||
"set_up_llm": "Set up",
|
||||
"generate_dub_multi": "Generate {{count}} dubs",
|
||||
"multi_translating": "Translating → {{lang}} ({{current}}/{{total}})…",
|
||||
"multi_lang_skipped": "Translation failed for {{langs}} — those dubs were skipped so you never get a wrong-language track.",
|
||||
"pipeline": "Dubbing pipeline",
|
||||
"preview_language": "Preview language",
|
||||
"target_language": "Dub into",
|
||||
@@ -1180,8 +1188,8 @@
|
||||
"no_matches": "No voices match these filters.",
|
||||
"load_more": "Load more",
|
||||
"saved_as_profile": "Added \"{{name}}\" to your voices.",
|
||||
"use_failed": "Could not create that voice — the engine may be loading.",
|
||||
"preview_failed": "Preview unavailable — the voice engine may still be loading.",
|
||||
"use_failed": "Could not create that voice: {{message}}",
|
||||
"preview_failed": "Preview unavailable: {{message}}",
|
||||
"import_explainer": "Paste a URL you have the rights to (or upload a file), trim the part you need, and save it as a voice. You are responsible for the licensing of anything you import.",
|
||||
"import_placeholder": "Paste a video/audio URL, or type to search…",
|
||||
"imported_clip": "Imported clip",
|
||||
@@ -1191,11 +1199,12 @@
|
||||
"no_imports": "Nothing imported yet. Paste a URL above to get started.",
|
||||
"search_results": "{{count}} results",
|
||||
"download_failed": "Download failed: {{msg}}",
|
||||
"search_failed": "Search failed.",
|
||||
"upload_failed": "Upload failed.",
|
||||
"save_failed": "Could not save profile.",
|
||||
"search_failed": "Search failed: {{message}}",
|
||||
"upload_failed": "Upload failed: {{message}}",
|
||||
"save_failed": "Could not save profile: {{message}}",
|
||||
"confirm_delete": "Delete \"{{name}}\"?",
|
||||
"trim_load_failed": "Could not load audio for trimming.",
|
||||
"delete_failed": "Could not delete: {{message}}",
|
||||
"trim_load_failed": "Could not load audio for trimming: {{message}}",
|
||||
"delete": "Delete",
|
||||
"community_empty": "No community voices loaded yet — connect to the internet and reopen, or be the first to submit one.",
|
||||
"community_explainer": "Designed presets and recorded voices shared by the community, loaded from the omnivoice-gallery.",
|
||||
@@ -1283,7 +1292,11 @@
|
||||
"try_dictation": "Try dictation",
|
||||
"next_try_dictation": "Next: Try dictation",
|
||||
"step_aria": "Step {{num}}: {{label}}",
|
||||
"step_completed": "completed"
|
||||
"step_completed": "completed",
|
||||
"mirror_rescue_title": "Can't reach Hugging Face? Use a mirror",
|
||||
"mirror_rescue_hint": "Model downloads switch to the mirror immediately — no restart needed during setup. hf-mirror.com is the community mirror commonly used in China.",
|
||||
"mirror_apply": "Apply & re-check",
|
||||
"mirror_apply_error": "Could not save the mirror setting."
|
||||
},
|
||||
"donate": {
|
||||
"title": "Support",
|
||||
@@ -1541,6 +1554,7 @@
|
||||
"loading": "Loading engines…",
|
||||
"refresh": "Refresh",
|
||||
"matrixTitle": "Engine Compatibility Matrix",
|
||||
"familyMatrixTitle": "{{family}} Engines",
|
||||
"loadFailed": "Failed to load engines: {{message}}",
|
||||
"couldNotLoad": "Could not load engines: {{message}}",
|
||||
"retry": "Retry",
|
||||
@@ -1580,7 +1594,9 @@
|
||||
"routingUnknown": "Unknown",
|
||||
"routingEffectiveChip": "Runs on {{device}} on this machine",
|
||||
"routingCaveatTitle": "GPU selected, but: {{reason}}",
|
||||
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}"
|
||||
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}",
|
||||
"curatedModelLabel": "Model",
|
||||
"curatedModelAria": "Model for {{engine}}"
|
||||
},
|
||||
"errors": {
|
||||
"title": "This tab hit a snag.",
|
||||
@@ -1591,6 +1607,20 @@
|
||||
"searchIssues": "Search similar issues",
|
||||
"unexpected": "Unexpected error: {{message}}"
|
||||
},
|
||||
"crash": {
|
||||
"notice": "The voice backend crashed ({{exit}}) {{ago}} ago and is being restarted automatically.",
|
||||
"view": "View crash details",
|
||||
"dismiss": "Dismiss",
|
||||
"details_title": "Backend crash details",
|
||||
"details_intro": "The backend process died unexpectedly ({{exit}}) {{ago}} ago. The error output it left behind is below — reporting it helps us fix the crash.",
|
||||
"field_exit": "Exit",
|
||||
"field_when": "When",
|
||||
"field_uptime": "Uptime before crash",
|
||||
"field_version": "Backend version",
|
||||
"uptime_value": "{{count}} s",
|
||||
"stderr_title": "Last error output (stderr)",
|
||||
"no_stderr": "No error output was captured."
|
||||
},
|
||||
"common": {
|
||||
"open": "Open",
|
||||
"cancel": "Cancel",
|
||||
@@ -1737,6 +1767,7 @@
|
||||
"flush": "Flush",
|
||||
"loaded_models": "Loaded Models",
|
||||
"no_models": "No models loaded",
|
||||
"model_not_active": "not active — safe to unload",
|
||||
"unload": "Unload",
|
||||
"flush_caches": "Flush caches",
|
||||
"unload_all_flush": "Unload all + flush"
|
||||
@@ -1985,7 +2016,18 @@
|
||||
"mirror_preset_hint": "On a restricted network, route model downloads through a mirror. Leave empty for the official endpoint.",
|
||||
"mirror_restart_note": "Model Store downloads use the new mirror immediately. Only model loads (transformers) pick it up after a restart.",
|
||||
"mirror_load_error": "Failed to load mirror setting",
|
||||
"mirror_save_error": "Failed to save"
|
||||
"mirror_save_error": "Failed to save",
|
||||
"asrOpenAICompatTitle": "OpenAI-compatible ASR (remote server)",
|
||||
"asrOpenAICompatDescription": "Point transcription at Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own API.",
|
||||
"asrOpenAICompatBaseUrlTitle": "Server URL",
|
||||
"asrOpenAICompatBaseUrlHint": "The base URL of an OpenAI-compatible transcription server. To use this engine, also set OMNIVOICE_ASR_BACKEND=openai-compat-asr — there's no in-app engine picker for ASR yet.",
|
||||
"asrOpenAICompatModelTitle": "Model",
|
||||
"asrOpenAICompatApiKeyTitle": "API key",
|
||||
"asrOpenAICompatApiKeyHint": "Optional — many self-hosted servers don't require one.",
|
||||
"asrOpenAICompatApiKeyOptional": "optional",
|
||||
"asrOpenAICompatKeyConfigured": "A key is saved. Leave blank to keep it, or type a new one to replace it.",
|
||||
"asrOpenAICompatLoadError": "Failed to load ASR server setting",
|
||||
"asrOpenAICompatSaveError": "Failed to save"
|
||||
},
|
||||
"enterprise_faq": {
|
||||
"q_internal_tools": "Do I need a license for internal tools?",
|
||||
|
||||
@@ -515,7 +515,9 @@
|
||||
"routingRemote": "远程",
|
||||
"routingUnknown": "未知",
|
||||
"routingEffectiveChip": "在此机器上的 {{device}} 上运行",
|
||||
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}"
|
||||
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}",
|
||||
"curatedModelLabel": "模型",
|
||||
"curatedModelAria": "{{engine}} 的模型"
|
||||
},
|
||||
"capture": {
|
||||
"desc": "全局热键仅在桌面应用中有效。网页界面使用页面内快捷键 <1>Ctrl+Shift+Space</1>(窗口聚焦时可用)。",
|
||||
@@ -1144,7 +1146,11 @@
|
||||
"next_try_dictation": "下一步:试用听写",
|
||||
"step_aria": "第 {{num}} 步:{{label}}",
|
||||
"step_completed": "已完成",
|
||||
"cache_label": "模型缓存"
|
||||
"cache_label": "模型缓存",
|
||||
"mirror_rescue_title": "无法连接 Hugging Face?使用镜像",
|
||||
"mirror_rescue_hint": "模型下载会立即切换到镜像地址,安装过程无需重启。hf-mirror.com 是国内常用的社区镜像。",
|
||||
"mirror_apply": "应用并重新检查",
|
||||
"mirror_apply_error": "镜像设置保存失败。"
|
||||
},
|
||||
"donate": {
|
||||
"title": "支持项目",
|
||||
@@ -1504,6 +1510,7 @@
|
||||
"flush": "冲洗",
|
||||
"loaded_models": "加载模型",
|
||||
"no_models": "没有加载模型",
|
||||
"model_not_active": "非当前引擎 — 可安全卸载",
|
||||
"unload": "卸载",
|
||||
"flush_caches": "刷新缓存",
|
||||
"unload_all_flush": "全部卸载+冲洗"
|
||||
|
||||
@@ -84,7 +84,12 @@ export default function DubTab(props) {
|
||||
const dubLang = useAppStore((s) => s.dubLang);
|
||||
const setDubLang = useAppStore((s) => s.setDubLang);
|
||||
const dubLangCode = useAppStore((s) => s.dubLangCode);
|
||||
const setDubLangCode = useAppStore((s) => s.setDubLangCode);
|
||||
// User-driven language switches go through switchDubLangCode (P1.2): it
|
||||
// swaps segment text through the per-language `translations` map instead
|
||||
// of leaving the previous language's text on screen (and previously,
|
||||
// letting the next translate destroy it). Non-user rehydration paths
|
||||
// (project load, history restore) keep the plain setter.
|
||||
const switchDubLangCode = useAppStore((s) => s.switchDubLangCode);
|
||||
const dubNumSpeakers = useAppStore((s) => s.dubNumSpeakers);
|
||||
const setDubNumSpeakers = useAppStore((s) => s.setDubNumSpeakers);
|
||||
const dubDialect = useAppStore((s) => s.dubDialect);
|
||||
@@ -188,31 +193,96 @@ export default function DubTab(props) {
|
||||
const [exportOpen, setExportOpen] = useState(false);
|
||||
const [qcRunning, setQcRunning] = useState(false);
|
||||
|
||||
// Multi-language mode
|
||||
const [multiLangMode, setMultiLangMode] = useState(false);
|
||||
const [multiLangs, setMultiLangs] = useState([]);
|
||||
// Multi-language mode — store-backed (P1.4) so the picks survive tab
|
||||
// switches and ride the project save/load payload.
|
||||
const multiLangMode = useAppStore((s) => s.multiLangMode);
|
||||
const setMultiLangMode = useAppStore((s) => s.setMultiLangMode);
|
||||
const multiLangs = useAppStore((s) => s.multiLangs);
|
||||
const setMultiLangs = useAppStore((s) => s.setMultiLangs);
|
||||
// Landing "Advanced" disclosure (pre-upload options).
|
||||
const [landingAdvOpen, setLandingAdvOpen] = useState(false);
|
||||
|
||||
// Generate CTA — when multi-language mode has picks, dub each language
|
||||
// sequentially; every run appends its track to dubbed_tracks, so the
|
||||
// preview switcher pills fill up one by one.
|
||||
//
|
||||
// P1.1: each language is TRANSLATED first (`handleTranslateAll(code)`), then
|
||||
// generated — the backend synthesizes segment text verbatim, so without the
|
||||
// translate pass every "multi-language" track rendered the same words.
|
||||
// A pick whose translate fails is skipped (never render a wrong-language
|
||||
// track); the batch continues and the skips are reported at the end.
|
||||
const multiBatchRunningRef = useRef(false);
|
||||
const onGenerateClick = useCallback(async () => {
|
||||
if (multiLangMode && multiLangs.length > 0) {
|
||||
if (multiBatchRunningRef.current) return; // ignore re-clicks mid-batch
|
||||
multiBatchRunningRef.current = true;
|
||||
const skipped = [];
|
||||
// Skip the redundant translate ONLY for the first pick, and only when
|
||||
// it targets the language the editor text is already in (every segment
|
||||
// carries a translation differing from its original — i.e. the user
|
||||
// just ran Translate All into this exact language). After the first
|
||||
// pick the editor text is the previous pick's language, so every later
|
||||
// pick always translates. Correctness beats cleverness.
|
||||
const editorAlreadyTranslated =
|
||||
dubSegments.length > 0 &&
|
||||
dubSegments.every((s) => s.text_original && s.text !== s.text_original);
|
||||
try {
|
||||
for (const l of multiLangs) {
|
||||
for (let i = 0; i < multiLangs.length; i++) {
|
||||
const l = multiLangs[i];
|
||||
setDubLang(l.lang);
|
||||
setDubLangCode(l.code); // keep UI/exports in sync
|
||||
// Keep UI/exports in sync AND snapshot the previous pick's
|
||||
// translations before this pick's translate pass overwrites the
|
||||
// visible text (P1.2).
|
||||
switchDubLangCode(l.code);
|
||||
const skipTranslate = i === 0 && l.code === dubLangCode && editorAlreadyTranslated;
|
||||
if (!skipTranslate) {
|
||||
// Honest phase label: this pill slot otherwise only says
|
||||
// "Generating…", hiding the translate pass entirely.
|
||||
useAppStore.getState().showPill(
|
||||
'translating',
|
||||
t('dub.multi_translating', {
|
||||
lang: l.lang,
|
||||
current: i + 1,
|
||||
total: multiLangs.length,
|
||||
}),
|
||||
{ homeMode: 'dub' },
|
||||
);
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
const ok = await handleTranslateAll(l.code);
|
||||
if (!ok) {
|
||||
// Error already surfaced by handleTranslateAll (banner/toast);
|
||||
// drop the phase pill and move on to the next language.
|
||||
useAppStore.getState().dismissPill();
|
||||
skipped.push(l.lang);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
await handleDubGenerate({ langOverride: { language: l.lang, language_code: l.code } });
|
||||
}
|
||||
} catch {
|
||||
/* a failed language stops the batch; its error is already surfaced */
|
||||
}
|
||||
multiBatchRunningRef.current = false;
|
||||
if (skipped.length) {
|
||||
toast.error(t('dub.multi_lang_skipped', { langs: skipped.join(', ') }), {
|
||||
duration: 8000,
|
||||
});
|
||||
}
|
||||
} else {
|
||||
handleDubGenerate();
|
||||
}
|
||||
}, [multiLangMode, multiLangs, handleDubGenerate, setDubLang, setDubLangCode]);
|
||||
}, [
|
||||
multiLangMode,
|
||||
multiLangs,
|
||||
dubSegments,
|
||||
dubLangCode,
|
||||
handleTranslateAll,
|
||||
handleDubGenerate,
|
||||
setDubLang,
|
||||
switchDubLangCode,
|
||||
t,
|
||||
]);
|
||||
|
||||
// Live ETA while generating — elapsed ticks each second; remaining is
|
||||
// extrapolated from the current/total rate so it's only meaningful once
|
||||
@@ -349,11 +419,12 @@ export default function DubTab(props) {
|
||||
});
|
||||
setIngestUrl('');
|
||||
};
|
||||
const hasDubbedTrack =
|
||||
dubStep === 'done' &&
|
||||
dubLangCode &&
|
||||
dubLangCode !== 'und' &&
|
||||
(dubTracks?.length > 0 || !!dubTracks);
|
||||
// Track-switcher visibility is keyed to the persisted tracks ONLY — not the
|
||||
// language dropdown. Restored projects can carry finished tracks while
|
||||
// dubLangCode reads 'und' (older dub_history rows froze language_code at
|
||||
// ""), and the old `dubLangCode !== 'und'` guard hid their tabs until the
|
||||
// user re-picked a language.
|
||||
const hasDubbedTrack = dubStep === 'done' && dubTracks.length > 0;
|
||||
// Cache-busting nonce, bumped every time a generation completes (see
|
||||
// useDubWorkflow's done handler). The preview URL is otherwise identical
|
||||
// across re-dubs, so the WebView could keep serving the previously
|
||||
@@ -367,9 +438,14 @@ export default function DubTab(props) {
|
||||
: `${API}/dub/media/${dubJobId}`;
|
||||
// When a dub finishes, jump the preview to the freshly-dubbed language so the
|
||||
// result plays immediately — the user can tap back to Original any time.
|
||||
// Membership guard: only jump to a language that actually has a track,
|
||||
// otherwise fall back to the first track. Restored projects can have
|
||||
// dubLangCode out of sync with the tracks (e.g. 'en'/'und' with tracks
|
||||
// ['bn']) and an unguarded jump would point the player at
|
||||
// /dub/preview-video?lang=en — a guaranteed 404.
|
||||
useEffect(() => {
|
||||
if (hasDubbedTrack && previewMode === 'original' && dubLangCode && dubLangCode !== 'und') {
|
||||
setPreviewMode(dubLangCode);
|
||||
if (hasDubbedTrack && previewMode === 'original') {
|
||||
setPreviewMode(dubTracks.includes(dubLangCode) ? dubLangCode : dubTracks[0]);
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [hasDubbedTrack, dubLangCode]);
|
||||
@@ -478,7 +554,7 @@ export default function DubTab(props) {
|
||||
fetchYtSubs={fetchYtSubs}
|
||||
setFetchYtSubs={setFetchYtSubs}
|
||||
dubLangCode={dubLangCode}
|
||||
setDubLangCode={setDubLangCode}
|
||||
setDubLangCode={switchDubLangCode}
|
||||
setDubLang={setDubLang}
|
||||
landingAdvOpen={landingAdvOpen}
|
||||
setLandingAdvOpen={setLandingAdvOpen}
|
||||
@@ -502,6 +578,7 @@ export default function DubTab(props) {
|
||||
handleDubStop={handleDubStop}
|
||||
dubProgress={dubProgress}
|
||||
onGenerateClick={onGenerateClick}
|
||||
isTranslating={isTranslating}
|
||||
multiLangMode={multiLangMode}
|
||||
multiLangs={multiLangs}
|
||||
incrementalPlan={incrementalPlan}
|
||||
@@ -550,7 +627,7 @@ export default function DubTab(props) {
|
||||
hasAnyTranslation={hasAnyTranslation}
|
||||
handleCleanupSegments={handleCleanupSegments}
|
||||
setDubLang={setDubLang}
|
||||
setDubLangCode={setDubLangCode}
|
||||
setDubLangCode={switchDubLangCode}
|
||||
dubDialect={dubDialect}
|
||||
setDubDialect={setDubDialect}
|
||||
i18n={i18n}
|
||||
|
||||
@@ -22,6 +22,7 @@ import StoragePanel from '../components/settings/StoragePanel';
|
||||
import StorageTab from '../components/settings/StorageTab';
|
||||
import StorageUsagePanel from '../components/settings/StorageUsagePanel';
|
||||
import HFMirrorPanel from '../components/settings/HFMirrorPanel';
|
||||
import AsrOpenAICompatPanel from '../components/settings/AsrOpenAICompatPanel';
|
||||
import SharingPanel from '../components/settings/SharingPanel';
|
||||
import RemoteBackendPanel from '../components/settings/RemoteBackendPanel';
|
||||
import MCPBindingsPanel from '../components/settings/MCPBindingsPanel';
|
||||
@@ -367,6 +368,7 @@ export default function Settings() {
|
||||
<>
|
||||
<StoragePanel />
|
||||
<HFMirrorPanel />
|
||||
<AsrOpenAICompatPanel />
|
||||
<ModelStoreTab info={info} modelBadge={modelBadge} />
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -3,6 +3,7 @@ import { useTranslation } from 'react-i18next';
|
||||
import { Loader, RotateCw } from 'lucide-react';
|
||||
import { cn } from '@/lib/utils';
|
||||
import { useSetupStatus, usePreflight } from '../api/hooks';
|
||||
import { apiJson, apiFetch } from '../api/client';
|
||||
import WizardLibrary from '../components/WizardLibrary';
|
||||
import HfTokenCard from '../components/HfTokenCard';
|
||||
import DictationDemo from '../components/DictationDemo';
|
||||
@@ -133,6 +134,101 @@ function PreflightPanel({ report, loading, onRecheck }) {
|
||||
);
|
||||
}
|
||||
|
||||
/* ── Mirror rescue — restricted-network escape hatch on step 0 ─────────── */
|
||||
|
||||
/**
|
||||
* Shown when the network preflight check can't reach the Hugging Face
|
||||
* endpoint. Users behind restricted networks (e.g. China, where
|
||||
* huggingface.co is blocked) can't reach Settings yet — the wizard gates the
|
||||
* studio — so the mirror quick-pick has to live right here. PUT /hf-mirror
|
||||
* takes effect immediately for downloads (no restart during setup).
|
||||
*/
|
||||
function MirrorRescue({ onApplied }) {
|
||||
const { t } = useTranslation();
|
||||
const [presets, setPresets] = useState([]);
|
||||
const [url, setUrl] = useState('');
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
|
||||
useEffect(() => {
|
||||
let alive = true;
|
||||
apiJson('/api/settings/hf-mirror')
|
||||
.then((d) => {
|
||||
if (!alive) return;
|
||||
setPresets((d?.presets || []).filter((p) => p.url));
|
||||
setUrl(d?.configured || '');
|
||||
})
|
||||
.catch(() => {
|
||||
/* endpoint unavailable — keep the free-text input usable */
|
||||
});
|
||||
return () => {
|
||||
alive = false;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const apply = async (value) => {
|
||||
setSaving(true);
|
||||
setError(null);
|
||||
try {
|
||||
await apiFetch('/api/settings/hf-mirror', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url: value }),
|
||||
});
|
||||
setUrl(value);
|
||||
onApplied();
|
||||
} catch (e) {
|
||||
setError(e?.message || t('setup.mirror_apply_error'));
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="mt-3 flex flex-col gap-1.5 rounded-md border border-border px-3 py-2.5">
|
||||
<span className="text-sm font-semibold">{t('setup.mirror_rescue_title')}</span>
|
||||
<span className="text-xs leading-snug text-fg-muted">{t('setup.mirror_rescue_hint')}</span>
|
||||
{error && (
|
||||
<span className="text-xs text-danger" role="alert">
|
||||
{error}
|
||||
</span>
|
||||
)}
|
||||
<div className="mt-1 flex flex-wrap items-center gap-2">
|
||||
{presets.map((p) => (
|
||||
<Button
|
||||
key={p.url}
|
||||
variant="preset"
|
||||
size="sm"
|
||||
disabled={saving}
|
||||
onClick={() => apply(p.url)}
|
||||
data-testid={`wizard-mirror-${p.url}`}
|
||||
>
|
||||
{p.label}
|
||||
</Button>
|
||||
))}
|
||||
<input
|
||||
type="text"
|
||||
value={url}
|
||||
onChange={(e) => setUrl(e.target.value)}
|
||||
placeholder="https://hf-mirror.com"
|
||||
className="min-w-[220px] flex-1 rounded border border-border bg-transparent px-2 py-1 font-mono text-xs text-fg"
|
||||
data-testid="wizard-mirror-url"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
loading={saving}
|
||||
disabled={saving || !url.trim()}
|
||||
onClick={() => apply(url.trim())}
|
||||
data-testid="wizard-mirror-apply"
|
||||
>
|
||||
{t('setup.mirror_apply')}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
/* ── LED stepper rail ──────────────────────────────────────────────────── */
|
||||
|
||||
function StepperNav({ step, maxUnlockedStep, onStep }) {
|
||||
@@ -241,6 +337,10 @@ export default function SetupWizard({ onReady }) {
|
||||
|
||||
const modelsReady = !!status?.models_ready;
|
||||
const preflightOk = !!pre?.ok;
|
||||
// Offer the mirror quick-pick whenever the HF endpoint probe didn't pass —
|
||||
// the wizard is the only surface these users can reach (Settings is gated
|
||||
// behind setup), so the escape hatch must live here.
|
||||
const networkDown = (pre?.checks || []).some((c) => c.id === 'network' && c.status !== 'pass');
|
||||
|
||||
const cachePath = status?.hf_cache_dir || '~/.cache/huggingface';
|
||||
|
||||
@@ -288,6 +388,7 @@ export default function SetupWizard({ onReady }) {
|
||||
<div className="flex min-h-0 flex-auto flex-col gap-3" key="step-0">
|
||||
<div className="fr-rise min-h-0 flex-1 overflow-y-auto" style={{ '--rise': 1 }}>
|
||||
<PreflightPanel report={pre} loading={preLoading} onRecheck={recheckPreflight} />
|
||||
{networkDown && <MirrorRescue onApplied={recheckPreflight} />}
|
||||
</div>
|
||||
<div
|
||||
className="fr-rise flex shrink-0 items-center justify-between gap-4 border-t border-border pt-3"
|
||||
|
||||
@@ -138,7 +138,8 @@ export default function VoiceGallery() {
|
||||
stopPlayback();
|
||||
flash(
|
||||
t('gallery.preview_failed', {
|
||||
defaultValue: 'Preview unavailable — the voice engine may still be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Preview unavailable: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
@@ -222,7 +223,8 @@ export default function VoiceGallery() {
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
defaultValue: 'Could not create that voice — the engine may be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -45,6 +45,12 @@ interface DubPrepProgress {
|
||||
/** Segments are a loose shape — many optional fields added over time. */
|
||||
type DubSegment = Record<string, unknown> & { id: string; text: string };
|
||||
|
||||
/** One multi-language batch pick — display name + ISO code (MultiLangPicker). */
|
||||
export interface MultiLangPick {
|
||||
lang: string;
|
||||
code: string;
|
||||
}
|
||||
|
||||
type Updater<T> = T | ((prev: T) => T);
|
||||
|
||||
function resolve<T>(updater: Updater<T>, prev: T): T {
|
||||
@@ -103,6 +109,13 @@ export interface DubSlice {
|
||||
// paths (OpenAI/Ollama provider or Cinematic quality).
|
||||
dubDialect: string;
|
||||
|
||||
// Multi-language batch mode (P1.4) — the checkbox + language picks used by
|
||||
// the "Generate N dubs" loop. Lived in DubTab component state before, so a
|
||||
// tab switch or project reload silently dropped the picks; now they ride
|
||||
// the store and the project save/load payload.
|
||||
multiLangMode: boolean;
|
||||
multiLangs: MultiLangPick[];
|
||||
|
||||
// ── Generation options ────────────────────────────────────────────────
|
||||
dubInstruct: string;
|
||||
preserveBg: boolean;
|
||||
@@ -156,8 +169,21 @@ export interface DubSlice {
|
||||
bumpDubGenNonce: () => void;
|
||||
setDubLang: (v: Updater<string>) => void;
|
||||
setDubLangCode: (v: Updater<string>) => void;
|
||||
/**
|
||||
* User-driven target-language switch (P1.2). Unlike the plain setter it
|
||||
* also remaps segment text through the per-language `translations` store:
|
||||
* the outgoing language's text is snapshotted into `translations[prev]`
|
||||
* (only when it's an actual translation — differs from `text_original`),
|
||||
* and the incoming language's saved text is swapped into `text` when one
|
||||
* exists. Non-destructive: with no saved entry, `text` is left untouched —
|
||||
* exactly the legacy behaviour. Restore/rehydrate paths (project load, dub
|
||||
* history) must keep using `setDubLangCode`, which never touches segments.
|
||||
*/
|
||||
switchDubLangCode: (code: string) => void;
|
||||
setDubNumSpeakers: (v: Updater<number | null>) => void;
|
||||
setDubDialect: (v: Updater<string>) => void;
|
||||
setMultiLangMode: (v: Updater<boolean>) => void;
|
||||
setMultiLangs: (v: Updater<MultiLangPick[]>) => void;
|
||||
setDubInstruct: (v: Updater<string>) => void;
|
||||
setPreserveBg: (v: Updater<boolean>) => void;
|
||||
setDefaultTrack: (v: Updater<string>) => void;
|
||||
@@ -190,8 +216,11 @@ const INITIAL: Omit<
|
||||
| 'bumpDubGenNonce'
|
||||
| 'setDubLang'
|
||||
| 'setDubLangCode'
|
||||
| 'switchDubLangCode'
|
||||
| 'setDubNumSpeakers'
|
||||
| 'setDubDialect'
|
||||
| 'setMultiLangMode'
|
||||
| 'setMultiLangs'
|
||||
| 'setDubInstruct'
|
||||
| 'setPreserveBg'
|
||||
| 'setDefaultTrack'
|
||||
@@ -223,6 +252,8 @@ const INITIAL: Omit<
|
||||
dubLangCode: 'en',
|
||||
dubNumSpeakers: null,
|
||||
dubDialect: '',
|
||||
multiLangMode: false,
|
||||
multiLangs: [],
|
||||
dubInstruct: '',
|
||||
preserveBg: true,
|
||||
defaultTrack: 'original',
|
||||
@@ -255,8 +286,33 @@ export const createDubSlice: StateCreator<DubSlice, [], [], DubSlice> = (set, ge
|
||||
bumpDubGenNonce: () => set(() => ({ dubGenNonce: Date.now() })),
|
||||
setDubLang: (v) => set((s) => ({ dubLang: resolve(v, s.dubLang) })),
|
||||
setDubLangCode: (v) => set((s) => ({ dubLangCode: resolve(v, s.dubLangCode) })),
|
||||
switchDubLangCode: (code) =>
|
||||
set((s) => {
|
||||
const prev = s.dubLangCode;
|
||||
if (!code || code === prev) return {};
|
||||
const dubSegments = s.dubSegments.map((seg) => {
|
||||
const translations: Record<string, string> = {
|
||||
...(seg.translations as Record<string, string> | undefined),
|
||||
};
|
||||
// Snapshot the outgoing language's text — but only real translations
|
||||
// (differs from the source), so a never-translated row can't stamp
|
||||
// source-language text as the previous language's translation.
|
||||
// Legacy projects (no `translations` yet) get theirs seeded here.
|
||||
const text = typeof seg.text === 'string' ? seg.text : '';
|
||||
if (prev && text.trim() && text !== seg.text_original) translations[prev] = text;
|
||||
const incoming = translations[code];
|
||||
return {
|
||||
...seg,
|
||||
translations,
|
||||
...(typeof incoming === 'string' && incoming.trim() ? { text: incoming } : {}),
|
||||
};
|
||||
});
|
||||
return { dubLangCode: code, dubSegments };
|
||||
}),
|
||||
setDubNumSpeakers: (v) => set((s) => ({ dubNumSpeakers: resolve(v, s.dubNumSpeakers) })),
|
||||
setDubDialect: (v) => set((s) => ({ dubDialect: resolve(v, s.dubDialect) })),
|
||||
setMultiLangMode: (v) => set((s) => ({ multiLangMode: resolve(v, s.multiLangMode) })),
|
||||
setMultiLangs: (v) => set((s) => ({ multiLangs: resolve(v, s.multiLangs) })),
|
||||
setDubInstruct: (v) => set((s) => ({ dubInstruct: resolve(v, s.dubInstruct) })),
|
||||
setPreserveBg: (v) => set((s) => ({ preserveBg: resolve(v, s.preserveBg) })),
|
||||
setDefaultTrack: (v) => set((s) => ({ defaultTrack: resolve(v, s.defaultTrack) })),
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
import React, { createRef } from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render, screen, waitFor } from '@testing-library/react';
|
||||
import i18n from '../i18n';
|
||||
|
||||
// P0.2 — track-bar polish: the Original/<lang> pills enrich with per-track
|
||||
// metadata (duration + timing strategy) hydrated lazily from the existing
|
||||
// GET /dub/tracks/{job_id}. The fetch must be failure-silent: the pills are
|
||||
// the P0 visibility fix and can never depend on the enrichment call.
|
||||
|
||||
vi.mock('../components/WaveformTimeline', () => ({ default: () => <div data-testid="wf" /> }));
|
||||
vi.mock('../components/MultiLangPicker', () => ({ default: () => <div data-testid="mlp" /> }));
|
||||
vi.mock('react-hot-toast', () => ({
|
||||
default: { error: vi.fn(), success: vi.fn(), loading: vi.fn() },
|
||||
}));
|
||||
const dubListTracks = vi.hoisted(() => vi.fn());
|
||||
vi.mock('../api/dub', () => ({ dubListTracks: (...a) => dubListTracks(...a) }));
|
||||
|
||||
import DubLeftColumn from '../components/dub/DubLeftColumn';
|
||||
|
||||
const t = i18n.t.bind(i18n);
|
||||
|
||||
function makeProps(over = {}) {
|
||||
return {
|
||||
hasDubbedTrack: true,
|
||||
t,
|
||||
i18n,
|
||||
previewMode: 'bn',
|
||||
setPreviewMode: vi.fn(),
|
||||
dubTracks: ['bn'],
|
||||
videoSrc: '',
|
||||
waveformRef: createRef(),
|
||||
dubJobId: 'job1',
|
||||
dubSegments: [{ id: '1', text: 'hi' }],
|
||||
timelineOnsets: [],
|
||||
timelineSelSegId: null,
|
||||
setTimelineSelSegId: vi.fn(),
|
||||
incrementalPlan: null,
|
||||
segmentMoveResize: vi.fn(),
|
||||
segmentDelete: vi.fn(),
|
||||
onTimelinePreviewSegment: vi.fn(),
|
||||
dubStep: 'done',
|
||||
dubProgress: { current: 0, total: 0, text: '' },
|
||||
fmtDur: (s) => `${s}s`,
|
||||
genElapsed: 0,
|
||||
genRemaining: null,
|
||||
speakerClones: {},
|
||||
setDubSegments: vi.fn(),
|
||||
profiles: [],
|
||||
settingsOpen: false,
|
||||
setSettingsOpen: vi.fn(),
|
||||
dubLang: 'Bengali',
|
||||
dubLangCode: 'bn',
|
||||
translateQuality: 'fast',
|
||||
activeEngineUnavailable: false,
|
||||
translateProvider: 'google',
|
||||
dubInstruct: '',
|
||||
setDubInstruct: vi.fn(),
|
||||
handleTranslateAll: vi.fn(),
|
||||
isTranslating: false,
|
||||
hasAnyTranslation: false,
|
||||
handleCleanupSegments: vi.fn(),
|
||||
setDubLang: vi.fn(),
|
||||
setDubLangCode: vi.fn(),
|
||||
dubDialect: '',
|
||||
setDubDialect: vi.fn(),
|
||||
enginesSandboxed: false,
|
||||
handleInstallEngine: vi.fn(),
|
||||
engineInstalling: null,
|
||||
activeEngineEntry: undefined,
|
||||
engines: [],
|
||||
setTranslateProvider: vi.fn(),
|
||||
setTranslateQuality: vi.fn(),
|
||||
llmEndpoint: { available: true },
|
||||
multiLangMode: false,
|
||||
setMultiLangMode: vi.fn(),
|
||||
multiLangs: [],
|
||||
setMultiLangs: vi.fn(),
|
||||
editSegments: vi.fn(),
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
describe('DubLeftColumn — track pill tooltips (P0.2)', () => {
|
||||
beforeEach(() => {
|
||||
dubListTracks.mockReset();
|
||||
});
|
||||
|
||||
it('hydrates duration + timing strategy from /dub/tracks and reflects the previewed track', async () => {
|
||||
dubListTracks.mockResolvedValue({
|
||||
bn: { duration: 72.4, timing_strategy: 'smart_fit', language: 'Bengali' },
|
||||
});
|
||||
render(<DubLeftColumn {...makeProps()} />);
|
||||
|
||||
const pill = screen.getByRole('radio', { name: 'Bengali' });
|
||||
// Selection indicator must track previewMode (accurate post-restore).
|
||||
expect(pill).toHaveAttribute('aria-checked', 'true');
|
||||
expect(screen.getByRole('radio', { name: t('dub.original_audio') })).toHaveAttribute(
|
||||
'aria-checked',
|
||||
'false',
|
||||
);
|
||||
|
||||
expect(dubListTracks).toHaveBeenCalledWith('job1');
|
||||
await waitFor(() => expect(pill).toHaveAttribute('title', 'Duration 72s · Timing Smart Fit'));
|
||||
});
|
||||
|
||||
it('is failure-silent: a failed metadata fetch leaves the pills fully usable', async () => {
|
||||
dubListTracks.mockRejectedValue(new Error('boom'));
|
||||
render(<DubLeftColumn {...makeProps()} />);
|
||||
|
||||
expect(dubListTracks).toHaveBeenCalledWith('job1');
|
||||
const pill = await screen.findByRole('radio', { name: 'Bengali' });
|
||||
await waitFor(() => expect(dubListTracks).toHaveBeenCalled());
|
||||
expect(pill).not.toHaveAttribute('title');
|
||||
});
|
||||
|
||||
it('does not call the endpoint when there are no dubbed tracks', () => {
|
||||
render(
|
||||
<DubLeftColumn
|
||||
{...makeProps({ hasDubbedTrack: false, dubTracks: [], dubStep: 'editing' })}
|
||||
/>,
|
||||
);
|
||||
expect(screen.queryByRole('radiogroup')).not.toBeInTheDocument();
|
||||
expect(dubListTracks).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,149 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render } from '@testing-library/react';
|
||||
import { useAppStore } from '../store';
|
||||
|
||||
// Regression guard for the "completed dub tracks' tabs hidden until the
|
||||
// language is re-selected" P0:
|
||||
// - `hasDubbedTrack` must key off the persisted tracks ONLY. The old
|
||||
// expression required `dubLangCode !== 'und'` and ended in a tautology
|
||||
// (`dubTracks?.length > 0 || !!dubTracks`), so a restored project with
|
||||
// finished tracks but a frozen language_code ('und') hid the track
|
||||
// switcher — and a job with NO tracks showed it.
|
||||
// - The done-state auto-jump must be membership-guarded: jumping the preview
|
||||
// to a dubLangCode that has no track (restores fall back to 'en' with
|
||||
// tracks ['bn']) pointed the player at /dub/preview-video?lang=en → 404.
|
||||
|
||||
// Heavy children are stubbed; DubLeftColumn is the probe — DubTab owns both
|
||||
// `hasDubbedTrack` and the previewMode auto-jump, and hands them down as props.
|
||||
const captured = vi.hoisted(() => ({ left: [] }));
|
||||
vi.mock('../components/dub/DubLeftColumn', () => ({
|
||||
default: (props) => {
|
||||
captured.left.push(props);
|
||||
return <div data-testid="left-col" />;
|
||||
},
|
||||
}));
|
||||
vi.mock('../components/dub/DubHeader', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubRightColumn', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubFooter', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubPipelineStepper', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/IdleSkeleton', () => ({ default: () => null }));
|
||||
vi.mock('../components/ExportModal', () => ({ default: () => null }));
|
||||
vi.mock('../hooks/useTimelineOnsets', () => ({ default: () => ({ onsets: [] }) }));
|
||||
vi.mock('../api/dub', () => ({
|
||||
dubQc: vi.fn(),
|
||||
dubListTracks: vi.fn(() => new Promise(() => {})),
|
||||
}));
|
||||
// Never-resolving async deps keep the render synchronous (no post-test act noise).
|
||||
vi.mock('../api/engines', () => ({
|
||||
listTranslationEngines: vi.fn(() => new Promise(() => {})),
|
||||
installTranslationEngine: vi.fn(),
|
||||
}));
|
||||
vi.mock('../api/client', async (importOriginal) => {
|
||||
const mod = await importOriginal();
|
||||
return { ...mod, apiJson: vi.fn(() => new Promise(() => {})) };
|
||||
});
|
||||
|
||||
import DubTab from '../pages/DubTab';
|
||||
|
||||
const noop = () => {};
|
||||
function makeProps() {
|
||||
return {
|
||||
dubVideoFile: null,
|
||||
dubLocalBlobUrl: null,
|
||||
transcribeElapsed: 0,
|
||||
translateProvider: 'google',
|
||||
setTranslateProvider: noop,
|
||||
showTranscript: false,
|
||||
setShowTranscript: noop,
|
||||
onGlossaryChange: noop,
|
||||
profiles: [],
|
||||
segmentPreviewLoading: null,
|
||||
selectedSegIds: new Set(),
|
||||
setDubVideoFile: noop,
|
||||
setDubLocalBlobUrl: noop,
|
||||
handleDubAbort: noop,
|
||||
handleDubUpload: noop,
|
||||
handleDubIngestUrl: noop,
|
||||
handleDubRetryTranscribe: noop,
|
||||
handleDubStop: noop,
|
||||
handleDubGenerate: noop,
|
||||
handleDubImportSrt: noop,
|
||||
handleDubDownload: noop,
|
||||
handleDubAudioDownload: noop,
|
||||
handleAudioExport: noop,
|
||||
handleSegmentPreview: noop,
|
||||
onDirectSegment: noop,
|
||||
handleTranslateAll: noop,
|
||||
handleCleanupSegments: noop,
|
||||
incrementalPlan: null,
|
||||
triggerDownload: noop,
|
||||
fileToMediaUrl: noop,
|
||||
editSegments: noop,
|
||||
saveProject: noop,
|
||||
resetDub: noop,
|
||||
segmentEditField: noop,
|
||||
segmentDelete: noop,
|
||||
segmentRestoreOriginal: noop,
|
||||
segmentSplit: noop,
|
||||
segmentMerge: noop,
|
||||
segmentMoveResize: noop,
|
||||
timelineSelSegId: null,
|
||||
setTimelineSelSegId: noop,
|
||||
toggleSegSelect: noop,
|
||||
selectAllSegs: noop,
|
||||
clearSegSelection: noop,
|
||||
bulkApplyToSelected: noop,
|
||||
bulkDeleteSelected: noop,
|
||||
};
|
||||
}
|
||||
|
||||
const baseState = useAppStore.getState();
|
||||
|
||||
function renderDone({ tracks, langCode, lang = 'Auto' }) {
|
||||
useAppStore.setState({
|
||||
dubJobId: 'job1',
|
||||
dubStep: 'done',
|
||||
dubTracks: tracks,
|
||||
dubLangCode: langCode,
|
||||
dubLang: lang,
|
||||
});
|
||||
render(<DubTab {...makeProps()} />);
|
||||
return captured.left.at(-1);
|
||||
}
|
||||
|
||||
describe('DubTab — completed tracks always show their tabs (restore P0)', () => {
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
captured.left.length = 0;
|
||||
});
|
||||
|
||||
it("restored project (tracks ['bn'], language_code frozen at 'und'): switcher shows and preview jumps to the track", () => {
|
||||
const left = renderDone({ tracks: ['bn'], langCode: 'und' });
|
||||
// Pre-fix: `dubLangCode !== 'und'` hid the finished tracks' tabs.
|
||||
expect(left.hasDubbedTrack).toBe(true);
|
||||
// Auto-jump falls back to the only real track — never a lang without one.
|
||||
expect(left.previewMode).toBe('bn');
|
||||
});
|
||||
|
||||
it("membership guard: dubLangCode 'en' with tracks ['bn'] previews tracks[0], not the 404 lang", () => {
|
||||
const left = renderDone({ tracks: ['bn'], langCode: 'en', lang: 'English' });
|
||||
expect(left.hasDubbedTrack).toBe(true);
|
||||
// Pre-fix the auto-jump previewed 'en' → /dub/preview-video?lang=en → 404.
|
||||
expect(left.previewMode).toBe('bn');
|
||||
});
|
||||
|
||||
it('dubLangCode that has a track previews that track (fresh-generate path unchanged)', () => {
|
||||
const left = renderDone({ tracks: ['bn', 'es'], langCode: 'es', lang: 'Spanish' });
|
||||
expect(left.hasDubbedTrack).toBe(true);
|
||||
expect(left.previewMode).toBe('es');
|
||||
});
|
||||
|
||||
it('done with NO persisted tracks hides the switcher and stays on Original (tautology guard)', () => {
|
||||
const left = renderDone({ tracks: [], langCode: 'es', lang: 'Spanish' });
|
||||
// Pre-fix `(dubTracks?.length > 0 || !!dubTracks)` was always true, so the
|
||||
// switcher appeared trackless and the auto-jump 404'd the preview.
|
||||
expect(left.hasDubbedTrack).toBe(false);
|
||||
expect(left.previewMode).toBe('original');
|
||||
});
|
||||
});
|
||||
@@ -702,4 +702,189 @@ describe('EngineCompatibilityMatrix', () => {
|
||||
// KittenTTS in the fixture carries no setup_snippet → no snippet block.
|
||||
expect(screen.queryByTestId('setup-snippet-kittentts')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
// ── #981 — mlx-audio curated-model picker ───────────────────────────────
|
||||
function mlxAudioResponse({ activeModelId = 'kokoro' } = {}) {
|
||||
return {
|
||||
tts: {
|
||||
active: 'mlx-audio',
|
||||
backends: [
|
||||
{
|
||||
id: 'mlx-audio',
|
||||
display_name: 'MLX-Audio (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['mps', 'cpu'],
|
||||
curated_models: [
|
||||
{
|
||||
key: 'kokoro',
|
||||
label: 'Kokoro (default, fast)',
|
||||
repo_id: 'mlx-community/Kokoro-82M-bf16',
|
||||
},
|
||||
{ key: 'csm', label: 'CSM (voice cloning)', repo_id: 'mlx-community/csm-1b-8bit' },
|
||||
{
|
||||
key: 'outetts',
|
||||
label: 'OuteTTS',
|
||||
repo_id: 'mlx-community/Llama-OuteTTS-1.0-1B-4bit',
|
||||
},
|
||||
],
|
||||
active_model_id: activeModelId,
|
||||
},
|
||||
],
|
||||
},
|
||||
asr: { active: '', backends: [] },
|
||||
llm: { active: 'off', backends: [] },
|
||||
};
|
||||
}
|
||||
|
||||
it('renders the curated-model picker for mlx-audio, pre-selected to the active model', async () => {
|
||||
const apiListEngines = vi
|
||||
.fn()
|
||||
.mockResolvedValue(mlxAudioResponse({ activeModelId: 'outetts' }));
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={vi.fn()}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
const select = screen.getByTestId('curated-model-select-mlx-audio');
|
||||
expect(select).toHaveValue('outetts');
|
||||
// All curated models are offered as options.
|
||||
expect(
|
||||
within(select).getByRole('option', { name: 'Kokoro (default, fast)' }),
|
||||
).toBeInTheDocument();
|
||||
expect(within(select).getByRole('option', { name: 'CSM (voice cloning)' })).toBeInTheDocument();
|
||||
expect(within(select).getByRole('option', { name: 'OuteTTS' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('picking a different curated model calls onSelect with the model key and refreshes', async () => {
|
||||
let activeModelId = 'kokoro';
|
||||
const apiListEngines = vi.fn(async () => mlxAudioResponse({ activeModelId }));
|
||||
const onSelect = vi.fn(async (_family, _id, modelId) => {
|
||||
activeModelId = modelId;
|
||||
});
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={onSelect}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
const select = screen.getByTestId('curated-model-select-mlx-audio');
|
||||
fireEvent.change(select, { target: { value: 'csm' } });
|
||||
|
||||
await waitFor(() => {
|
||||
expect(onSelect).toHaveBeenCalledWith('tts', 'mlx-audio', 'csm');
|
||||
});
|
||||
// Reloaded after the pick — matrix reflects the new active_model_id.
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('curated-model-select-mlx-audio')).toHaveValue('csm');
|
||||
});
|
||||
expect(apiListEngines.mock.calls.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('does not render a curated-model picker for engines without curated_models', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={vi.fn()}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('OmniVoice (test)'));
|
||||
expect(screen.queryByTestId(/curated-model-select-/)).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('disables the curated-model picker when no onSelect is provided', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(mlxAudioResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
expect(screen.getByTestId('curated-model-select-mlx-audio')).toBeDisabled();
|
||||
});
|
||||
|
||||
// ── showFamilyTabs={false} — pinned per-family mount (Settings → Engines) ─
|
||||
function multiFamilyResponse() {
|
||||
return {
|
||||
tts: {
|
||||
active: 'omnivoice',
|
||||
backends: [
|
||||
{
|
||||
id: 'omnivoice',
|
||||
display_name: 'OmniVoice (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
},
|
||||
],
|
||||
},
|
||||
asr: {
|
||||
active: 'whisperx',
|
||||
backends: [
|
||||
{
|
||||
id: 'whisperx',
|
||||
display_name: 'WhisperX (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
},
|
||||
],
|
||||
},
|
||||
llm: { active: 'off', backends: [] },
|
||||
};
|
||||
}
|
||||
|
||||
it('pins to the given family and hides the TTS/ASR/LLM switcher when showFamilyTabs is false', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="asr"
|
||||
showFamilyTabs={false}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
// Pinned header names the family instead of the generic matrix title…
|
||||
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
|
||||
// …the TTS family never leaks into the pinned table…
|
||||
expect(screen.queryByText('OmniVoice (test)')).not.toBeInTheDocument();
|
||||
// …and there is no family switcher to wander off to.
|
||||
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps the family switcher by default (standalone mounts unchanged)', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('OmniVoice (test)'));
|
||||
expect(screen.getByText('Engine Compatibility Matrix')).toBeInTheDocument();
|
||||
expect(document.querySelectorAll('.engine-matrix__tab-family').length).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import { describe, it, expect, vi, afterEach } from 'vitest';
|
||||
import { apiFetch } from '../api/client';
|
||||
import { getUnacknowledgedBackendCrash } from '../utils/backendCrash';
|
||||
|
||||
// #941: when the transport failure coincides with a recorded backend crash,
|
||||
// the vague "Can't reach the local OmniVoice backend" must become the honest
|
||||
// story — exit code + how long ago — and the crash-notice event must fire so
|
||||
// the UI can offer "View crash details".
|
||||
vi.mock('../utils/backendCrash', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../utils/backendCrash')>();
|
||||
return {
|
||||
...actual,
|
||||
getUnacknowledgedBackendCrash: vi.fn().mockResolvedValue(null),
|
||||
};
|
||||
});
|
||||
|
||||
const crashMock = vi.mocked(getUnacknowledgedBackendCrash);
|
||||
|
||||
function markerSecondsAgo(s: number) {
|
||||
return {
|
||||
ts: Math.floor(Date.now() / 1000) - s,
|
||||
exit_code: 3221226505,
|
||||
signal: null,
|
||||
exit_desc: 'exit code: 3221226505',
|
||||
backend_version: '0.3.10',
|
||||
uptime_s: 42,
|
||||
last_stderr: 'OSError: [WinError 1455] The paging file is too small',
|
||||
acknowledged: false,
|
||||
};
|
||||
}
|
||||
|
||||
describe('apiFetch — crash-marker honesty (#941)', () => {
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
vi.restoreAllMocks();
|
||||
vi.unstubAllGlobals();
|
||||
crashMock.mockClear();
|
||||
crashMock.mockResolvedValue(null);
|
||||
});
|
||||
|
||||
it('replaces the vague unreachable error with the honest crash story', async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new TypeError('Failed to fetch')));
|
||||
crashMock.mockResolvedValue(markerSecondsAgo(15));
|
||||
const events: unknown[] = [];
|
||||
const onCrash = (e: Event) => events.push((e as CustomEvent).detail);
|
||||
window.addEventListener('ov:backend-crashed', onCrash);
|
||||
|
||||
const p = apiFetch('/generate');
|
||||
const assertion = expect(p).rejects.toMatchObject({
|
||||
status: 0,
|
||||
// fake timers advance Date.now() during the retry backoff, so assert
|
||||
// the shape (exit code + a seconds-scale age), not an exact second.
|
||||
message: expect.stringMatching(/crashed \(exit code 3221226505\) \d+ s ago/),
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(400 + 900 + 1600 + 100);
|
||||
await assertion;
|
||||
|
||||
// The crash-notice affordance is driven by this event.
|
||||
expect(events).toHaveLength(1);
|
||||
expect(events[0]).toMatchObject({ exit_code: 3221226505 });
|
||||
window.removeEventListener('ov:backend-crashed', onCrash);
|
||||
});
|
||||
|
||||
it('keeps the generic message when no unacknowledged crash exists', async () => {
|
||||
vi.useFakeTimers();
|
||||
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new TypeError('Failed to fetch')));
|
||||
crashMock.mockResolvedValue(null);
|
||||
|
||||
const p = apiFetch('/generate');
|
||||
const assertion = expect(p).rejects.toMatchObject({
|
||||
status: 0,
|
||||
message: expect.stringContaining("Can't reach the local OmniVoice backend"),
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(400 + 900 + 1600 + 100);
|
||||
await assertion;
|
||||
});
|
||||
|
||||
it('never turns an HTTP error into a crash story (backend responded)', async () => {
|
||||
vi.stubGlobal(
|
||||
'fetch',
|
||||
vi.fn().mockResolvedValue(new Response('nope', { status: 500, statusText: 'Server Error' })),
|
||||
);
|
||||
crashMock.mockResolvedValue(markerSecondsAgo(5));
|
||||
await expect(apiFetch('/x')).rejects.toMatchObject({ status: 500 });
|
||||
expect(crashMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,205 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { render, act } from '@testing-library/react';
|
||||
import toast from 'react-hot-toast';
|
||||
import { useAppStore } from '../store';
|
||||
|
||||
// P1.1 — the multi-language generate loop must TRANSLATE each pick before it
|
||||
// generates it. Pre-fix the loop only called handleDubGenerate per language,
|
||||
// so "Generate 3 dubs" synthesized the same (untranslated) text three times —
|
||||
// at most one track was actually in its language.
|
||||
//
|
||||
// Contract under test (call order, per pick):
|
||||
// translate(code) → generate({ langOverride: { language, language_code } })
|
||||
// and on a failed translate: skip that pick's generate, keep going, report
|
||||
// the skipped languages in a final toast.
|
||||
|
||||
const captured = vi.hoisted(() => ({ header: [] }));
|
||||
vi.mock('../components/dub/DubHeader', () => ({
|
||||
default: (props) => {
|
||||
captured.header.push(props);
|
||||
return null;
|
||||
},
|
||||
}));
|
||||
vi.mock('../components/dub/DubLeftColumn', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubRightColumn', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubFooter', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/DubPipelineStepper', () => ({ default: () => null }));
|
||||
vi.mock('../components/dub/IdleSkeleton', () => ({ default: () => null }));
|
||||
vi.mock('../components/ExportModal', () => ({ default: () => null }));
|
||||
vi.mock('../hooks/useTimelineOnsets', () => ({ default: () => ({ onsets: [] }) }));
|
||||
vi.mock('../api/dub', () => ({ dubQc: vi.fn() }));
|
||||
// Never-resolving async deps keep the render synchronous (no post-test act noise).
|
||||
vi.mock('../api/engines', () => ({
|
||||
listTranslationEngines: vi.fn(() => new Promise(() => {})),
|
||||
installTranslationEngine: vi.fn(),
|
||||
}));
|
||||
vi.mock('../api/client', async (importOriginal) => {
|
||||
const mod = await importOriginal();
|
||||
return { ...mod, apiJson: vi.fn(() => new Promise(() => {})) };
|
||||
});
|
||||
|
||||
import DubTab from '../pages/DubTab';
|
||||
|
||||
const noop = () => {};
|
||||
function makeProps(over = {}) {
|
||||
return {
|
||||
dubVideoFile: null,
|
||||
dubLocalBlobUrl: null,
|
||||
transcribeElapsed: 0,
|
||||
translateProvider: 'google',
|
||||
setTranslateProvider: noop,
|
||||
showTranscript: false,
|
||||
setShowTranscript: noop,
|
||||
onGlossaryChange: noop,
|
||||
profiles: [],
|
||||
segmentPreviewLoading: null,
|
||||
selectedSegIds: new Set(),
|
||||
setDubVideoFile: noop,
|
||||
setDubLocalBlobUrl: noop,
|
||||
handleDubAbort: noop,
|
||||
handleDubUpload: noop,
|
||||
handleDubIngestUrl: noop,
|
||||
handleDubRetryTranscribe: noop,
|
||||
handleDubStop: noop,
|
||||
handleDubGenerate: noop,
|
||||
handleDubImportSrt: noop,
|
||||
handleDubDownload: noop,
|
||||
handleDubAudioDownload: noop,
|
||||
handleAudioExport: noop,
|
||||
handleSegmentPreview: noop,
|
||||
onDirectSegment: noop,
|
||||
handleTranslateAll: noop,
|
||||
handleCleanupSegments: noop,
|
||||
incrementalPlan: null,
|
||||
triggerDownload: noop,
|
||||
fileToMediaUrl: noop,
|
||||
editSegments: noop,
|
||||
saveProject: noop,
|
||||
resetDub: noop,
|
||||
segmentEditField: noop,
|
||||
segmentDelete: noop,
|
||||
segmentRestoreOriginal: noop,
|
||||
segmentSplit: noop,
|
||||
segmentMerge: noop,
|
||||
segmentMoveResize: noop,
|
||||
timelineSelSegId: null,
|
||||
setTimelineSelSegId: noop,
|
||||
toggleSegSelect: noop,
|
||||
selectAllSegs: noop,
|
||||
clearSegSelection: noop,
|
||||
bulkApplyToSelected: noop,
|
||||
bulkDeleteSelected: noop,
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
const baseState = useAppStore.getState();
|
||||
const PICKS = [
|
||||
{ lang: 'Bengali', code: 'bn' },
|
||||
{ lang: 'Spanish', code: 'es' },
|
||||
];
|
||||
|
||||
/** Render DubTab in multi-lang mode and return { onGenerateClick, calls, mocks }. */
|
||||
function setup({ translateOk = () => true, langCode = 'en', segments } = {}) {
|
||||
const calls = [];
|
||||
const handleTranslateAll = vi.fn(async (code) => {
|
||||
calls.push(`translate:${code}`);
|
||||
return translateOk(code);
|
||||
});
|
||||
const handleDubGenerate = vi.fn(async (opts) => {
|
||||
calls.push(`generate:${opts?.langOverride?.language_code ?? 'default'}`);
|
||||
});
|
||||
useAppStore.setState({
|
||||
dubJobId: 'job1',
|
||||
dubStep: 'editing',
|
||||
dubLangCode: langCode,
|
||||
dubLang: 'English',
|
||||
multiLangMode: true,
|
||||
multiLangs: PICKS,
|
||||
dubSegments: segments ?? [{ id: '1', text: 'hello', text_original: 'hello' }],
|
||||
});
|
||||
render(<DubTab {...makeProps({ handleTranslateAll, handleDubGenerate })} />);
|
||||
return {
|
||||
onGenerateClick: captured.header.at(-1).onGenerateClick,
|
||||
calls,
|
||||
handleTranslateAll,
|
||||
handleDubGenerate,
|
||||
};
|
||||
}
|
||||
|
||||
describe('DubTab — multi-language generate translates each language first (P1.1)', () => {
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
captured.header.length = 0;
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("picks ['bn','es']: each language's translate runs BEFORE its generate, in order", async () => {
|
||||
const { onGenerateClick, calls, handleDubGenerate } = setup();
|
||||
await act(async () => {
|
||||
await onGenerateClick();
|
||||
});
|
||||
// Pre-fix this was ['generate:bn', 'generate:es'] — translate never ran.
|
||||
expect(calls).toEqual(['translate:bn', 'generate:bn', 'translate:es', 'generate:es']);
|
||||
// langOverride keeps the existing handleDubGenerate call shape.
|
||||
expect(handleDubGenerate).toHaveBeenNthCalledWith(1, {
|
||||
langOverride: { language: 'Bengali', language_code: 'bn' },
|
||||
});
|
||||
expect(handleDubGenerate).toHaveBeenNthCalledWith(2, {
|
||||
langOverride: { language: 'Spanish', language_code: 'es' },
|
||||
});
|
||||
});
|
||||
|
||||
it('a failed translate skips ONLY that language’s generate, continues, and reports it', async () => {
|
||||
const errorSpy = vi.spyOn(toast, 'error');
|
||||
const { onGenerateClick, calls } = setup({ translateOk: (code) => code !== 'bn' });
|
||||
await act(async () => {
|
||||
await onGenerateClick();
|
||||
});
|
||||
expect(calls).toEqual(['translate:bn', 'translate:es', 'generate:es']);
|
||||
expect(errorSpy).toHaveBeenCalledTimes(1);
|
||||
expect(errorSpy.mock.calls[0][0]).toContain('Bengali');
|
||||
});
|
||||
|
||||
it('skips the redundant translate only when the FIRST pick already matches freshly-translated editor text', async () => {
|
||||
const { onGenerateClick, calls } = setup({
|
||||
langCode: 'bn',
|
||||
// text differs from text_original on every segment = a translation into
|
||||
// dubLangCode ('bn') is already applied — pick 1 can go straight to generate.
|
||||
segments: [{ id: '1', text: 'ওহে', text_original: 'hello' }],
|
||||
});
|
||||
await act(async () => {
|
||||
await onGenerateClick();
|
||||
});
|
||||
expect(calls).toEqual(['generate:bn', 'translate:es', 'generate:es']);
|
||||
});
|
||||
|
||||
it('untranslated editor text is ALWAYS translated, even when the first pick matches dubLangCode', async () => {
|
||||
const { onGenerateClick, calls } = setup({
|
||||
langCode: 'bn',
|
||||
segments: [{ id: '1', text: 'hello', text_original: 'hello' }],
|
||||
});
|
||||
await act(async () => {
|
||||
await onGenerateClick();
|
||||
});
|
||||
expect(calls).toEqual(['translate:bn', 'generate:bn', 'translate:es', 'generate:es']);
|
||||
});
|
||||
|
||||
it('single-language mode is untouched: generate only, no translate, no override', async () => {
|
||||
const { onGenerateClick, calls, handleDubGenerate, handleTranslateAll } = setup();
|
||||
act(() => {
|
||||
useAppStore.setState({ multiLangMode: false });
|
||||
});
|
||||
void onGenerateClick; // stale capture — re-read after the mode flip
|
||||
const fresh = captured.header.at(-1).onGenerateClick;
|
||||
await act(async () => {
|
||||
await fresh();
|
||||
});
|
||||
expect(handleTranslateAll).not.toHaveBeenCalled();
|
||||
expect(handleDubGenerate).toHaveBeenCalledWith();
|
||||
expect(calls).toEqual(['generate:default']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,117 @@
|
||||
import { describe, it, expect, beforeEach } from 'vitest';
|
||||
import { useAppStore } from '../store';
|
||||
import { restoreProjectExtras } from '../utils/projectState';
|
||||
import appSrc from '../App.jsx?raw';
|
||||
|
||||
// P1.4 — multi-language picks live in the dub store slice (not DubTab-local
|
||||
// state) and ride the project save/load payload, so "Generate 3 dubs" setups
|
||||
// survive tab switches and project reopens. Legacy payloads (saved before
|
||||
// these fields existed) must default cleanly: multi-lang off/empty, and the
|
||||
// in-session exportTracks left untouched.
|
||||
|
||||
const baseState = useAppStore.getState();
|
||||
|
||||
describe('dub slice — multiLangMode / multiLangs', () => {
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
});
|
||||
|
||||
it('defaults to off/empty', () => {
|
||||
expect(useAppStore.getState().multiLangMode).toBe(false);
|
||||
expect(useAppStore.getState().multiLangs).toEqual([]);
|
||||
});
|
||||
|
||||
it('setters accept values and functional updaters (slice pattern)', () => {
|
||||
const s = useAppStore.getState();
|
||||
s.setMultiLangMode(true);
|
||||
s.setMultiLangs([{ lang: 'Bengali', code: 'bn' }]);
|
||||
expect(useAppStore.getState().multiLangMode).toBe(true);
|
||||
expect(useAppStore.getState().multiLangs).toEqual([{ lang: 'Bengali', code: 'bn' }]);
|
||||
s.setMultiLangs((prev) => [...prev, { lang: 'Spanish', code: 'es' }]);
|
||||
expect(useAppStore.getState().multiLangs).toHaveLength(2);
|
||||
s.setMultiLangMode((prev) => !prev);
|
||||
expect(useAppStore.getState().multiLangMode).toBe(false);
|
||||
});
|
||||
|
||||
it('resetDubState clears the picks with the rest of the pipeline state', () => {
|
||||
const s = useAppStore.getState();
|
||||
s.setMultiLangMode(true);
|
||||
s.setMultiLangs([{ lang: 'Bengali', code: 'bn' }]);
|
||||
s.resetDubState();
|
||||
expect(useAppStore.getState().multiLangMode).toBe(false);
|
||||
expect(useAppStore.getState().multiLangs).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('project payload — save/load round-trip (restoreProjectExtras)', () => {
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
});
|
||||
|
||||
it('round-trips multiLangMode, multiLangs and exportTracks through the payload', () => {
|
||||
const s = useAppStore.getState();
|
||||
s.setMultiLangMode(true);
|
||||
s.setMultiLangs([
|
||||
{ lang: 'Bengali', code: 'bn' },
|
||||
{ lang: 'Spanish', code: 'es' },
|
||||
]);
|
||||
s.setExportTracks({ original: true, bn: true, es: false });
|
||||
// Mirror App.jsx's saveProject: the store values land in state as-is.
|
||||
const cur = useAppStore.getState();
|
||||
const payload = {
|
||||
multiLangMode: cur.multiLangMode,
|
||||
multiLangs: cur.multiLangs,
|
||||
exportTracks: cur.exportTracks,
|
||||
};
|
||||
const restored = restoreProjectExtras(JSON.parse(JSON.stringify(payload)));
|
||||
expect(restored.multiLangMode).toBe(true);
|
||||
expect(restored.multiLangs).toEqual([
|
||||
{ lang: 'Bengali', code: 'bn' },
|
||||
{ lang: 'Spanish', code: 'es' },
|
||||
]);
|
||||
expect(restored.exportTracks).toEqual({ original: true, bn: true, es: false });
|
||||
});
|
||||
|
||||
it('legacy payload (fields absent) defaults to off/empty and leaves exportTracks alone', () => {
|
||||
const restored = restoreProjectExtras({ dubJobId: 'old', dubSegments: [] });
|
||||
expect(restored.multiLangMode).toBe(false);
|
||||
expect(restored.multiLangs).toEqual([]);
|
||||
expect(restored.exportTracks).toBeNull(); // null = don't touch the current value
|
||||
});
|
||||
|
||||
it('is shape-safe: malformed picks are dropped, junk exportTracks is ignored', () => {
|
||||
const restored = restoreProjectExtras({
|
||||
multiLangMode: 'yes', // not boolean true → off
|
||||
multiLangs: [{ lang: 'Bengali', code: 'bn' }, { code: 'es' }, 'fr', null],
|
||||
exportTracks: ['original'],
|
||||
});
|
||||
expect(restored.multiLangMode).toBe(false);
|
||||
expect(restored.multiLangs).toEqual([{ lang: 'Bengali', code: 'bn' }]);
|
||||
expect(restored.exportTracks).toBeNull();
|
||||
expect(restoreProjectExtras(undefined)).toEqual({
|
||||
multiLangMode: false,
|
||||
multiLangs: [],
|
||||
exportTracks: null,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('App.jsx wiring guard (raw source — keeps the util honest)', () => {
|
||||
it('saveProject persists the three fields in statePayload.state', () => {
|
||||
const start = appSrc.indexOf('const statePayload');
|
||||
expect(start).toBeGreaterThan(-1);
|
||||
const block = appSrc.slice(start, appSrc.indexOf('apiSaveProject', start));
|
||||
for (const key of ['multiLangMode', 'multiLangs', 'exportTracks']) {
|
||||
expect(block, `statePayload.state must include ${key}`).toContain(key);
|
||||
}
|
||||
});
|
||||
|
||||
it('loadProject restores through restoreProjectExtras', () => {
|
||||
const start = appSrc.indexOf('const loadProject');
|
||||
expect(start).toBeGreaterThan(-1);
|
||||
const block = appSrc.slice(start, start + 3000);
|
||||
expect(block).toContain('restoreProjectExtras');
|
||||
expect(block).toContain('setMultiLangMode');
|
||||
expect(block).toContain('setMultiLangs');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,235 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { renderHook, act } from '@testing-library/react';
|
||||
import { useAppStore } from '../store';
|
||||
|
||||
// P1.2 / P1.3 — per-language translation storage + per-track fingerprints.
|
||||
//
|
||||
// `dubSegments[].text` is single-slot (the currently-shown language); before
|
||||
// this fix, switching the dub target language left the previous language's
|
||||
// text on screen and the next Translate All DESTROYED it. Now every
|
||||
// translation is kept in `s.translations[langCode]`, `switchDubLangCode`
|
||||
// swaps `text` through that map non-destructively, manual edits update the
|
||||
// current language's entry, and the incremental-plan fingerprints are stored
|
||||
// per language so "Regen N changed" judges the ACTIVE track.
|
||||
|
||||
const dubApi = vi.hoisted(() => ({
|
||||
dubUpload: vi.fn(),
|
||||
dubIngestUrl: vi.fn(),
|
||||
dubAbort: vi.fn(),
|
||||
dubCleanupSegments: vi.fn(),
|
||||
dubTranslate: vi.fn(),
|
||||
dubGenerate: vi.fn(),
|
||||
tasksStreamUrl: vi.fn(() => ''),
|
||||
tasksCancel: vi.fn(),
|
||||
transcribeStreamUrl: vi.fn(() => ''),
|
||||
dubImportSrt: vi.fn(),
|
||||
}));
|
||||
vi.mock('../api/dub', () => dubApi);
|
||||
const clientApi = vi.hoisted(() => ({
|
||||
apiPost: vi.fn(),
|
||||
apiFetch: vi.fn(),
|
||||
apiJson: vi.fn(),
|
||||
API: '',
|
||||
}));
|
||||
vi.mock('../api/client', () => clientApi);
|
||||
|
||||
import useDubWorkflow from '../hooks/useDubWorkflow';
|
||||
import useSegmentEditing from '../hooks/useSegmentEditing';
|
||||
|
||||
const baseState = useAppStore.getState();
|
||||
|
||||
function renderWorkflow() {
|
||||
return renderHook(() =>
|
||||
useDubWorkflow({
|
||||
loadProjects: vi.fn(),
|
||||
loadProfiles: vi.fn(),
|
||||
loadDubHistory: vi.fn(),
|
||||
setLastGenFingerprints: vi.fn(),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
const seg = (over = {}) => ({
|
||||
id: '1',
|
||||
text: 'hello there',
|
||||
text_original: 'hello there',
|
||||
start: 0,
|
||||
end: 2,
|
||||
...over,
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
dubApi.dubTranslate.mockReset();
|
||||
clientApi.apiPost.mockReset();
|
||||
useAppStore.setState({
|
||||
dubJobId: 'job1',
|
||||
dubStep: 'editing',
|
||||
dubLangCode: 'bn',
|
||||
dubSegments: [seg()],
|
||||
});
|
||||
});
|
||||
|
||||
const translateTo = async (result, lang, text) => {
|
||||
dubApi.dubTranslate.mockResolvedValueOnce({
|
||||
translated: [{ id: '1', text }],
|
||||
target_lang: lang,
|
||||
});
|
||||
await act(async () => {
|
||||
await result.current.handleTranslateAll(lang);
|
||||
});
|
||||
};
|
||||
|
||||
describe('per-language translations (P1.2)', () => {
|
||||
it('translate bn then es retains BOTH languages in s.translations (pre-fix: bn lost)', async () => {
|
||||
const { result } = renderWorkflow();
|
||||
await translateTo(result, 'bn', 'ওহে');
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
await translateTo(result, 'es', 'hola');
|
||||
|
||||
const s = useAppStore.getState().dubSegments[0];
|
||||
expect(s.text).toBe('hola'); // text stays the shown language (legacy slot)
|
||||
expect(s.translations).toMatchObject({ bn: 'ওহে', es: 'hola' });
|
||||
});
|
||||
|
||||
it('switching the target language swaps text non-destructively, both directions', async () => {
|
||||
const { result } = renderWorkflow();
|
||||
await translateTo(result, 'bn', 'ওহে');
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
await translateTo(result, 'es', 'hola');
|
||||
|
||||
act(() => useAppStore.getState().switchDubLangCode('bn'));
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('hola');
|
||||
});
|
||||
|
||||
it('switching to a never-translated language leaves text unchanged (legacy behaviour)', () => {
|
||||
useAppStore.setState({
|
||||
dubSegments: [seg({ text: 'ওহে', translations: { bn: 'ওহে' } })],
|
||||
});
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
// Non-destructive: no es entry → keep showing what was there.
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
|
||||
expect(useAppStore.getState().dubLangCode).toBe('es');
|
||||
});
|
||||
|
||||
it('legacy segments (no translations field) survive a switch round-trip', () => {
|
||||
// A pre-upgrade project where bn text was already translated in place.
|
||||
useAppStore.setState({
|
||||
dubSegments: [seg({ text: 'ওহে' })], // text !== text_original, no map
|
||||
});
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
act(() => useAppStore.getState().switchDubLangCode('bn'));
|
||||
// The switch snapshotted bn's text into the map instead of losing it.
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
|
||||
expect(useAppStore.getState().dubSegments[0].translations.bn).toBe('ওহে');
|
||||
});
|
||||
|
||||
it('never stamps untranslated (source) text as a translation on switch', () => {
|
||||
// text === text_original → not a translation, must not be snapshotted.
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
expect(useAppStore.getState().dubSegments[0].translations.bn).toBeUndefined();
|
||||
});
|
||||
|
||||
it('manual segment edit updates the CURRENT language entry only', () => {
|
||||
useAppStore.setState({
|
||||
dubLangCode: 'es',
|
||||
dubSegments: [seg({ text: 'hola', translations: { bn: 'ওহে', es: 'hola' } })],
|
||||
});
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.segmentEditField('1', 'text', 'hola editada'));
|
||||
const s = useAppStore.getState().dubSegments[0];
|
||||
expect(s.text).toBe('hola editada');
|
||||
expect(s.translations).toEqual({ bn: 'ওহে', es: 'hola editada' });
|
||||
});
|
||||
|
||||
it('restore-original records the decision under the current language', () => {
|
||||
useAppStore.setState({
|
||||
dubLangCode: 'es',
|
||||
dubSegments: [seg({ text: 'hola', translations: { es: 'hola' } })],
|
||||
});
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.segmentRestoreOriginal('1'));
|
||||
const s = useAppStore.getState().dubSegments[0];
|
||||
expect(s.text).toBe('hello there');
|
||||
expect(s.translations.es).toBe('hello there');
|
||||
});
|
||||
|
||||
it('merge joins per-language texts only where both rows carry the language', () => {
|
||||
useAppStore.setState({
|
||||
dubLangCode: 'es',
|
||||
dubSegments: [
|
||||
seg({ id: 'a', end: 1, translations: { es: 'uno', bn: 'এক' } }),
|
||||
seg({ id: 'b', start: 1, translations: { es: 'dos' } }),
|
||||
],
|
||||
});
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.segmentMerge('a'));
|
||||
const merged = useAppStore.getState().dubSegments[0];
|
||||
expect(merged.translations).toEqual({ es: 'uno dos' }); // bn half-known → dropped
|
||||
});
|
||||
|
||||
it('split drops the per-language map (new ids need fresh translations)', () => {
|
||||
useAppStore.setState({
|
||||
dubLangCode: 'es',
|
||||
dubSegments: [seg({ text: 'hola mundo', translations: { es: 'hola mundo' } })],
|
||||
});
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.segmentSplit('1', 5));
|
||||
const segs = useAppStore.getState().dubSegments;
|
||||
expect(segs).toHaveLength(2);
|
||||
expect(segs[0].translations).toBeUndefined();
|
||||
expect(segs[1].translations).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('per-track fingerprints (P1.3)', () => {
|
||||
it('lastGenFingerprints follows the ACTIVE language', () => {
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.setLastGenFingerprints({ 1: 'hash-bn' }, 'bn'));
|
||||
act(() => result.current.setLastGenFingerprints({ 1: 'hash-es' }, 'es'));
|
||||
expect(useAppStore.getState().dubLangCode).toBe('bn');
|
||||
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hash-bn' });
|
||||
act(() => useAppStore.getState().switchDubLangCode('es'));
|
||||
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hash-es' });
|
||||
});
|
||||
|
||||
it('recomputeIncremental sends the active lang + that language’s hashes', async () => {
|
||||
clientApi.apiPost.mockResolvedValue({ stale: [], fresh: ['1'], fingerprints: {} });
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.setLastGenFingerprints({ 1: 'hash-bn' }, 'bn'));
|
||||
await act(async () => {
|
||||
await result.current.recomputeIncremental();
|
||||
});
|
||||
expect(clientApi.apiPost).toHaveBeenCalledWith(
|
||||
'/tools/incremental',
|
||||
expect.objectContaining({ lang: 'bn', stored_hashes: { 1: 'hash-bn' } }),
|
||||
);
|
||||
expect(result.current.incrementalPlan).toEqual({ stale: [], fresh: ['1'] });
|
||||
});
|
||||
|
||||
it('a language with no stored hashes yields no plan (unknowable ≠ stale)', async () => {
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.setLastGenFingerprints({ 1: 'hash-es' }, 'es'));
|
||||
// Active language is bn → no hashes → plan cleared, no API call.
|
||||
await act(async () => {
|
||||
await result.current.recomputeIncremental();
|
||||
});
|
||||
expect(clientApi.apiPost).not.toHaveBeenCalled();
|
||||
expect(result.current.incrementalPlan).toBeNull();
|
||||
});
|
||||
|
||||
it('setFingerprintsByLang restores every track at once (project/history load)', () => {
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.setFingerprintsByLang({ bn: { 1: 'hb' }, es: { 1: 'he' } }));
|
||||
expect(result.current.lastGenFingerprints).toEqual({ 1: 'hb' }); // active = bn
|
||||
expect(result.current.fingerprintsByLang.es).toEqual({ 1: 'he' });
|
||||
});
|
||||
|
||||
it('setLastGenFingerprints without a lang defaults to the store selection', () => {
|
||||
const { result } = renderHook(() => useSegmentEditing());
|
||||
act(() => result.current.setLastGenFingerprints({ 1: 'h' }));
|
||||
expect(result.current.fingerprintsByLang).toEqual({ bn: { 1: 'h' } });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,152 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { renderHook, act } from '@testing-library/react';
|
||||
import { useAppStore } from '../store';
|
||||
|
||||
// P1.1 — `handleTranslateAll(langOverride?)`:
|
||||
// - no-arg keeps the existing Translate All behavior (target = store's
|
||||
// dubLangCode),
|
||||
// - a string override translates INTO that language (the multi-language
|
||||
// generate loop passes each pick's code),
|
||||
// - a non-string first arg (the `onClick={handleTranslateAll}` click event)
|
||||
// must be ignored, not treated as a language,
|
||||
// - resolves true only when a translation actually landed — the batch loop
|
||||
// keys "skip this language's generate" off that.
|
||||
|
||||
const dubApi = vi.hoisted(() => ({
|
||||
dubUpload: vi.fn(),
|
||||
dubIngestUrl: vi.fn(),
|
||||
dubAbort: vi.fn(),
|
||||
dubCleanupSegments: vi.fn(),
|
||||
dubTranslate: vi.fn(),
|
||||
dubGenerate: vi.fn(),
|
||||
tasksStreamUrl: vi.fn(() => ''),
|
||||
tasksCancel: vi.fn(),
|
||||
transcribeStreamUrl: vi.fn(() => ''),
|
||||
dubImportSrt: vi.fn(),
|
||||
}));
|
||||
vi.mock('../api/dub', () => dubApi);
|
||||
vi.mock('../api/client', () => ({
|
||||
apiPost: vi.fn(),
|
||||
apiFetch: vi.fn(),
|
||||
apiJson: vi.fn(),
|
||||
API: '',
|
||||
}));
|
||||
|
||||
import useDubWorkflow from '../hooks/useDubWorkflow';
|
||||
|
||||
const baseState = useAppStore.getState();
|
||||
|
||||
function renderWorkflow() {
|
||||
return renderHook(() =>
|
||||
useDubWorkflow({
|
||||
loadProjects: vi.fn(),
|
||||
loadProfiles: vi.fn(),
|
||||
loadDubHistory: vi.fn(),
|
||||
setLastGenFingerprints: vi.fn(),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
describe('handleTranslateAll(langOverride) — multi-language target override', () => {
|
||||
beforeEach(() => {
|
||||
useAppStore.setState(baseState, true);
|
||||
dubApi.dubTranslate.mockReset();
|
||||
useAppStore.setState({
|
||||
dubJobId: 'job1',
|
||||
dubStep: 'editing',
|
||||
dubLangCode: 'es',
|
||||
dubSegments: [
|
||||
{ id: '1', text: 'hello there', text_original: 'hello there', start: 0, end: 2 },
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it('no-arg behavior unchanged: translates into the store dubLangCode and applies the text', async () => {
|
||||
dubApi.dubTranslate.mockResolvedValue({
|
||||
translated: [{ id: '1', text: 'hola' }],
|
||||
target_lang: 'es',
|
||||
});
|
||||
const { result } = renderWorkflow();
|
||||
let ok;
|
||||
await act(async () => {
|
||||
ok = await result.current.handleTranslateAll();
|
||||
});
|
||||
expect(dubApi.dubTranslate).toHaveBeenCalledTimes(1);
|
||||
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('es');
|
||||
expect(ok).toBe(true);
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('hola');
|
||||
});
|
||||
|
||||
it('string override translates INTO the override language, not the store selection', async () => {
|
||||
dubApi.dubTranslate.mockResolvedValue({
|
||||
translated: [{ id: '1', text: 'ওহে' }],
|
||||
target_lang: 'bn',
|
||||
});
|
||||
const { result } = renderWorkflow();
|
||||
let ok;
|
||||
await act(async () => {
|
||||
ok = await result.current.handleTranslateAll('bn');
|
||||
});
|
||||
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('bn');
|
||||
expect(ok).toBe(true);
|
||||
expect(useAppStore.getState().dubSegments[0].text).toBe('ওহে');
|
||||
});
|
||||
|
||||
it('a click event as first arg (onClick={handleTranslateAll}) falls back to dubLangCode', async () => {
|
||||
dubApi.dubTranslate.mockResolvedValue({
|
||||
translated: [{ id: '1', text: 'hola' }],
|
||||
target_lang: 'es',
|
||||
});
|
||||
const { result } = renderWorkflow();
|
||||
await act(async () => {
|
||||
await result.current.handleTranslateAll({ preventDefault() {}, type: 'click' });
|
||||
});
|
||||
expect(dubApi.dubTranslate.mock.calls[0][0].target_lang).toBe('es');
|
||||
});
|
||||
|
||||
it('request failure resolves false and surfaces the existing error banner', async () => {
|
||||
dubApi.dubTranslate.mockRejectedValue(new Error('engine down'));
|
||||
const { result } = renderWorkflow();
|
||||
let ok;
|
||||
await act(async () => {
|
||||
ok = await result.current.handleTranslateAll('bn');
|
||||
});
|
||||
expect(ok).toBe(false);
|
||||
expect(useAppStore.getState().dubError).toMatch(/engine down/);
|
||||
expect(useAppStore.getState().isTranslating).toBe(false);
|
||||
});
|
||||
|
||||
it('an all-errors result resolves false (nothing translated → no wrong-language dub)', async () => {
|
||||
dubApi.dubTranslate.mockResolvedValue({
|
||||
translated: [{ id: '1', text: '', error: 'boom' }],
|
||||
target_lang: 'bn',
|
||||
});
|
||||
const { result } = renderWorkflow();
|
||||
let ok;
|
||||
await act(async () => {
|
||||
ok = await result.current.handleTranslateAll('bn');
|
||||
});
|
||||
expect(ok).toBe(false);
|
||||
});
|
||||
|
||||
it('reads segments from the store at call time (stale click-time closure is the loop bug class)', async () => {
|
||||
dubApi.dubTranslate.mockResolvedValue({
|
||||
translated: [{ id: '2', text: 'nuevo' }],
|
||||
target_lang: 'es',
|
||||
});
|
||||
const { result } = renderWorkflow();
|
||||
const stale = result.current.handleTranslateAll; // captured before the segments change
|
||||
act(() => {
|
||||
useAppStore
|
||||
.getState()
|
||||
.setDubSegments([{ id: '2', text: 'fresh', text_original: 'fresh', start: 0, end: 1 }]);
|
||||
});
|
||||
await act(async () => {
|
||||
await stale();
|
||||
});
|
||||
const sent = dubApi.dubTranslate.mock.calls[0][0].segments;
|
||||
expect(sent).toHaveLength(1);
|
||||
expect(sent[0].id).toBe('2');
|
||||
expect(sent[0].text).toBe('fresh');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,26 @@
|
||||
// Regression guard: VoiceGallery/CommunityZone/ImportsZone catch blocks used
|
||||
// to discard the real error and show a hardcoded, often-wrong generic guess
|
||||
// (e.g. "the engine may be loading" on ANY failure, including ones that had
|
||||
// nothing to do with loading). Fixed to interpolate the real `e.message`
|
||||
// (already a clean, user-facing string from api/client.js's ApiError),
|
||||
// matching the `{{message}}` convention used everywhere else in this file.
|
||||
// This test only pins the i18n keys, not the call sites, deliberately: it's
|
||||
// a cheap net against reverting to a hardcoded string, not a full behavior test.
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import en from '../i18n/locales/en.json';
|
||||
|
||||
describe('gallery error messages interpolate the real error', () => {
|
||||
const keys = [
|
||||
'use_failed',
|
||||
'preview_failed',
|
||||
'search_failed',
|
||||
'upload_failed',
|
||||
'save_failed',
|
||||
'delete_failed',
|
||||
'trim_load_failed',
|
||||
];
|
||||
|
||||
it.each(keys)('gallery.%s contains {{message}}', (key) => {
|
||||
expect(en.gallery[key]).toContain('{{message}}');
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user