Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a721c1fdcf | ||
|
|
80f10289fe | ||
|
|
a0ad314736 | ||
|
|
1808a373a1 | ||
|
|
67789fb31c | ||
|
|
da4bef8e42 | ||
|
|
0076d0067e | ||
|
|
faf34348c8 | ||
|
|
69ce697ee5 | ||
|
|
93a3cb260a | ||
|
|
33379890ad | ||
|
|
7f8a42ce51 | ||
|
|
4de3c824d0 | ||
|
|
11fbbd3f6d | ||
|
|
f546f04c8e | ||
|
|
270b3b1c4c | ||
|
|
5a7d9cc05c | ||
|
|
5871ec9d28 | ||
|
|
58b023ff16 | ||
|
|
b61d17dd61 | ||
|
|
93aab6dadc | ||
|
|
7098be1c8b | ||
|
|
8520b84b68 | ||
|
|
549fa4009f | ||
|
|
dfe2bd1bc3 | ||
|
|
324d417d27 | ||
|
|
637020b82b | ||
|
|
dab6456581 | ||
|
|
015d29cafb | ||
|
|
1a03c59f82 | ||
|
|
8281b7c798 | ||
|
|
efe57ee49d | ||
|
|
7f77f4d7bf | ||
|
|
087309259b |
@@ -149,7 +149,7 @@ jobs:
|
||||
- os: windows-2022
|
||||
label: Windows
|
||||
rust_target: x86_64-pc-windows-msvc
|
||||
- os: ubuntu-22.04
|
||||
- os: ubuntu-24.04
|
||||
label: Linux
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
@@ -213,13 +213,24 @@ jobs:
|
||||
bundles: "msi,updater"
|
||||
|
||||
# Linux: ship .AppImage only. AppImage is universal (no distro
|
||||
# package-manager dep), runs on any glibc-2.31+ host, and is the
|
||||
# package-manager dep), runs on any glibc-2.39+ host, and is the
|
||||
# Linux auto-update target. The .deb target was dropped: tauri-bundler
|
||||
# fails it with "Failed to create control scripts: No such file or
|
||||
# directory" (no custom deb config of ours is at fault) — revisit on a
|
||||
# tauri-cli bump. FUSE unavailability on GH runners is handled via
|
||||
# APPIMAGE_EXTRACT_AND_RUN=1.
|
||||
- os: ubuntu-22.04
|
||||
#
|
||||
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
|
||||
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
|
||||
# repos resolve (see the "Linux system deps" step below) — 22.04's
|
||||
# was meaningfully stale relative to what current Ubuntu/Fedora
|
||||
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
|
||||
# take priority over a healthy system WebKitGTK at runtime. Raises
|
||||
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
|
||||
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
|
||||
# something that old has come in, and the project's own install
|
||||
# docs already assume Debian 12 / Ubuntu 22.04+.
|
||||
- os: ubuntu-24.04
|
||||
arch: x86_64-unknown-linux-gnu
|
||||
label: "Linux x64"
|
||||
rust_target: x86_64-unknown-linux-gnu
|
||||
|
||||
@@ -6,6 +6,80 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
|
||||
The bundled TTS model package (`pyproject.toml`) is versioned independently.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.3.14] — 2026-07-09
|
||||
|
||||
A fast follow to v0.3.13: **every engine family now has a visible picker.** Settings → Engines showed only a TTS table, with the ASR and LLM pickers hidden behind a low-discoverability tab — so the 10 transcription engines (including the new OpenAI-compatible backend) looked unswitchable without env vars. Now all three families get their own table. Also in: the Linux AppImage's white-screen auto-workaround now checks the WebKitGTK it actually ships (not whatever your system reports), and installing to a different drive on Windows is properly documented.
|
||||
|
||||
### Added
|
||||
|
||||
- **ASR engines get the same Settings picker TTS has.** Settings → Engines now shows a visible picker table per family — TTS, ASR, and LLM — instead of a single TTS-titled table with the other families tucked behind a tab (README even promised a Settings ASR picker that didn't exist). The OpenAI-compatible backend and the 9 local ASR engines become selectable with one click, no env vars needed; an explicit `OMNIVOICE_ASR_BACKEND` still wins over the Settings pick, so pinned setups behave exactly as before. (no issue — UX gap found during #877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage's white-screen auto-workaround now checks the right WebKitGTK.** The launcher decided whether to apply the compositing workaround by asking the *system's* `pkg-config` — but the version that actually runs is the *bundled* one, which the AppImage prioritizes. On any machine where the two diverge (e.g. building from source with newer dev packages installed), the detection read the wrong number and could skip a workaround the running library needed. The build now stamps the bundled version into the AppImage at package time, and the launcher reads that stamp — correct by construction. The launcher's shell tests also now run in CI, which they previously never did. (#961 follow-up)
|
||||
|
||||
### Docs
|
||||
|
||||
- **Windows: installing to a different drive is documented** — the wizard's directory picker works for any local drive; mapped network drives are a Windows Installer limitation (not installable-to by design); and the big data (models/voices) moves independently via Settings → Storage or Portable mode. (#938)
|
||||
|
||||
## [0.3.13] — 2026-07-09
|
||||
|
||||
The community-fixes release. Two contributors didn't just report bugs — they diagnosed them to the exact line and submitted the fixes that shipped: **voice cloning on mlx-audio's CSM model works for the first time**, and **macOS live recording finally gets its microphone permission prompt** (both @MahdiHedhli). A third reporter's A/B analysis fixed **cross-language dubs speaking the wrong language**. On top of that: a backend shutdown race that produced confusing crash-on-quit reports is fixed, the Linux AppImage stops shipping a stale WebKitGTK that white-screened current distros, and a new OpenAI-compatible transcription backend opens a path to Qwen3-ASR today. Thank you to everyone who filed, diagnosed, and contributed — this release is mostly yours.
|
||||
|
||||
### Added
|
||||
|
||||
- **A path to Qwen3-ASR today: generic OpenAI-compatible transcription.** The direct integration is still blocked on `transformers>=5.13` stabilizing upstream, but a community member proposed splitting the work — add a backend that talks to any OpenAI-compatible transcription server right now. Point OmniVoice at a self-hosted Qwen3-ASR/FunASR/SenseVoice server, or OpenAI's own API, configured in Settings → Models. No install; audio does leave your machine to whichever server you configure, unlike every other ASR engine. (#877)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The Linux AppImage no longer white-screens on current distros with a healthy system WebKitGTK.** The release build ran on an older CI base image, and the resulting AppImage bundles whatever `libwebkit2gtk` that image's apt repos resolve — which the AppImage's own `LD_LIBRARY_PATH` then prioritizes over your system's newer, healthy copy at runtime. A from-source build (which links straight against your system library) worked fine on the exact same machine where the shipped AppImage didn't — that split was the tell. Bumped the release build to a current Ubuntu LTS. Raises the AppImage's minimum host to glibc 2.39 (Ubuntu 24.04+); no reports from anyone on an older distro. (#961)
|
||||
- **Backend shutdown no longer races a still-loading model, surfacing a confusing crash on restart.** Quitting the app while a model was still loading in the background let shutdown report itself "done" while a background thread was still mid-import; tearing the process down under that thread produced a misleading error (a generic transformers import-failure message, unrelated to the real cause) that looked like a real crash rather than a timing issue. All background tasks are now properly cancelled and awaited before shutdown proceeds. (#1000, likely the same class behind #941 and #979)
|
||||
- **Cross-language dub no longer speaks the source-language reference line verbatim.** Auto-generated speaker clones pair an audio slice with the ASR segment's own text field, assuming the two agree — but ASR segment text and its timestamps routinely drift (a trailing word audible in the clip but missing from the text, or vice versa). A mismatched (reference audio, reference text) pair breaks zero-shot TTS prompt priming badly enough that the clone can emit the reference text itself instead of the target-language line it was asked to speak. Each reference clip is now re-transcribed after it's written, so the pair matches by construction — reported with an exceptionally clear root-cause diagnosis and a working A/B repro. (#1004)
|
||||
- **Voice Gallery errors now say what actually went wrong.** "Use voice", "Preview", search, upload, save, delete, and trim in the Gallery all showed the same hardcoded guess ("the engine may be loading") on ANY failure — a 500, a validation error, a genuinely unrelated bug — discarding the real, already-clean backend error message in the process. Every one of those now shows the actual error.
|
||||
- **Voice cloning on mlx-audio's CSM model no longer crashes with an opaque "list index out of range".** `MLXAudioBackend.generate()` read `voice`/`ref_audio`/`language`/`speed` from its kwargs but silently dropped `ref_text` — CSM only builds its cloning context when both `ref_audio` and `ref_text` are present, so cloning on this engine could never have worked as shipped. Reported with the exact root cause and a working fix. (#1012, #1013)
|
||||
- **A dub segment's free-text style tags no longer 400 the segment preview.** A validator-safe instruct builder already keeps Studio and Clone generation from round-tripping a 400 on unsupported free-text (a preset's raw attrs, an old profile's stray descriptive phrase) — but the Dub tab's segment preview, and saving a profile from a clone or from history, built their instruct strings directly and skipped it. Same guard now applies everywhere an instruct string is sent. (#1010)
|
||||
- **The dub editor's play button no longer sticks permanently disabled after an audio-decode hiccup.** When the initial WaveSurfer decode fails, the timeline falls back to loading pre-computed peaks — the waveform draws fine, but the button's enabled state only relied on the `ready` event firing again for that recovery load, which it didn't reliably do. Each fallback path now confirms readiness explicitly once it settles.
|
||||
- **macOS: live recording finally works — the microphone permission prompt now actually appears.** The app never showed up in System Settings → Privacy & Security → Microphone because macOS never saw a legitimate request: Tauri enables Hardened Runtime by default, which blocks microphone hardware access unless the matching entitlement is in the signed bundle — and it wasn't. Diagnosed to the exact mechanism and fixed by a community contributor (@MahdiHedhli), who also corrected our initial mis-read of this as an upstream WebKit limitation. (#1013, #1016)
|
||||
- **Quitting during a slow model load waits longer before giving up.** A post-merge code review of the shutdown-race fix flagged that its 3-second wait could still be outrun by a cold model import on a slow disk, reproducing the original confusing-crash-on-quit in rare cases. The wait is now 20 seconds — imperceptible on a normal quit (tasks finish or cancel in milliseconds), only felt in the exact case it protects. (#1020)
|
||||
|
||||
### Changed
|
||||
|
||||
- **Removed the donate heart from the nav rail.** Support OmniVoice is still one click away from Settings and the Contact page.
|
||||
|
||||
### CI
|
||||
|
||||
- **The "flaky trio" is root-caused and neutralized.** Three tests failed intermittently on CI — never locally — across unrelated PRs, costing a re-run each time. Cause: a leaked half-precision torch default from some earlier test in CI's ordering (the giveaway: a failing assertion's observed value was exactly float16(0.1)). An autouse test-suite guard now resets the leak between tests and names the offending test in CI output when it fires. (#1021)
|
||||
|
||||
## [0.3.12] — 2026-07-08
|
||||
|
||||
A community-issue sweep — nineteen open reports triaged in one pass, most fixed same-day. The through-line: **your active engine selection is now honored everywhere** (dubbing, batch, and — new in this release — MLX-Audio's own curated models are finally selectable instead of always silently defaulting to Kokoro), **first-run stops dead-ending users on restricted networks or behind corporate TLS proxies**, and a run of sharp community diagnoses (a one-line ROCm index fix, a Windows-only focus-stealing bug, a genuine crash regression) got fixed largely because reporters did the hard diagnostic work themselves. Thank you.
|
||||
|
||||
### Added
|
||||
|
||||
- **MLX-Audio's other 6 curated models are finally selectable.** The engine multiplexes Kokoro, CSM, Qwen3-TTS, Dia, Chatterbox, MeloTTS, and OuteTTS, but there was no way anywhere in the UI or API to pick which one loads — downloading a model via Settings → Models did nothing, since the backend always defaulted to Kokoro regardless. Settings → Engines now shows a model picker on the mlx-audio row; switching takes effect immediately, no restart needed. (#981)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **First-run no longer dead-ends behind restricted networks (e.g. China).** The system check probed hardcoded huggingface.co, and any failure locked the Continue button — users behind the Great Firewall were stuck on the very first screen, even when they had already configured a working mirror. The check now probes the Hugging Face endpoint actually in effect, an unreachable endpoint is a warning instead of a blocker (models already on disk keep working offline), and when huggingface.co is blocked but the hf-mirror.com community mirror answers, the wizard says so and offers a one-click mirror switch right on the check screen — no restart needed. (#984)
|
||||
- **Installs behind a corporate or antivirus TLS-inspecting proxy no longer fail with a raw SSL error.** `SSLV3_ALERT_HANDSHAKE_FAILURE` happens when a proxy re-signs HTTPS traffic with a root CA your OS trusts but Python's bundled certificate list doesn't — a different failure mode from the network-blocking case above. OmniVoice now trusts your OS's certificate store directly, which should resolve the handshake outright rather than just explain it better. (#976)
|
||||
- **The loaded-models panel now says when a resident model is not your active engine.** Switching TTS engines keeps the previous model in VRAM (so switching back is instant) — but the panel showed it with no context, so "OmniVoice TTS — 1.9 GB" after selecting VoxCPM2 looked like the selection was ignored. A field report confirmed the confusion. Resident-but-inactive models are now tagged "not active — safe to unload", and the API self-describes each entry's engine. (#985)
|
||||
- **Voices no longer ship with a hidden echo.** Every non-raw synthesis was getting a small room reverb baked in by the mastering pre-stage — on top of whatever effect preset you chose, so even "Podcast" (which promises *no reverb*) had some, and Cinematic/Warm got it twice. A field report ("a lot of echo/reverb on some of the voices") led straight to it. The mastering stage is now highpass + compressor only; reverb happens only when a preset explicitly declares it. Also documented: cloned voices reproduce the reference clip's room acoustics — dry, close-mic references clone cleanest. (#986)
|
||||
- **Your engine selection now actually applies to Dubbing and Batch TTS.** Both hardcoded OmniVoice regardless of what was picked in Settings → Engines — pick VoxCPM2, dub anyway with OmniVoice, no error. Both now resolve the active engine up front; an engine that can't clone from reference audio (KittenTTS, Sherpa-ONNX, Supertonic 3 — fixed preset voices only) fails the job immediately with a clear message naming which engines do support it, instead of silently substituting OmniVoice or mis-cloning every speaker into one voice. Batch only requires cloning when a specific voice is pinned — an unpinned batch job runs on any engine. (#987)
|
||||
- **AMD ROCm torch install no longer silently falls back to CPU.** A community member (Kaihui-AMD) diagnosed it precisely: the ROCm wheel index we pointed at tops out at PyTorch 2.5.1, but the app pins `torch==2.8.0` — the reinstall was unsatisfiable and silently kept the default CUDA build, which runs on CPU on an AMD GPU. Bumped the default index to one that actually carries the pinned version. (#972)
|
||||
- **mlx-audio no longer crashes on unsupported languages.** Selecting a language like Dutch, Spanish, or Portuguese with mlx-audio's Kokoro model crashed with a raw, unreadable internal-details dump instead of a real error — the code was guessing an ISO language code by truncating the language name, which only worked by coincidence for a few languages. Unsupported languages now fail cleanly with a message naming what's actually supported, and no engine can leak a raw crash-internals dump into an error message again. (#977)
|
||||
- **The voice-design panel no longer crashes on certain saved voice profiles.** A genuine regression: an earlier translation fix accidentally introduced a crash when a saved design profile's data was incomplete (possible from an older app version or a partial save). Fixed at every layer — the render no longer crashes, both places that restore saved data complete it first, and profiles can no longer be *saved* with incomplete data in the first place. (#983)
|
||||
- **Windows: the dictation pill no longer steals focus.** Pressing the dictation shortcut activated the pill window, which meant the auto-paste landed back in OmniVoice instead of whatever app you were dictating into, and the pill would get stuck on screen. Precisely diagnosed by a community reporter; fixed to match how this already worked on macOS. (#982)
|
||||
- **The nemo-parakeet ASR engine's install hint no longer breaks your backend.** Following the in-app "pip install nemo_toolkit[asr]" instruction silently downgraded core packages your backend needs to start — the install reported success, and the breakage only showed up on the next restart. The hint now says plainly that this isn't safe to install into the shared environment. (#974)
|
||||
- **A stuck generate now tells you the actual fix.** When a job times out from GPU/VRAM contention, the error explained why but never mentioned Flush/Unload — the one action that actually resolves it, and one the sibling ASR-timeout error already recommended. (#939)
|
||||
|
||||
### Changed
|
||||
|
||||
- **README and Linux docs no longer advertise a `.deb` package that isn't published.** `.deb` bundling is disabled in the release pipeline pending a tauri-cli fix; the docs now say so honestly instead of pointing at a file that was never in any release. (#961 investigation, #990)
|
||||
- **Linux install docs mention `yt-dlp` as an optional prerequisite** — previously only surfaced via an in-app warning after the fact. (#973)
|
||||
- **A benign Tauri startup warning no longer looks like an app problem.** On some Windows configurations, Tauri's own internal IPC fallback logs a warning that's fully harmless (it silently and successfully falls back to another transport) — it was spuriously flipping the Settings → Logs footer to show "1 warning" on every launch. Filtered out of the diagnostic capture. (#975)
|
||||
|
||||
## [0.3.11] — 2026-07-05
|
||||
|
||||
The multi-language release — dubbing into several languages at once is finally a mature, honest workflow: **"Generate N dubs" now translates each language before rendering it** (with visible per-language progress), **switching languages never destroys your work** (every track keeps its own text, subtitles, and audio cache), completed tracks always show their tabs, and dialogue stops starting seconds early because of footsteps — a community reporter's theory, confirmed exactly. Around it, a reliability sweep driven by same-day field reports: your LLM provider finally survives a restart, SOCKS-proxy users can synthesize again (installed models now load without touching the network at all), timeline boxes are visible on every WebView2 runtime, running from source works again — and when the backend crashes, **it now tells you the exit code and attaches the evidence to your bug report automatically**.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
**OmniVoice Studio**
|
||||
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The latest stable release is **v0.3.5**; `main` rolls ahead at **v0.3.6** (latest release + 1 patch — see the Versioning rule below).
|
||||
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
|
||||
|
||||
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
|
||||
|
||||
@@ -16,175 +16,21 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
|
||||
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
|
||||
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
|
||||
- **Local-first guarantee preserved**: Auto bug reporting (new addition) must be **opt-in**, must submit only to GitHub Issues (no third-party telemetry endpoint), and the app must remain fully functional with reporting disabled. No required cloud calls, accounts, or API keys.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release (currently **v0.3.5**). ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
|
||||
<!-- GSD:project-end -->
|
||||
|
||||
<!-- GSD:stack-start source:research/STACK.md -->
|
||||
## Technology Stack
|
||||
|
||||
## Recommended Stack — Per Capability
|
||||
### Capability 1 — HuggingFace Token Persistence (issue #35)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `huggingface_hub` (already pinned transitively by `transformers>=5.3.0`) | `≥1.12.x` (latest 2026) | Auth + cache + token storage | Canonical, used by every HF library already in the stack. `HfFolder` is **superseded** in v1.x by the higher-level `login()` / `auth_list()` / `auth_switch()` API. |
|
||||
| `keyring` (Python) | `≥25.x` | Optional OS-keychain backing | Only adopt if a future hardening pass wants Keychain/Credential-Manager/SecretService. **Not recommended for this milestone** — adds a native dep (`dbus`, `pywin32`) per platform with no real security win over `0600` file storage in `HF_HOME`. |
|
||||
| Shell | One-liner to persist `HF_TOKEN` |
|
||||
|-------|---------------------------------|
|
||||
| macOS zsh (default since 10.15) | `echo 'export HF_TOKEN=hf_xxx' >> ~/.zshrc && source ~/.zshrc` |
|
||||
| Linux bash | `echo 'export HF_TOKEN=hf_xxx' >> ~/.bashrc && source ~/.bashrc` |
|
||||
| Windows PowerShell (user scope) | `[Environment]::SetEnvironmentVariable("HF_TOKEN","hf_xxx","User")` (new shells only) |
|
||||
| Windows cmd | `setx HF_TOKEN "hf_xxx"` (user scope, new shells only) |
|
||||
- [HF environment variables docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH confidence (official, current)
|
||||
- [HF authentication API docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH confidence
|
||||
- [Microsoft `setx` docs](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH confidence
|
||||
### Capability 2 — In-App Structured Bug Reporting (opt-in, GitHub Issues)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| GitHub REST API `POST /repos/{owner}/{repo}/issues` | `2026-03-10` API version | Server-side issue creation | Official, stable. Requires auth. |
|
||||
| **Prefilled-URL pattern** (`github.com/{owner}/{repo}/issues/new?title=…&body=…&labels=…`) | n/a | Zero-auth fallback | **This is the recommended primary path for v0.3.x.** No token needed, no GitHub App registration needed, user's browser opens with a prefilled form, they review and click Submit. They own the issue, the OSS project gets the report, and OmniVoice never holds a credential. |
|
||||
| `gh-app-jwt` + GitHub App (Rust crate `octocrab` or Python `pygithub`) | only if we later want fully-automated submission | Programmatic posting under an app identity | **Defer to a later milestone.** Requires registering a public GitHub App, hosting a token-exchange endpoint, and managing rate-limit quotas — disproportionate for stabilization scope. |
|
||||
| `platform`, `psutil`, `torch.cuda` (already in deps) | already pinned | Capture OS, CPU/GPU/VRAM info | No new deps. |
|
||||
| `httpx` (already in `dev-dependencies`, promote to runtime if needed) | `≥0.28.1` | HTTP for the API call path (if/when we add auth) | Modern async-first, already used in test suite. |
|
||||
- ✓ No token storage in OmniVoice → no security surface
|
||||
- ✓ Opt-in by definition (user has to click Submit on github.com)
|
||||
- ✓ User owns the issue → can be replied to, edited, closed by them
|
||||
- ✓ Zero infra cost — no proxy, no app, no rate-limit management
|
||||
- ✓ Works identically on macOS / Windows / Linux via Tauri's `shell.open`
|
||||
- ✓ Survives our project being forked (just change the URL)
|
||||
- OS name + version (`platform.platform()`)
|
||||
- Python version (`sys.version`)
|
||||
- OmniVoice version (`pyproject.toml`)
|
||||
- Backend git SHA (if installed from source) or installer build ID
|
||||
- CPU model, RAM (`psutil.cpu_count()`, `psutil.virtual_memory()`)
|
||||
- GPU vendor/model/VRAM (`torch.cuda.get_device_name()`, `torch.cuda.mem_get_info()`, MPS detect)
|
||||
- Active TTS engine + list of installed engines
|
||||
- Frontend: bun version, OS shell
|
||||
- Last error message + stack trace if launched from an error toast
|
||||
- Audio file contents (privacy — reference samples may contain user's voice)
|
||||
- File paths containing `/Users/<name>/` (strip home dir → `~/`)
|
||||
- HF token, OpenAI keys, any env var matching `*TOKEN*|*KEY*|*SECRET*`
|
||||
- [GitHub URL query parameters for issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/creating-an-issue#creating-an-issue-from-a-url-query) — HIGH confidence
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (widely used reference impl)
|
||||
- [GitHub REST API: Create an issue](https://docs.github.com/en/rest/issues/issues#create-an-issue) — HIGH confidence (for the future auto-submit path)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — reviewed, **rejected for milestone** due to local-first constraint
|
||||
### Capability 3 — `uv venv` Mirror Fallback for Restricted Networks (issues #57, #60)
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `uv` (already used) | `≥0.5.x` | Python+venv bootstrap | Existing dep. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` env var | uv `0.4.x`+ | Override python-build-standalone download URL | **Official, current.** Replaces `https://github.com/astral-sh/python-build-standalone/releases/download/...` in download URL construction. No built-in fallback if mirror fails. |
|
||||
| `UV_PYTHON_PREFERENCE=only-system` (or CLI flag `--python-preference only-system`) | uv `0.4.x`+ | Skip the python-build-standalone download entirely; use the user's system Python | **The reliable escape hatch** when no mirror works. Requires a compatible Python `>=3.11` to already be on PATH. |
|
||||
| `UV_HTTP_TIMEOUT`, `UV_HTTP_CONNECT_TIMEOUT`, `UV_HTTP_RETRIES` | uv `0.4.x`+ | Tune retry behavior for flaky links | Defaults are 30s / 10s / 3 — bump to 120s / 30s / 5 for restricted networks. |
|
||||
# Pseudocode for the bootstrap
|
||||
# Final fallback: don't download Python at all
|
||||
- `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (Tsinghua — fastest in China)
|
||||
- `UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple` (Aliyun fallback)
|
||||
- Russia: no major government-blessed PyPI mirror; users typically tunnel via VPN. Document this honestly rather than ship a broken default.
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (official)
|
||||
- [uv issue #5224 — python-build-standalone mirror support](https://github.com/astral-sh/uv/issues/5224) — HIGH (the feature was added)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms real user pain, no built-in fallback)
|
||||
- [uv python-versions concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH (documents `python-preference` semantics)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained mirror list; verify each URL still works before shipping)
|
||||
### Capability 4 — Supertonic-3 TTS Engine
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| `supertonic` (PyPI) | `1.3.1` (latest, May 18 2026 — Phase 3 Wave 1 to verify constructor signature before bump) | Official Supertonic-3 inference SDK | Authoritative wrapper from Supertone Inc. Wraps the ONNX session orchestration so we don't have to. |
|
||||
| `onnxruntime` | `≥1.17.x` (any recent) | ONNX inference runtime | Already a transitive dep of WhisperX (via CTranslate2 path is separate, but `onnxruntime` itself ships for kittentts and audioseal). Verify with `uv tree` after adding — should resolve cleanly. |
|
||||
| `huggingface_hub` (already pinned) | `≥1.12.x` | Model weight download (~400 MB on first use) | Reuses existing HF token + cache infrastructure. The user's existing `HF_TOKEN` (Capability 1) works for the Supertonic model download too. |
|
||||
| `numpy`, `soundfile` (already pinned) | already pinned | Audio I/O + array math | No new deps. |
|
||||
- `text_encoder.onnx`
|
||||
- `latent_denoiser.onnx`
|
||||
- `voice_decoder.onnx`
|
||||
- 44.1 kHz sample rate, 24-dim latent, 128-dim style
|
||||
- ~99M parameters total
|
||||
- Tokenizer: `AutoTokenizer.from_pretrained(model_path)` — loads from `tokenizer.json` shipped with model
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official)
|
||||
- [supertonic PyPI page](https://pypi.org/project/supertonic/) — HIGH (`1.3.1` confirmed 2026-05-18; same publisher, MIT, same 4 deps)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure details)
|
||||
### Capability 5 — Cross-Platform Documentation Tooling
|
||||
| Technology | Version | Purpose | Why Recommended |
|
||||
|------------|---------|---------|-----------------|
|
||||
| Plain Markdown in `docs/` + GitHub-rendered (current state) | n/a | Install tutorial, troubleshooting | Zero new infra. Renders inline on GitHub for issue-replies. No build step to break. |
|
||||
| Existing `scripts/smoke-test.sh` + Playwright `tests/` (already in `package.json`) | already pinned | Verify install paths actually work | **This is the real solution to "docs drift."** If smoke-test exercises the install path described in docs, docs that drift will break CI. |
|
||||
| **Future** (defer): Astro Starlight | `≥0.30` | Standalone docs site at `docs.omnivoice.studio` | Adopt only when docs exceed ~20 markdown files and need search/versioning. Tauri, the framework OmniVoice already depends on, uses Starlight — well-traveled choice. Material for MkDocs entered maintenance mode in November 2025 per Docsio's 2026 review — **avoid** for new docs. |
|
||||
| Project | What they do |
|
||||
|---------|--------------|
|
||||
| **OBS Studio** | Docs at `obsproject.com/docs` (Sphinx, separate repo). Install paths in README, wiki for community-contributed. CI doesn't gate on docs drift. |
|
||||
| **Audacity** | Manual at `manual.audacityteam.org` (MediaWiki). README is minimal. Install path = "use the installer." No automated sync. |
|
||||
| **Tauri** | Docs at `v2.tauri.app` (Astro Starlight, separate repo `tauri-apps/tauri-docs`). README is minimal. Heavy reliance on community contributions and PR review. |
|
||||
| **VS Code** | Docs at `code.visualstudio.com/docs` (separate repo, Markdown). README is minimal. Manual sync; docs team is staffed. |
|
||||
- [Tauri docs (Astro Starlight)](https://github.com/tauri-apps/tauri-docs) — HIGH (reference for "if we ever move off README")
|
||||
- [OBS Studio docs](https://docs.obsproject.com/) — HIGH (Sphinx, separate site reference)
|
||||
- [Audacity Manual](https://manual.audacityteam.org/) — HIGH (MediaWiki reference)
|
||||
- [Docsio: Material for MkDocs 2026 review (maintenance mode)](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with project's own GitHub activity)
|
||||
- [Docsio: Starlight 2026 review](https://docsio.co/blog/starlight-docs) — MEDIUM
|
||||
## Installation
|
||||
# No new Python dependencies needed for Capabilities 1, 2, 3, 5.
|
||||
# Only Capability 4 adds a runtime dep:
|
||||
# Verify no regressions:
|
||||
# Should show single versions of each; no duplicates.
|
||||
## Alternatives Considered
|
||||
| Recommended | Alternative | When to Use Alternative |
|
||||
|-------------|-------------|-------------------------|
|
||||
| HF token via in-app Settings → `huggingface_hub.login()` | OS keyring via `keyring` package | Only if a security hardening milestone later demands OS-native credential storage. Not worth the cross-platform native-dep cost for v0.3.x. |
|
||||
| Prefilled-URL GitHub Issues | GitHub App + device flow + authenticated POST | When milestone budget can afford registering a public GitHub App and hosting a token-exchange function. Defer. |
|
||||
| Prefilled-URL GitHub Issues | Sentry / `sentry-tauri` | Never — violates the "no third-party telemetry endpoint" constraint in PROJECT.md. |
|
||||
| `UV_PYTHON_INSTALL_MIRROR` chain + `only-system` fallback | Bundle Python in the Tauri installer | Adds ~30 MB to every installer for ~5% of users. Revisit if the bootstrap is still a top complaint in v0.4. |
|
||||
| In-repo Markdown docs | Astro Starlight standalone site | When docs grow past ~20 pages and need full-text search. Tauri provides a precedent if/when we get there. |
|
||||
| In-repo Markdown docs | MkDocs / Material for MkDocs | **Avoid** for new sites — Material for MkDocs is in maintenance mode as of Nov 2025. |
|
||||
## What NOT to Use
|
||||
| Avoid | Why | Use Instead |
|
||||
|-------|-----|-------------|
|
||||
| `HfFolder.save_token()` directly | Older API; v1.x `login()` does the same plus git-credential integration and is the documented path | `huggingface_hub.login(token=val, add_to_git_credential=False)` |
|
||||
| Setting `HF_TOKEN` via shell rc files as the *only* persistence mechanism | Different per OS, fragile, opaque to the user, breaks in installer-launched processes that don't source shell rc | Write to `$HF_HOME/token` via `login()`. Document env var as override only. |
|
||||
| `setx` for HF token persistence | Doesn't propagate to current shell; common source of "I set it but it's empty" bug reports | `[Environment]::SetEnvironmentVariable(...,"User")` in PowerShell, or the in-app Settings field |
|
||||
| PAT-based GitHub Issues posting from OmniVoice | Would require shipping or asking for a token; breaks local-first promise | Prefilled-URL pattern (user submits from their browser) |
|
||||
| `sentry-tauri` for OmniVoice | Third-party telemetry endpoint — violates PROJECT.md constraint | Local-only `backend.log` rotation + opt-in prefilled-URL reporter |
|
||||
| `hf_transfer` for downloads | Deprecated in favor of `hf-xet` per HF docs | Default `huggingface_hub` (uses `hf-xet` automatically when available) |
|
||||
| `--python-preference managed` (default) without mirror config in restricted-network installers | Hits GitHub CDN, times out, user sees raw `uv` error | Configure `UV_PYTHON_INSTALL_MIRROR` + retry chain + `only-system` final fallback |
|
||||
| Material for MkDocs as a *new* docs choice | Entered maintenance mode November 2025 | If docs site is eventually needed, use Astro Starlight (Tauri precedent) |
|
||||
## Stack Patterns by Variant
|
||||
- Set `UV_PYTHON_INSTALL_MIRROR` to one of the gh-proxy URLs at install time
|
||||
- Set `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (China) or document VPN requirement (Russia)
|
||||
- Fall back to `UV_PYTHON_PREFERENCE=only-system` if all mirrors fail
|
||||
- Increase `UV_HTTP_TIMEOUT=120`, `UV_HTTP_RETRIES=5`
|
||||
- Default path: in-app Settings field → `login()` → file at `$HF_HOME/token`
|
||||
- Power-user path: `export HF_TOKEN=...` in shell rc (documented but not promoted)
|
||||
- Both paths are read at HF library import time; env var wins on conflict
|
||||
- Default path: in-app "Report a bug" → prefilled GitHub Issues URL → user reviews + submits in browser
|
||||
- All optional capture toggles default ON except "include reproduction file" (privacy)
|
||||
- No path posts to any URL except `github.com/{owner}/{repo}/issues/new` (rendered locally as a URL, opened via `shell.open`)
|
||||
- `uv add supertonic` → new TTSBackend subclass in `backend/services/tts_backend.py`
|
||||
- Auto-detected and added to the engine picker in Settings
|
||||
- ~400 MB model download on first synthesize call, cached in `$HF_HUB_CACHE`
|
||||
- Existing IndexTTS/CosyVoice/etc. installs are untouched (no shared model weights)
|
||||
## Version Compatibility
|
||||
| Package A | Compatible With | Notes |
|
||||
|-----------|-----------------|-------|
|
||||
| `supertonic@1.3.1` | `onnxruntime>=1.17`, `numpy>=1.24`, `huggingface_hub>=0.20` | All deps already satisfied transitively by current `pyproject.toml`. |
|
||||
| `huggingface_hub>=1.12` | `transformers>=5.3.0` (current pin) | `HfFolder` retained as deprecated alias; `login()`/`get_token()` are the canonical APIs. |
|
||||
| `uv>=0.5` | `UV_PYTHON_INSTALL_MIRROR`, `UV_PYTHON_PREFERENCE` | Both env vars stable since uv 0.4.x. |
|
||||
| Tauri v2 + `@tauri-apps/api/shell` | `shell.open()` for the prefilled-URL pattern | Already in the desktop app; no new permission needed beyond what the existing "open external link" plugin grants. |
|
||||
## Sources
|
||||
- [Hugging Face Hub environment variables](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH (verified against v1.12.1 docs, current 2026)
|
||||
- [Hugging Face Hub authentication API](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH (verified `login()` is the canonical 1.x API)
|
||||
- [Microsoft `setx` reference](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH (confirms "current shell" gotcha)
|
||||
- [PowerShell `about_Environment_Variables`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables) — HIGH
|
||||
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (verified all mirror + retry env vars)
|
||||
- [uv issue #5224 — python-build-standalone mirror](https://github.com/astral-sh/uv/issues/5224) — HIGH (feature shipped)
|
||||
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms user pain, justifies fallback chain)
|
||||
- [uv `python-preference` semantics](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH
|
||||
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official, 99M params, 31 languages, OpenRAIL-M)
|
||||
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official inference API)
|
||||
- [supertonic 1.3.1 on PyPI](https://pypi.org/project/supertonic/) — HIGH (released 2026-05-18, MIT code license; bumped from 1.2.3 after Phase 3 research)
|
||||
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure)
|
||||
- [GitHub Docs: Authenticating to the REST API](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api) — HIGH
|
||||
- [GitHub Docs: Generating a user access token for a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) — HIGH (device flow reference)
|
||||
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (canonical prefilled-URL reference impl)
|
||||
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — MEDIUM (reviewed, rejected on PROJECT.md constraint, not on quality)
|
||||
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained, verify URLs are still live before pinning in production)
|
||||
- [Tauri 2 docs (Astro Starlight reference)](https://v2.tauri.app/) — HIGH (precedent for docs framework if we ever migrate)
|
||||
- [Docsio: Material for MkDocs entered maintenance mode Nov 2025](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with the project's own GitHub commit activity)
|
||||
The May-2026 stack research that used to live here served five capabilities that have all since shipped (HF-token Settings panel, prefilled-URL bug reporting, uv mirror fallback for restricted networks, the Supertonic-3 engine, in-repo Markdown docs). Follow the patterns in the code itself; the durable *don'ts* that research established:
|
||||
|
||||
- **No third-party telemetry endpoints, ever** (`sentry-tauri` was evaluated and rejected) — bug reporting stays opt-in via prefilled GitHub-issue URLs.
|
||||
- **No PAT/token-based GitHub posting from the app** — the user submits from their own browser.
|
||||
- **Don't recommend `setx` for env vars on Windows** (silent truncation, no current-shell propagation) — use the in-app Settings panel or PowerShell `[Environment]::SetEnvironmentVariable`.
|
||||
- **Don't adopt Material for MkDocs** for any future docs site (maintenance mode since Nov 2025) — Astro Starlight is the precedent if docs ever outgrow the repo.
|
||||
- **`hf_transfer` is deprecated** — default `huggingface_hub` (hf-xet) handles downloads.
|
||||
|
||||
For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/package.json`, and check `uv tree` for conflicts before adding a dependency.
|
||||
<!-- GSD:stack-end -->
|
||||
|
||||
<!-- GSD:conventions-start source:CONVENTIONS.md -->
|
||||
@@ -222,16 +68,9 @@ No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skill
|
||||
<!-- GSD:skills-end -->
|
||||
|
||||
<!-- GSD:workflow-start source:GSD defaults -->
|
||||
## GSD Workflow Enforcement
|
||||
## Workflow
|
||||
|
||||
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
|
||||
|
||||
Use these entry points:
|
||||
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
|
||||
- `/gsd-debug` for investigation and bug fixing
|
||||
- `/gsd-execute-phase` for planned phase work
|
||||
|
||||
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
|
||||
Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command gate that used to live here referenced `/gsd-quick` / `/gsd-debug` / `/gsd-execute-phase` skills that are not installed in this environment; the owner chose to keep working directly rather than restore them. The working conventions that matter are in **Conventions** above — versioning, docs-sync, changelog, localization, fix quality, keep-main-green — plus: gate every merge on the "Tests (backend + frontend)" check passing and the PR being MERGEABLE, and check the open-PR queue before implementing any community-reported fix (contributors may have already submitted one).
|
||||
<!-- GSD:workflow-end -->
|
||||
|
||||
|
||||
|
||||
@@ -190,7 +190,6 @@ The eight headliners — and twelve more waiting under the fold.
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
|
||||
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
|
||||
<br/>
|
||||
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy & Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
|
||||
<br/>
|
||||
@@ -267,7 +266,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
| | **Minimum** | **Recommended** |
|
||||
|---|---|---|
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 20.04+ | Any modern 64-bit OS |
|
||||
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
|
||||
| **RAM** | 8 GB | 16 GB+ |
|
||||
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
|
||||
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
|
||||
@@ -287,7 +286,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🗣️ TTS Engines
|
||||
|
||||
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
|
||||
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var — the selection applies everywhere synthesis happens: single-clip generation, Voice Cloning, Video Dubbing, and Batch TTS.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
|
||||
@@ -313,6 +312,8 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
|
||||
>
|
||||
> **Clone** matters beyond single-clip generation: Video Dubbing (and any Batch job with a pinned voice) needs reference-audio cloning to preserve speaker identity, so picking a Clone-less engine (KittenTTS, Sherpa-ONNX, Supertonic 3) as the active engine fails those jobs up front with an actionable message instead of silently falling back to OmniVoice.
|
||||
>
|
||||
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
|
||||
|
||||
</details>
|
||||
@@ -321,10 +322,10 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
|
||||
### 🎧 ASR Engines
|
||||
|
||||
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
|
||||
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines** (the ASR Engines table — same picker TTS has), or pin one with the `OMNIVOICE_ASR_BACKEND` env var (the env var wins over the Settings pick). Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
|
||||
|
||||
<details>
|
||||
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
|
||||
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -339,6 +340,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
|
||||
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
|
||||
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
|
||||
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
|
||||
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
|
||||
|
||||
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
|
||||
|
||||
|
||||
@@ -179,6 +179,21 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
job["status"] = "failed"
|
||||
return
|
||||
|
||||
# ── Engine resolution (issue #312 class) ────────────────────────────
|
||||
# Batch used to hardcode OmniVoice via get_model() regardless of the
|
||||
# engine selected in Settings → Engines. require_cloning only when a
|
||||
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
|
||||
# any active engine. Resolved ONCE for the whole job (every language
|
||||
# below shares the same active engine); an uncaught ValueError here
|
||||
# propagates to _worker()'s existing except-Exception handling, which
|
||||
# already records a structured job failure via core.failure.build_failure.
|
||||
from services.tts_backend import resolve_generation_backend
|
||||
backend = await resolve_generation_backend(
|
||||
require_cloning=bool(job.get("voice_id")),
|
||||
cloning_purpose="this batch job's pinned voice",
|
||||
)
|
||||
sr = backend.sample_rate
|
||||
|
||||
# ── 3. Translate + Generate per language ───────────────────────────
|
||||
total_langs = len(langs)
|
||||
outputs = {}
|
||||
@@ -243,13 +258,10 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
total_segments=len(translated_segments),
|
||||
)
|
||||
|
||||
from services.model_manager import get_model
|
||||
from services.audio_dsp import apply_mastering, normalize_audio
|
||||
from services.audio_io import atomic_save_wav
|
||||
import torch
|
||||
|
||||
_model = await get_model()
|
||||
sr = _model.sampling_rate
|
||||
total_samples = int(duration * sr)
|
||||
full_audio = torch.zeros(1, total_samples)
|
||||
total_segs = len(translated_segments)
|
||||
@@ -295,22 +307,16 @@ async def _run_batch_pipeline(job_id: str, job: dict):
|
||||
ref_text = row.get("ref_text")
|
||||
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
duration=dur, num_step=16,
|
||||
guidance_scale=2.0, speed=1.0,
|
||||
denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered = apply_mastering(
|
||||
audio_out,
|
||||
sample_rate=sr,
|
||||
)
|
||||
return normalize_audio(mastered, target_dBFS=-2.0)
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
audio_out = apply_mastering(audio_out, sample_rate=sr)
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
except Exception as e:
|
||||
logger.warning("TTS failed for seg %d (lang=%s): %s", i, lang, e)
|
||||
return torch.zeros(1, int(dur * sr))
|
||||
|
||||
@@ -1006,6 +1006,11 @@ async def dub_transcribe_stream(
|
||||
clones = done.pop().result()
|
||||
break
|
||||
yield _sse_event("ping", {})
|
||||
if clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(clones, _asr_backend),
|
||||
)
|
||||
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
|
||||
# own reference from the vocals so the dub of each line matches the
|
||||
# prosody of its source line. Short lines fall back to the
|
||||
@@ -1025,6 +1030,10 @@ async def dub_transcribe_stream(
|
||||
),
|
||||
)
|
||||
if seg_clones:
|
||||
from services.speaker_clone import refine_ref_texts
|
||||
seg_clones = await loop.run_in_executor(
|
||||
_gpu_pool, lambda: refine_ref_texts(seg_clones, _asr_backend),
|
||||
)
|
||||
job["segment_clones"] = seg_clones
|
||||
except Exception as e:
|
||||
logger.warning("per-segment clone refs skipped: %s", e)
|
||||
|
||||
@@ -11,7 +11,8 @@ from core.db import db_conn
|
||||
from core.config import DUB_DIR, VOICES_DIR, dub_seg_path
|
||||
from core.tasks import task_manager
|
||||
from schemas.requests import DubRequest
|
||||
from services.model_manager import get_model, _gpu_pool, run_on_gpu_pool_guarded
|
||||
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
|
||||
from services.tts_backend import resolve_generation_backend
|
||||
from services.audio_dsp import apply_mastering, normalize_audio, apply_effects_chain, get_effect_chain
|
||||
from services.audio_io import atomic_save_wav, _safe_torchaudio_save
|
||||
from services.ffmpeg_utils import (
|
||||
@@ -156,7 +157,19 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
detail="This dub session has expired or was never created. Re-upload the video to start a new one.",
|
||||
)
|
||||
|
||||
_model = await get_model()
|
||||
# ── Engine resolution (issue #312 class) ────────────────────────────────
|
||||
# Dub used to hardcode OmniVoice via get_model() regardless of the engine
|
||||
# selected in Settings → Engines — a SILENT fallback. Every real dub
|
||||
# segment's ref_audio resolves to either an auto:<speaker>/auto-seg:<id>
|
||||
# clone cut from the source video or a saved voice-profile row (see
|
||||
# `_gen` below), so require_cloning=True: an engine that can't clone
|
||||
# would either mis-clone per segment or fail deep into the job. Checked
|
||||
# ONCE here, before the streaming task starts, so a doomed job fails fast
|
||||
# with one clear message instead of N per-segment ones.
|
||||
try:
|
||||
backend = await resolve_generation_backend(require_cloning=True)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
async def _stream(task_id):
|
||||
total = len(req.segments)
|
||||
@@ -343,7 +356,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
|
||||
seg_duration = seg.end - seg.start
|
||||
if seg_duration <= 0.05 or not seg.text.strip():
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
# max(0, …): a zero/negative-duration slot must not feed a
|
||||
# negative length to torch.zeros (raises) — _store_mix_wav
|
||||
# turns the empty buffer into a harmless in-memory entry.
|
||||
@@ -377,21 +390,21 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
try:
|
||||
_t_cache_0 = time.perf_counter()
|
||||
cached_wav, cached_sr = torchaudio.load(seg_wav_path)
|
||||
if cached_sr != _model.sampling_rate:
|
||||
if cached_sr != backend.sample_rate:
|
||||
import torchaudio.functional as AF
|
||||
cached_wav = AF.resample(cached_wav, cached_sr, _model.sampling_rate)
|
||||
cached_wav = AF.resample(cached_wav, cached_sr, backend.sample_rate)
|
||||
# Pad/trim to slot — except smart_fit, whose mix
|
||||
# loop needs the natural-rate length to compute the
|
||||
# audio/video split (the seg_wav_kind guard above
|
||||
# guarantees these cached WAVs are natural-rate).
|
||||
if strategy != "smart_fit":
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = cached_wav.shape[-1]
|
||||
if target_samples > current_samples:
|
||||
cached_wav = torch.nn.functional.pad(cached_wav, (0, target_samples - current_samples))
|
||||
elif current_samples > target_samples:
|
||||
cached_wav = cached_wav[..., :target_samples]
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, _model.sampling_rate, f"mix_{seg_id}"))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, backend.sample_rate, f"mix_{seg_id}"))
|
||||
try:
|
||||
del cached_wav
|
||||
except Exception:
|
||||
@@ -404,7 +417,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# Fall through to a silent placeholder if the cached WAV
|
||||
# is broken — cleaner than aborting the whole mix.
|
||||
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'cached seg lost, padding silence: {str(e)[:120]}'})}\n\n"
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
silence = torch.zeros(1, max(0, int(seg_duration * sr)))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, silence, sr, f"mix_{seg_id}"))
|
||||
try:
|
||||
@@ -488,25 +501,23 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
torch.manual_seed(used_seed)
|
||||
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang if lang != "Auto" else None,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
instruct=instruct_str if instruct_str else None,
|
||||
duration=dur_s, num_step=nstep, guidance_scale=cfg,
|
||||
speed=spd, denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
|
||||
sr = backend.sample_rate
|
||||
|
||||
# Apply per-segment DSP effect preset (default: broadcast)
|
||||
seg_effect_preset = effect_preset or "broadcast"
|
||||
if seg_effect_preset == "raw":
|
||||
return audio_out
|
||||
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
mastered_audio = audio_out
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
effect_chain = get_effect_chain(seg_effect_preset)
|
||||
if effect_chain:
|
||||
mastered_audio = apply_effects_chain(
|
||||
@@ -537,24 +548,22 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
nstep, retry_steps,
|
||||
)
|
||||
try:
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=text, language=lang if lang != "Auto" else None,
|
||||
ref_audio=ref_audio, ref_text=ref_text,
|
||||
instruct=instruct_str if instruct_str else None,
|
||||
duration=dur_s, num_step=retry_steps, guidance_scale=cfg,
|
||||
speed=spd, denoise=True, postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
|
||||
sr = backend.sample_rate
|
||||
|
||||
seg_effect_preset = effect_preset or "broadcast"
|
||||
if seg_effect_preset == "raw":
|
||||
return audio_out
|
||||
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
mastered_audio = audio_out
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
|
||||
effect_chain = get_effect_chain(seg_effect_preset)
|
||||
if effect_chain:
|
||||
mastered_audio = apply_effects_chain(
|
||||
@@ -641,7 +650,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
yield f"data: {json.dumps({'type': 'cancelled', 'segments_processed': i + 1})}\n\n"
|
||||
return
|
||||
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = audio_tensor.shape[-1]
|
||||
|
||||
if strategy == "strict_slot":
|
||||
@@ -659,7 +668,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# trim, slip, stretch the video, or split audio/video
|
||||
# retiming (smart_fit) to accommodate it.
|
||||
|
||||
generated_dur = audio_tensor.shape[-1] / _model.sampling_rate
|
||||
generated_dur = audio_tensor.shape[-1] / backend.sample_rate
|
||||
sync_ratio = round(generated_dur / max(seg_duration, 0.01), 3)
|
||||
|
||||
sync_scores.append(sync_ratio)
|
||||
@@ -683,20 +692,20 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
except Exception as e:
|
||||
logger.debug("seg fingerprint skipped for %s: %s", seg_id, e)
|
||||
|
||||
_pending_seg_writes.append((i, _model.sampling_rate, seg_id, _seg_fp, _num_step))
|
||||
_pending_seg_writes.append((i, backend.sample_rate, seg_id, _seg_fp, _num_step))
|
||||
|
||||
# RVC needs the WAV on disk, so write it immediately only
|
||||
# when RVC is active (uncommon path).
|
||||
if rvc_is_enabled():
|
||||
seg_wav_path = _seg_lang_path(seg_id)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
|
||||
try:
|
||||
await loop.run_in_executor(_gpu_pool, apply_rvc, seg_wav_path)
|
||||
rvc_wav, rvc_sr = torchaudio.load(seg_wav_path)
|
||||
if rvc_sr == _model.sampling_rate:
|
||||
if rvc_sr == backend.sample_rate:
|
||||
audio_tensor = rvc_wav
|
||||
|
||||
target_samples = int(seg_duration * _model.sampling_rate)
|
||||
target_samples = int(seg_duration * backend.sample_rate)
|
||||
current_samples = audio_tensor.shape[-1]
|
||||
if target_samples > current_samples:
|
||||
audio_tensor = torch.nn.functional.pad(audio_tensor, (0, target_samples - current_samples))
|
||||
@@ -713,25 +722,25 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
# no double-mark. Cached-reuse audio is already marked;
|
||||
# silence/zero slots carry no speech to mark, so neither is
|
||||
# re-watermarked.
|
||||
audio_tensor = embed_watermark(audio_tensor, _model.sampling_rate)
|
||||
audio_tensor = embed_watermark(audio_tensor, backend.sample_rate)
|
||||
|
||||
seg_wav_path = _seg_lang_path(seg_id)
|
||||
try:
|
||||
# Keep the existing per-segment WAV contract for previews
|
||||
# and partial regeneration, but do not keep the tensor in RAM.
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
|
||||
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
|
||||
except Exception as e:
|
||||
logger.warning("seg write failed for %s: %s", seg_id, e)
|
||||
# If the durable segment write fails, still preserve a mix
|
||||
# copy so this generation can finish.
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, _model.sampling_rate, f"mix_{seg_id}"))
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, backend.sample_rate, f"mix_{seg_id}"))
|
||||
try:
|
||||
del audio_tensor
|
||||
except Exception:
|
||||
pass
|
||||
_release_audio_tensors()
|
||||
else:
|
||||
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, _model.sampling_rate))
|
||||
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, backend.sample_rate))
|
||||
try:
|
||||
del audio_tensor
|
||||
except Exception:
|
||||
@@ -739,7 +748,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
_release_audio_tensors()
|
||||
except Exception as e:
|
||||
yield f"data: {json.dumps({'type': 'error', 'segment': i, 'error': str(e)})}\n\n"
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, torch.zeros(1, max(0, int(seg_duration * sr))), sr, f"mix_{seg_id}"))
|
||||
sync_scores.append(1.0)
|
||||
|
||||
@@ -768,7 +777,7 @@ async def dub_generate(job_id: str, req: DubRequest):
|
||||
_save_job(job_id, job)
|
||||
_t_diskw = time.perf_counter() - _t_diskw_0
|
||||
|
||||
sr = _model.sampling_rate
|
||||
sr = backend.sample_rate
|
||||
slot_fit = (req.slot_fit or "time_stretch").lower()
|
||||
overflow_budget_s = max(0.0, float(req.overflow_budget_s or 0.0))
|
||||
|
||||
@@ -1196,7 +1205,13 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
_model = await get_model()
|
||||
# See the /dub/generate/{job_id} resolution above (issue #312 class) —
|
||||
# a segment preview resolves ref_audio from the same auto-clone /
|
||||
# voice-profile sources, so it needs the same cloning-capable gate.
|
||||
try:
|
||||
backend = await resolve_generation_backend(require_cloning=True)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
def _gen():
|
||||
ref_audio = None
|
||||
@@ -1231,7 +1246,7 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
instruct_str = row["instruct"]
|
||||
|
||||
lang = req.language if req.language != "Auto" else None
|
||||
audios = _model.generate(
|
||||
audio_out = backend.generate(
|
||||
text=req.text,
|
||||
language=lang,
|
||||
ref_audio=ref_audio,
|
||||
@@ -1244,21 +1259,15 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
|
||||
denoise=True,
|
||||
postprocess_output=True,
|
||||
)
|
||||
audio_out = audios[0]
|
||||
# TODO(#312): this route runs the OmniVoice model directly (not the active
|
||||
# backend), so VoxCPM2 never reaches it. When these routes become
|
||||
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
|
||||
mastered = apply_mastering(
|
||||
audio_out,
|
||||
sample_rate=getattr(_model, "sampling_rate", 24000),
|
||||
)
|
||||
return normalize_audio(mastered, target_dBFS=-2.0)
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
audio_out = apply_mastering(audio_out, sample_rate=backend.sample_rate)
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
|
||||
# Bounded + pool-reset on hang so a wedged preview generate can't starve the
|
||||
# GPU pool and brick the backend (#730 class).
|
||||
audio_tensor = await run_on_gpu_pool_guarded(_gen, what="Dub preview generate")
|
||||
|
||||
sr = getattr(_model, "sampling_rate", 24000)
|
||||
sr = backend.sample_rate
|
||||
buf = io.BytesIO()
|
||||
_safe_torchaudio_save(buf, audio_tensor, sr, format="wav")
|
||||
buf.seek(0)
|
||||
|
||||
@@ -16,6 +16,7 @@ Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`,
|
||||
a backend without Settings silently undoing it.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from time import perf_counter
|
||||
|
||||
@@ -429,6 +430,11 @@ def engine_selftest(engine_id: str):
|
||||
class SelectEngineRequest(BaseModel):
|
||||
family: str # "tts" | "asr" | "llm"
|
||||
backend_id: str
|
||||
# Only meaningful for family="tts", backend_id="mlx-audio" (#981) — picks
|
||||
# which of mlx-audio's curated models is actually loaded. A curated key
|
||||
# ("kokoro") or a raw HF repo id ("mlx-community/Kokoro-82M-bf16") — the
|
||||
# same tolerance MLXAudioBackend.__init__ already has. Ignored otherwise.
|
||||
model_id: str | None = None
|
||||
|
||||
|
||||
class SelectEngineResponse(BaseModel):
|
||||
@@ -471,6 +477,24 @@ def select_engine(req: SelectEngineRequest):
|
||||
f"Backend {req.backend_id} can't run on this machine: {why}. "
|
||||
f"Pick an engine with a CPU path, or one that supports this host's GPU.",
|
||||
)
|
||||
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
|
||||
# persist the model pick alongside the backend id so the UI can actually
|
||||
# select which curated model gets loaded (previously it always defaulted
|
||||
# to Kokoro no matter what the user downloaded in Settings → Models).
|
||||
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
|
||||
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
|
||||
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
|
||||
# tolerance MLXAudioBackend.__init__ already has for power users.
|
||||
# Anything else (typo'd key, malformed id) is rejected outright
|
||||
# rather than silently persisted as a "custom repo" that then fails
|
||||
# to resolve at load time.
|
||||
if req.model_id not in known_keys and not re.fullmatch(r"[\w.-]+/[\w.-]+", req.model_id):
|
||||
raise HTTPException(
|
||||
400,
|
||||
f"Unknown mlx-audio model: {req.model_id!r}. Expected one of "
|
||||
f"{sorted(known_keys)} or a HF repo id like 'owner/name'.",
|
||||
)
|
||||
prefs.set_("mlx_audio_model_id", req.model_id)
|
||||
prefs.set_(pref_key, req.backend_id)
|
||||
return {
|
||||
"family": req.family,
|
||||
|
||||
@@ -105,8 +105,8 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
|
||||
|
||||
``skip_mastering`` honors a backend's ``applies_own_mastering`` flag
|
||||
(issue #312): studio engines (e.g. VoxCPM2's native 48 kHz output)
|
||||
opt out of the broadcast Compressor + Reverb chain that's tuned for
|
||||
OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
opt out of the broadcast highpass + Compressor pre-stage that's tuned
|
||||
for OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
it's a benign peak scale. Mirrors ``_run_tts`` in openai_compat.py.
|
||||
"""
|
||||
from services.audio_dsp import (
|
||||
@@ -143,6 +143,28 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
|
||||
return normalize_audio(audio_out, target_dBFS=-2.0)
|
||||
|
||||
|
||||
def _safe_exc_text(e: BaseException) -> str:
|
||||
"""``f"{type(e).__name__}: {e}"`` — the house style used for
|
||||
unrecognized-error formatting throughout the backend (grep
|
||||
``type(e).__name__`` in settings.py / asr_backend.py / model_manager.py
|
||||
/ engines.py) — with a guard against leaking a raw container repr.
|
||||
|
||||
#977: an AssertionError raised deep inside a vendored dependency
|
||||
(mlx-audio's Kokoro pipeline) had ``.args`` shaped like
|
||||
``('du', {'a': 'American English', ...})`` — a tuple containing a dict.
|
||||
``str(e)`` on that renders the WHOLE table straight into the user-facing
|
||||
message. Any engine's ``generate()`` can raise something shaped like
|
||||
this (not just Kokoro), so guard generically: if any element of
|
||||
``e.args`` is a container rather than a plain string, don't interpolate
|
||||
``str(e)`` at all — name the exception type and point at the log
|
||||
instead.
|
||||
"""
|
||||
args = getattr(e, "args", ())
|
||||
if any(isinstance(a, (dict, list, tuple, set, frozenset)) for a in args):
|
||||
return f"{type(e).__name__} — see Settings → Logs → Backend for details"
|
||||
return f"{type(e).__name__}: {e}"
|
||||
|
||||
|
||||
def _exception_chain(e):
|
||||
"""Yield ``e`` plus every ``__cause__``/``__context__`` beneath it
|
||||
(cycle-safe). Engines and hub libraries routinely wrap the original
|
||||
@@ -412,7 +434,7 @@ def _oom_friendly_reraise(e):
|
||||
raise RuntimeError(
|
||||
f"TTS engine stopped mid-generation with an error OmniVoice doesn't "
|
||||
f"recognize. Retry once; if it keeps failing, please report it with "
|
||||
f"the full trace. Underlying error: {e}"
|
||||
f"the full trace. Underlying error: {_safe_exc_text(e)}"
|
||||
) from e
|
||||
|
||||
|
||||
@@ -934,7 +956,7 @@ async def generate_speech(
|
||||
status_code=500,
|
||||
detail=(
|
||||
f"Couldn't synthesize audio. See Settings → Logs → Backend for the full trace. "
|
||||
f"Underlying error: {e}"
|
||||
f"Underlying error: {_safe_exc_text(e)}"
|
||||
),
|
||||
)
|
||||
finally:
|
||||
|
||||
@@ -237,10 +237,10 @@ def _run_tts(backend, text: str, kw: dict):
|
||||
sr = backend.sample_rate
|
||||
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
|
||||
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
|
||||
# That chain's Compressor + 8% Reverb is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump and a
|
||||
# reverb tail that degrade the very output we want clean. Loudness
|
||||
# normalisation still runs — it's a benign peak scale, not dynamics.
|
||||
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump that
|
||||
# degrades the very output we want clean. Loudness normalisation still
|
||||
# runs — it's a benign peak scale, not dynamics.
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
wav = apply_mastering(wav, sample_rate=sr)
|
||||
wav = normalize_audio(wav, target_dBFS=-2.0)
|
||||
|
||||
@@ -73,6 +73,17 @@ async def create_profile(
|
||||
raise ValueError("not an object")
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=422, detail="vd_states must be a JSON object")
|
||||
# Root-cause close for #983: a design profile must never be PERSISTED
|
||||
# with a partial vd_states shape, regardless of which client (older
|
||||
# frontend build, hand-edited payload, third-party API caller) created
|
||||
# it — a missing category key crashes DesignMethodPanel's render on
|
||||
# every future client that selects this profile. CATEGORY_ORDER is the
|
||||
# same single source of truth the frontend's CATEGORIES keys mirror
|
||||
# (core/describe_voice.py), so this can't drift from the picker.
|
||||
from core.describe_voice import CATEGORY_ORDER
|
||||
for _cat in CATEGORY_ORDER:
|
||||
parsed.setdefault(_cat, "Auto")
|
||||
vd_states = _json.dumps(parsed)
|
||||
# An all-Auto design (every category left on "Auto") yields an empty
|
||||
# instruct — that's still a valid, saveable voice: synthesis falls back
|
||||
# to neutral instruct-only conditioning (see generation.py design path).
|
||||
|
||||
@@ -750,6 +750,51 @@ def set_hf_mirror(body: _HFMirrorBody):
|
||||
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
|
||||
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
|
||||
# without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
# base_url/model are plain settings_store text rows; the key is encrypted via
|
||||
# settings_store.set_secret — same convention as /llm-providers, never
|
||||
# returned to the client, '' clears it, omitted/None leaves it unchanged.
|
||||
|
||||
|
||||
class _ASROpenAICompatBody(BaseModel):
|
||||
base_url: str | None = None
|
||||
model: str | None = None
|
||||
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
|
||||
|
||||
|
||||
@router.get("/asr-openai-compat")
|
||||
def get_asr_openai_compat():
|
||||
from services import asr_backend
|
||||
|
||||
return {
|
||||
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
|
||||
"model": asr_backend.resolve_openai_compat_asr_model(),
|
||||
"has_key": asr_backend.openai_compat_asr_has_key(),
|
||||
}
|
||||
|
||||
|
||||
@router.put("/asr-openai-compat")
|
||||
def set_asr_openai_compat(body: _ASROpenAICompatBody):
|
||||
from services import asr_backend, settings_store
|
||||
|
||||
if body.base_url is not None:
|
||||
url = body.base_url.strip().rstrip("/")
|
||||
if url and not url.startswith(("http://", "https://")):
|
||||
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
|
||||
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
|
||||
if body.model is not None:
|
||||
settings_store.set_text(
|
||||
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
|
||||
)
|
||||
if body.api_key is not None:
|
||||
settings_store.set_secret(
|
||||
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
|
||||
)
|
||||
return get_asr_openai_compat()
|
||||
|
||||
|
||||
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
|
||||
# (feat/safe-updates). Both are read-only, local-first surfaces for
|
||||
# Settings → Updates: the "What's new" viewer reads the CHANGELOG.md that
|
||||
|
||||
@@ -168,16 +168,40 @@ def _detect_gpu() -> dict:
|
||||
return info
|
||||
|
||||
|
||||
def _probe_network(host: str = "huggingface.co", timeout: float = 2.0) -> bool:
|
||||
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 2.0) -> bool:
|
||||
"""Tiny TCP connect test."""
|
||||
import socket
|
||||
try:
|
||||
with socket.create_connection((host, 443), timeout=timeout):
|
||||
with socket.create_connection((host, port), timeout=timeout):
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _hf_endpoint_host() -> tuple[str, int]:
|
||||
"""Host/port of the Hugging Face endpoint actually in effect.
|
||||
|
||||
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
|
||||
point HF_ENDPOINT at a mirror via Settings → Models → Hugging Face
|
||||
mirror. Probing hardcoded huggingface.co would fail them even when their
|
||||
configured mirror works fine.
|
||||
"""
|
||||
try:
|
||||
from core.failure import configured_hf_mirror
|
||||
mirror = configured_hf_mirror()
|
||||
except Exception:
|
||||
mirror = ""
|
||||
if mirror:
|
||||
try:
|
||||
from urllib.parse import urlsplit
|
||||
u = urlsplit(mirror)
|
||||
if u.hostname:
|
||||
return u.hostname, u.port or (80 if u.scheme == "http" else 443)
|
||||
except Exception:
|
||||
pass
|
||||
return "huggingface.co", 443
|
||||
|
||||
|
||||
def _ram_gb() -> float:
|
||||
try:
|
||||
import psutil
|
||||
@@ -400,15 +424,49 @@ def preflight():
|
||||
"status": r_status, "detail": r_detail, "fix": r_fix,
|
||||
})
|
||||
|
||||
# ── Network
|
||||
net_ok = _probe_network()
|
||||
# ── Network — probes the HF endpoint actually in effect (mirror-aware),
|
||||
# and a dead network is a WARNING, not a blocker. The app is local-first:
|
||||
# already-downloaded models work offline, and a hard fail here dead-ends
|
||||
# restricted-network users (e.g. China, where huggingface.co is blocked)
|
||||
# on the very first screen — before they can reach the mirror setting
|
||||
# that fixes it. Model downloads surface their own actionable errors.
|
||||
net_host, net_port = _hf_endpoint_host()
|
||||
net_ok = _probe_network(net_host, net_port)
|
||||
mirror_reachable = False
|
||||
if not net_ok and net_host == "huggingface.co":
|
||||
# Official endpoint blocked — if the community mirror is reachable,
|
||||
# tell the user exactly which switch unblocks them.
|
||||
mirror_reachable = _probe_network("hf-mirror.com")
|
||||
if net_ok:
|
||||
net_fix = None
|
||||
elif mirror_reachable:
|
||||
net_fix = (
|
||||
"huggingface.co is blocked on this network, but the hf-mirror.com "
|
||||
"community mirror is reachable — apply it below and re-check. "
|
||||
"Model downloads will use the mirror immediately."
|
||||
)
|
||||
elif net_host != "huggingface.co":
|
||||
net_fix = (
|
||||
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
|
||||
"— it may be down or blocked. Pick another mirror or the official "
|
||||
"endpoint below, or continue offline: models already downloaded "
|
||||
"keep working."
|
||||
)
|
||||
else:
|
||||
net_fix = (
|
||||
"Check internet connection, VPN, or corporate firewall whitelist "
|
||||
"for huggingface.co. You can continue — models already downloaded "
|
||||
"keep working offline; new downloads need a connection or a "
|
||||
"mirror (configurable below)."
|
||||
)
|
||||
checks.append({
|
||||
"id": "network", "label": "Network (huggingface.co)",
|
||||
"status": "pass" if net_ok else "fail",
|
||||
"detail": "Reachable" if net_ok else "Unreachable on port 443",
|
||||
"fix": None if net_ok else
|
||||
"Check internet connection, VPN, or corporate firewall "
|
||||
"whitelist for huggingface.co.",
|
||||
"id": "network", "label": f"Network ({net_host})",
|
||||
"status": "pass" if net_ok else "warn",
|
||||
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
|
||||
"fix": net_fix,
|
||||
# Frontend affordance hint: the wizard offers the mirror quick-pick
|
||||
# when the endpoint is unreachable (PreflightCheck allows extras).
|
||||
"mirror_reachable": mirror_reachable,
|
||||
})
|
||||
|
||||
# Aggregate
|
||||
|
||||
@@ -42,6 +42,7 @@ _HINTS: dict[str, str] = {
|
||||
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
|
||||
"OS_INVALID_ARGUMENT": "The OS rejected a file operation (Errno 22 / invalid argument) — in the transcribe path this is the temporary WAV write before ASR. It's almost always the temp directory: missing, read-only, on a full or removed drive, or blocked by antivirus. Check that your system TEMP/TMP folder exists and is writable and the drive has free space (add an OmniVoice antivirus exclusion if you use one), then retry.",
|
||||
"SOCKS_PROXY_SUPPORT_MISSING": "A SOCKS proxy is configured in your environment (ALL_PROXY/HTTPS_PROXY=socks5://…) and the backend's HTTP client is missing SOCKS support. Newer OmniVoice builds ship SOCKS support (the socksio package) — update the app. If you still see this, unset ALL_PROXY/HTTPS_PROXY for OmniVoice, or run `uv pip install 'httpx[socks]'` in the backend venv, then restart.",
|
||||
"SSL_HANDSHAKE_FAILURE": "A corporate or antivirus proxy is intercepting HTTPS traffic and re-signing certificates with its own CA — your OS trusts that CA, but Python's bundled certifi CA list doesn't, so the TLS handshake fails even though the connection reached the server. Newer OmniVoice builds trust the OS certificate store at startup (the truststore package), which should already fix this — update the app and retry. If you still see this, add an HTTPS-scanning exclusion for OmniVoice/Python in your antivirus, or ask IT for the proxy's CA bundle and set SSL_CERT_FILE to it, then restart.",
|
||||
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
|
||||
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
|
||||
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
|
||||
@@ -177,6 +178,7 @@ def append_hf_mirror_hint(text: str) -> str:
|
||||
# hint on a model-load timeout that leaks through the 500 handler.
|
||||
_CONTEXT_FREE_HINT_CLASSES = frozenset({
|
||||
"SOCKS_PROXY_SUPPORT_MISSING",
|
||||
"SSL_HANDSHAKE_FAILURE",
|
||||
})
|
||||
|
||||
|
||||
@@ -257,6 +259,19 @@ def classify(reason: str) -> str:
|
||||
# a message that also carries HF wording still names this class.
|
||||
if "socks proxy" in low or "socksio" in low:
|
||||
return "SOCKS_PROXY_SUPPORT_MISSING"
|
||||
# #976: a TLS handshake failing AFTER the TCP connection succeeds — the
|
||||
# signature of a corporate/antivirus proxy that TLS-inspects traffic and
|
||||
# re-signs certificates with a CA the OS trusts but Python's bundled
|
||||
# certifi list doesn't (a different failure mode from #984's TCP-level
|
||||
# "can't reach the host at all"). Requires "ssl" plus a handshake/cert-
|
||||
# verify marker so a generic connection error isn't mislabelled.
|
||||
if "ssl" in low and (
|
||||
"handshake" in low
|
||||
or "certificate verify failed" in low
|
||||
or "sslv3_alert" in low
|
||||
or "sslcertverificationerror" in low
|
||||
):
|
||||
return "SSL_HANDSHAKE_FAILURE"
|
||||
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
|
||||
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
|
||||
):
|
||||
|
||||
@@ -24,7 +24,7 @@ from pathlib import Path
|
||||
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
|
||||
# release.yml's version-bump job, so it stays equal to
|
||||
# pyproject/tauri.conf/Cargo/package.json.
|
||||
_FALLBACK_VERSION = "0.3.11"
|
||||
_FALLBACK_VERSION = "0.3.14"
|
||||
|
||||
|
||||
def _fallback_version() -> str:
|
||||
|
||||
@@ -70,6 +70,7 @@ class Supertonic3Backend(SubprocessBackend):
|
||||
id = "supertonic3"
|
||||
display_name = "Supertonic-3 (31 langs, CPU ONNX, 7 preset voices, OpenRAIL-M)"
|
||||
supports_voice_design = False # preset voices only
|
||||
supports_cloning = False # preset voices only; generate() never reads ref_audio
|
||||
# TTS-04: honest hardware reporting. Supertonic-3 has no CUDA / MPS
|
||||
# path in the SDK ‑‑ ONNX Runtime CPU EP only.
|
||||
gpu_compat: tuple[str, ...] = ("cpu",)
|
||||
|
||||
+79
-8
@@ -139,6 +139,24 @@ os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
|
||||
os.environ.setdefault("HF_HUB_ETAG_TIMEOUT", "15")
|
||||
os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "30")
|
||||
|
||||
# ── OS trust store for TLS (#976) ───────────────────────────────────────────
|
||||
# Users behind a corporate/antivirus proxy that TLS-inspects HTTPS traffic get
|
||||
# a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure"
|
||||
# on every model install — the TCP connection succeeds (a different failure
|
||||
# mode from #984's TCP-level blocked-host case), but the proxy re-signs the
|
||||
# certificate with its own root CA, which the OS trusts (Windows CryptoAPI/
|
||||
# SChannel) and Python's bundled `certifi` CA list does not. `inject_into_ssl`
|
||||
# patches `ssl.SSLContext` process-wide to verify against the OS trust store
|
||||
# instead, which is the actual fix (not just a nicer error message). Must run
|
||||
# here — at MODULE level, before huggingface_hub/requests/httpx do any network
|
||||
# I/O — not inside lifespan(), which runs too late. Not platform-gated: it's a
|
||||
# correctness improvement everywhere. Best-effort: never block startup.
|
||||
try:
|
||||
import truststore
|
||||
|
||||
truststore.inject_into_ssl()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Prevent torchaudio from lazy-importing torchcodec (broken on some installs).
|
||||
# Proper fix = exclude torchcodec in pyproject.toml; this is a belt-and-braces guard.
|
||||
@@ -486,6 +504,35 @@ async def _start_mcp_session_manager(session_manager, *, timeout: float):
|
||||
return task, stop, mounted
|
||||
|
||||
|
||||
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
|
||||
"""Cancel each background task and give it a bounded chance to actually
|
||||
finish before shutdown proceeds — ``None`` entries are skipped (a task
|
||||
that's conditionally created, e.g. ``capture_preload_task``, may not
|
||||
exist).
|
||||
|
||||
``task.cancel()`` alone is not enough for a task awaiting
|
||||
``run_in_executor()``: once the underlying OS thread is inside blocking
|
||||
native/import work, cancellation can't stop it, so cancel-and-move-on lets
|
||||
shutdown finish while that thread is still running — invisible to
|
||||
asyncio, but very much alive when the interpreter starts tearing down
|
||||
module state under it (#1000 class). Awaiting with a bound (instead of
|
||||
just cancelling) gives an early-stage task a real chance to exit cleanly
|
||||
first; a task that's genuinely still deep in blocking work times out here
|
||||
same as before, and the caller's own GPU-pool reset handles that case.
|
||||
"""
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
t.cancel()
|
||||
for t in tasks:
|
||||
if t is None:
|
||||
continue
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=timeout)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
|
||||
@@ -560,6 +607,7 @@ async def lifespan(app: FastAPI):
|
||||
# lean and the first dictation is instant instead of a cold model load.
|
||||
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
|
||||
# under 4 GB free RAM (checked at warm time, not boot time).
|
||||
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
|
||||
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
|
||||
async def _preload_capture_asr():
|
||||
await asyncio.sleep(_capture_preload_delay_s())
|
||||
@@ -628,14 +676,33 @@ async def lifespan(app: FastAPI):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
idle_task.cancel()
|
||||
worker_task.cancel()
|
||||
# Wait for tasks to finish their current iteration
|
||||
for t in (idle_task, worker_task):
|
||||
try:
|
||||
await asyncio.wait_for(t, timeout=3.0)
|
||||
except (asyncio.CancelledError, asyncio.TimeoutError):
|
||||
pass
|
||||
# preload_task/capture_preload_task matter most here (#1000 class): a quit
|
||||
# mid-preload used to fall straight through to "Shutdown: done." while the
|
||||
# model load was still running on a GPU-pool thread — cancel() can't stop
|
||||
# a thread already inside blocking import/load work, so the process
|
||||
# reported a clean exit while that background thread was still mid-
|
||||
# `import transformers`, and got torn down by interpreter finalization
|
||||
# instead. That surfaced as a misleading "Could not import module
|
||||
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
|
||||
# not a real dependency problem. Awaiting here lets an early-stage load
|
||||
# (still importing, not yet mid weight-download) finish cleanly before we
|
||||
# report done; a load that's genuinely deep into a multi-GB download still
|
||||
# times out — _reset_gpu_pool() below abandons it either way.
|
||||
#
|
||||
# 20s, not the original 3s (code-review finding post-merge): a cold
|
||||
# transformers import alone can take longer than 3s on a slow disk or a
|
||||
# first-ever launch, so the original bound left a real residual window —
|
||||
# cancellation detaches the asyncio task, but the underlying OS thread
|
||||
# keeps running past it, and shutdown could still report "done" while
|
||||
# that thread was alive. Python cannot forcibly kill a running thread, so
|
||||
# no finite bound eliminates this outright — 20s just shrinks the window
|
||||
# from "any preload" to "an unusually slow cold-import," which is the
|
||||
# practical ceiling before a longer shutdown itself becomes the
|
||||
# complaint. A thread that's still running past 20s was never going to
|
||||
# finish in a shutdown-appropriate timeframe regardless.
|
||||
await _cancel_and_await_tasks(
|
||||
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
|
||||
)
|
||||
# Unload the model and free GPU memory
|
||||
try:
|
||||
import services.model_manager as mm
|
||||
@@ -643,6 +710,10 @@ async def lifespan(app: FastAPI):
|
||||
mm.model = None
|
||||
logger.info("Shutdown: model unloaded.")
|
||||
mm.free_vram()
|
||||
# Abandon a still-running preload's GPU-pool thread (Python can't kill
|
||||
# a thread mid blocking call) so it can't outlive this shutdown block
|
||||
# holding a reference into module state that's about to be torn down.
|
||||
mm._reset_gpu_pool()
|
||||
except Exception:
|
||||
pass
|
||||
# Run GC to release any remaining references
|
||||
|
||||
@@ -29,6 +29,7 @@ import os
|
||||
import re
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger("omnivoice.asr")
|
||||
|
||||
@@ -1634,6 +1635,161 @@ class FunASRBackend(ASRBackend):
|
||||
pass
|
||||
|
||||
|
||||
# ── OpenAI-compatible remote transcription (#877 — Qwen3-ASR / FunASR / any
|
||||
# compatible server, today, without waiting on transformers to catch up) ──
|
||||
#
|
||||
# transformers doesn't yet ship a stable Qwen3-ASR integration (issue #877),
|
||||
# but a self-hosted Qwen3-ASR/FunASR/SenseVoice server exposing an
|
||||
# OpenAI-compatible `POST /v1/audio/transcriptions` endpoint — or OpenAI's own
|
||||
# Whisper API — is usable right now. This backend is a pure network client:
|
||||
# no model runs locally, so it needs no install and claims no GPU.
|
||||
#
|
||||
# Settings mirror the LLM-providers convention exactly (services/
|
||||
# llm_providers.py): base_url/model are plain settings_store text rows; the
|
||||
# API key is Fernet-encrypted via settings_store.set_secret/get_secret — never
|
||||
# a .env row, never echoed back to the client. Optional: some self-hosted
|
||||
# servers (vLLM, LM Studio-style) don't check the key at all.
|
||||
|
||||
_ASR_OPENAI_COMPAT_BASE_URL_KEY = "asr.openai_compat.base_url"
|
||||
_ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_base_url() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_BASE_URL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_BASE_URL_KEY)
|
||||
or ""
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_model() -> str:
|
||||
from services import settings_store
|
||||
return (
|
||||
os.environ.get("ASR_OPENAI_COMPAT_MODEL")
|
||||
or settings_store.get_text(_ASR_OPENAI_COMPAT_MODEL_KEY)
|
||||
or "whisper-1"
|
||||
)
|
||||
|
||||
|
||||
def resolve_openai_compat_asr_api_key() -> Optional[str]:
|
||||
"""Env → encrypted stored key → None. Unlike LLM providers, no 'local'
|
||||
sentinel: many self-hosted transcription servers accept an empty/omitted
|
||||
Authorization header outright, so the OpenAI SDK is constructed with
|
||||
``api_key="not-needed"`` (a non-empty placeholder the SDK requires) when
|
||||
this returns None, rather than treating a keyless server as unconfigured.
|
||||
"""
|
||||
from services import settings_store
|
||||
return os.environ.get("ASR_OPENAI_COMPAT_API_KEY") or settings_store.get_secret(
|
||||
_ASR_OPENAI_COMPAT_SECRET_NAME
|
||||
)
|
||||
|
||||
|
||||
def openai_compat_asr_has_key() -> bool:
|
||||
"""Whether a key is configured, without ever decrypting it — mirrors
|
||||
llm_providers.has_key()'s no-plaintext-round-trip contract."""
|
||||
from services import settings_store
|
||||
if os.environ.get("ASR_OPENAI_COMPAT_API_KEY"):
|
||||
return True
|
||||
return _ASR_OPENAI_COMPAT_SECRET_NAME in settings_store.list_secret_names()
|
||||
|
||||
|
||||
class OpenAICompatASRBackend(ASRBackend):
|
||||
"""Remote transcription via any OpenAI-compatible server.
|
||||
|
||||
Adapts whatever the server returns into this module's expected shape.
|
||||
Prefers `response_format="verbose_json"` for real per-segment timestamps
|
||||
(OpenAI's own API and most compatible servers support it); falls back to
|
||||
plain text with rough single-segment bounds — mirroring
|
||||
MoonshineASRBackend's degraded shape — for minimal servers that reject it.
|
||||
"""
|
||||
id = "openai-compat-asr"
|
||||
display_name = "OpenAI-compatible (remote server)"
|
||||
gpu_compat = ("cpu",) # network client only — no local compute
|
||||
|
||||
def __init__(self):
|
||||
self._base_url = resolve_openai_compat_asr_base_url()
|
||||
self._model = resolve_openai_compat_asr_model()
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> tuple[bool, str]:
|
||||
if not resolve_openai_compat_asr_base_url():
|
||||
return False, "Configure a server endpoint in Settings → Engines"
|
||||
try:
|
||||
import openai # noqa: F401
|
||||
except ImportError:
|
||||
return False, "openai package not installed. Install with: uv pip install openai"
|
||||
return True, "ready"
|
||||
|
||||
def _client(self):
|
||||
from openai import OpenAI
|
||||
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
|
||||
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
|
||||
# rate-limited/slow server retrying inside the SDK would blow past
|
||||
# whatever bounded timeout the caller (dub transcribe, dictation)
|
||||
# expects from a single call.
|
||||
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
|
||||
|
||||
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
|
||||
logger.info(
|
||||
"OpenAI-compat ASR transcribing %s (base_url=%s, model=%s)",
|
||||
audio_path, self._base_url, self._model,
|
||||
)
|
||||
client = self._client()
|
||||
try:
|
||||
with open(audio_path, "rb") as f:
|
||||
try:
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="verbose_json",
|
||||
)
|
||||
except Exception:
|
||||
# Minimal/older compatible servers reject verbose_json
|
||||
# outright — retry plain before treating it as a real
|
||||
# failure. Re-open: the SDK may have partially consumed
|
||||
# the file handle on the first attempt.
|
||||
f.seek(0)
|
||||
resp = client.audio.transcriptions.create(
|
||||
file=f, model=self._model, response_format="json",
|
||||
)
|
||||
except Exception as exc:
|
||||
# Never leak a raw SDK/httpx exception object (auth headers,
|
||||
# connection internals) straight into a user-facing message —
|
||||
# same convention as generation.py's _safe_exc_text (#977 class).
|
||||
raise RuntimeError(
|
||||
f"OpenAI-compatible ASR server at {self._base_url!r} failed: "
|
||||
f"{type(exc).__name__}: {exc}"
|
||||
) from exc
|
||||
return self._adapt_response(resp)
|
||||
|
||||
@staticmethod
|
||||
def _adapt_response(resp) -> dict:
|
||||
segments_out = []
|
||||
# verbose_json: resp.segments is a list of objects with start/end/text.
|
||||
raw_segments = getattr(resp, "segments", None)
|
||||
if raw_segments:
|
||||
for seg in raw_segments:
|
||||
seg_dict = seg if isinstance(seg, dict) else seg.model_dump()
|
||||
segments_out.append({
|
||||
"text": (seg_dict.get("text") or "").strip(),
|
||||
"start": seg_dict.get("start", 0.0),
|
||||
"end": seg_dict.get("end", 0.0),
|
||||
"words": [], # word-level timing isn't part of this API
|
||||
})
|
||||
else:
|
||||
# Plain text response (json/text format) — single-segment shape,
|
||||
# matching MoonshineASRBackend's degraded fallback exactly.
|
||||
text = (getattr(resp, "text", None) or "").strip()
|
||||
if text:
|
||||
segments_out.append({"text": text, "start": 0.0, "end": None, "words": []})
|
||||
chunks = [
|
||||
{"text": seg["text"], "timestamp": (seg["start"], seg["end"])}
|
||||
for seg in segments_out
|
||||
]
|
||||
language = getattr(resp, "language", None) or "en"
|
||||
return {"chunks": chunks, "segments": segments_out, "language": language}
|
||||
|
||||
|
||||
def _isolated_faster_whisper():
|
||||
"""Lazy import so the subprocess_asr → subprocess_backend chain isn't
|
||||
pulled in at registry definition time."""
|
||||
@@ -1689,6 +1845,7 @@ _REGISTRY: dict[str, type[ASRBackend]] = _LazyASRRegistry({
|
||||
"moonshine": MoonshineASRBackend,
|
||||
"funasr": FunASRBackend,
|
||||
"sherpa-onnx-asr": SherpaDictationBackend,
|
||||
"openai-compat-asr": OpenAICompatASRBackend,
|
||||
# "faster-whisper-isolated": resolved lazily (crash-isolated subprocess).
|
||||
})
|
||||
|
||||
@@ -1700,10 +1857,26 @@ _INSTALL_HINTS: dict[str, str] = {
|
||||
"faster-whisper": "pip install faster-whisper (CTranslate2; cross-platform, CUDA or CPU)",
|
||||
"mlx-whisper": "pip install mlx-whisper (Apple Silicon only)",
|
||||
"pytorch-whisper": "Bundled with transformers — no extra install (CUDA/MPS/CPU)",
|
||||
"nemo-parakeet": "pip install nemo_toolkit[asr] (NVIDIA Parakeet; CUDA or CPU)",
|
||||
"nemo-parakeet": (
|
||||
"No safe install path in this app yet — nemo_toolkit's ASR extras pin "
|
||||
"transformers>=4.57,<4.58, which conflicts with OmniVoice's own "
|
||||
"transformers>=5.3 requirement and WILL break the backend "
|
||||
"(ImportError on startup) if installed into this shared venv. Do NOT "
|
||||
"install nemo_toolkit here. If you want to try Parakeet TDT, set it "
|
||||
"up in a separate/dedicated Python environment — not the one "
|
||||
"OmniVoice manages; in-app isolation for this engine is tracked "
|
||||
"separately."
|
||||
),
|
||||
"moonshine": "pip install useful-moonshine (edge/CPU-optimized ASR)",
|
||||
"funasr": "pip install funasr (SenseVoiceSmall + FSMN-VAD; CUDA or CPU)",
|
||||
"sherpa-onnx-asr": "uv add sherpa-onnx (ONNX live dictation; CPU, cross-platform)",
|
||||
"openai-compat-asr": (
|
||||
"No install needed — configure a server endpoint in Settings → "
|
||||
"Engines. Points OmniVoice at any OpenAI-compatible transcription "
|
||||
"server (a self-hosted Qwen3-ASR/FunASR/SenseVoice server, OpenAI's "
|
||||
"own Whisper API, or similar) — a path to Qwen3-ASR today, without "
|
||||
"waiting on a direct transformers integration."
|
||||
),
|
||||
"faster-whisper-isolated": (
|
||||
"No extra install (reuses faster-whisper). Escape hatch for hanging "
|
||||
"transcribes: runs ASR in a separate process that can be force-killed "
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
"""
|
||||
Audio DSP pipeline — broadcast-grade mastering + configurable effects chain.
|
||||
|
||||
The default `apply_mastering()` is the same chain shipped since v0.1.0
|
||||
(highpass + compressor + light reverb). The new `apply_effects_chain()`
|
||||
lets callers build custom pipelines from a list of named effects.
|
||||
`apply_mastering()` is the shared pre-stage that runs before the user's
|
||||
effect preset: highpass + gentle compression only (see `MASTERING_CHAIN`).
|
||||
Reverb is deliberately NOT part of it — it is preset-declared only (e.g.
|
||||
cinematic, warm); a hidden reverb here used to bake echo into every non-raw
|
||||
synthesis, which field reports flagged. `apply_effects_chain()` lets callers
|
||||
build custom pipelines from a list of named effects.
|
||||
|
||||
All effects use Spotify's `pedalboard` library. When pedalboard isn't
|
||||
installed, every function degrades gracefully (returns audio unmodified).
|
||||
@@ -97,24 +100,26 @@ def get_effect_chain(preset_id: str) -> list[dict]:
|
||||
|
||||
# ── Core DSP functions ──────────────────────────────────────────────────
|
||||
|
||||
#: Shared pre-preset mastering stage: highpass + gentle compression ONLY.
|
||||
#: Reverb must never live here — a hidden Reverb in this chain baked echo
|
||||
#: into every non-raw synthesis regardless of the chosen preset (field
|
||||
#: reports of echoey voices; the podcast preset even promises "no reverb").
|
||||
#: Reverb is preset-declared only (see EFFECT_PRESETS: cinematic, warm).
|
||||
MASTERING_CHAIN = [
|
||||
{"type": "highpass", "cutoff_hz": 60},
|
||||
{"type": "compressor", "threshold_db": -15, "ratio": 1.5, "attack_ms": 2.0, "release_ms": 100},
|
||||
]
|
||||
|
||||
|
||||
def apply_mastering(audio_tensor, sample_rate=24000):
|
||||
"""Applies professional Broadcast-grade DSP (EQ, Compressor, light Reverb) to the clone voice."""
|
||||
"""Applies the broadcast pre-stage (highpass + gentle compression) to the clone voice.
|
||||
|
||||
Reverb is intentionally absent — only user-chosen effect presets declare
|
||||
it. Degrades gracefully: pedalboard missing or any DSP error returns the
|
||||
input unmodified.
|
||||
"""
|
||||
try:
|
||||
from pedalboard import Pedalboard, Compressor, Reverb, HighpassFilter
|
||||
import numpy as np
|
||||
board = Pedalboard([
|
||||
HighpassFilter(cutoff_frequency_hz=60),
|
||||
Compressor(threshold_db=-15, ratio=1.5, attack_ms=2.0, release_ms=100),
|
||||
Reverb(room_size=0.10, wet_level=0.08, dry_level=0.95)
|
||||
])
|
||||
audio_np = audio_tensor.cpu().numpy()
|
||||
if audio_np.ndim == 1:
|
||||
audio_np = audio_np[np.newaxis, :]
|
||||
effected = board(audio_np, sample_rate, reset=False)
|
||||
return torch.from_numpy(effected).to(audio_tensor.device)
|
||||
except ImportError:
|
||||
return audio_tensor # Fail gracefully if pedalboard isn't installed
|
||||
return apply_effects_chain(audio_tensor, sample_rate, MASTERING_CHAIN)
|
||||
except Exception as e:
|
||||
logger.warning("Mastering DSP Error: %s", e)
|
||||
return audio_tensor
|
||||
|
||||
@@ -45,11 +45,33 @@ def _asr_device() -> str:
|
||||
return "cpu"
|
||||
|
||||
|
||||
def _active_tts_id() -> Optional[str]:
|
||||
"""Configured TTS engine id, or None if it can't be resolved. Attribution
|
||||
is advisory — a prefs/import hiccup must never break /model/loaded."""
|
||||
try:
|
||||
from services.tts_backend import active_backend_id
|
||||
return active_backend_id()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _tts_attribution(engine_id: str, active: Optional[str]) -> dict:
|
||||
"""Per-entry engine attribution for TTS-family models. A model can stay
|
||||
resident in VRAM after the user switches engines (freed only by unload/
|
||||
idle-evict), so the panel needs to know which entry synthesis actually
|
||||
routes to. ``is_active_engine`` is None when the active id is unknown."""
|
||||
return {
|
||||
"engine_id": engine_id,
|
||||
"is_active_engine": (engine_id == active) if active is not None else None,
|
||||
}
|
||||
|
||||
|
||||
def list_loaded() -> dict:
|
||||
"""Enumerate every currently-loaded model. Shape: ``{"models": [...],
|
||||
"count": n}`` with per-model id/name/checkpoint/device/vram_mb/unloadable
|
||||
(+ optional ``note``)."""
|
||||
models: list[dict] = []
|
||||
active_tts = _active_tts_id()
|
||||
|
||||
# 1. In-process TTS model (OmniVoice)
|
||||
if mm.model is not None:
|
||||
@@ -64,6 +86,7 @@ def list_loaded() -> dict:
|
||||
"device": device,
|
||||
"vram_mb": round(_tts_vram_mb(), 1),
|
||||
"unloadable": True,
|
||||
**_tts_attribution("omnivoice", active_tts),
|
||||
})
|
||||
|
||||
# 2. ASR (WhisperX) — co-loaded with and released alongside the TTS model.
|
||||
@@ -105,6 +128,7 @@ def list_loaded() -> dict:
|
||||
"device": get_best_device(),
|
||||
"vram_mb": round(float(s.get("vram_mb") or 0), 1),
|
||||
"unloadable": True,
|
||||
**_tts_attribution(s["id"], active_tts),
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -277,9 +277,11 @@ def _timeout_guidance(what: str, timeout: float) -> str:
|
||||
)
|
||||
return common + (
|
||||
"most often the GPU is VRAM-starved (a resident model and this job "
|
||||
"contend for memory). For a durable fix try shorter text, a lighter "
|
||||
"engine, or set the engine to CPU in Settings → Models. (Raise "
|
||||
"OMNIVOICE_GENERATE_TIMEOUT_S for very long single generations.)"
|
||||
"contend for memory). For a durable fix, Flush caches / Unload the "
|
||||
"resident model (top toolbar or Settings → Models) before retrying, "
|
||||
"try shorter text, a lighter engine, or set the engine to CPU in "
|
||||
"Settings → Models. (Raise OMNIVOICE_GENERATE_TIMEOUT_S for very "
|
||||
"long single generations.)"
|
||||
)
|
||||
|
||||
|
||||
@@ -955,7 +957,13 @@ def _load_model_sync():
|
||||
except Exception: # never let failure-formatting mask the real error
|
||||
err_msg = str(exc)
|
||||
_set_loading("error", "Model loading failed", error=err_msg)
|
||||
logger.error("Model loading failed: %s", str(exc))
|
||||
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
|
||||
# to an inner import (including one interrupted by process teardown)
|
||||
# in a generic "Could not import module X. Are this object's
|
||||
# requirements defined correctly?" — logging only str(exc) discarded
|
||||
# the real cause in __cause__/__context__ and made a shutdown race
|
||||
# look like a broken install. exc_info surfaces the full chain.
|
||||
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
|
||||
raise
|
||||
finally:
|
||||
unregister_listener(lid)
|
||||
@@ -1087,7 +1095,11 @@ async def preload_model():
|
||||
model = await _load_model_with_timeout()
|
||||
logger.info("Preload complete — model ready.")
|
||||
except Exception as e:
|
||||
logger.warning("Model preload failed (non-fatal): %s", e)
|
||||
# See the matching exc_info note on the _load_model_sync handler above
|
||||
# (#1000 class) — the full chain, not just str(e), is what actually
|
||||
# distinguishes a real dependency problem from a shutdown-interrupted
|
||||
# import.
|
||||
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
|
||||
|
||||
def get_model_status():
|
||||
is_loaded = model is not None
|
||||
|
||||
@@ -223,6 +223,60 @@ def extract_segment_refs(
|
||||
return out
|
||||
|
||||
|
||||
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
|
||||
"""Re-transcribe a written reference clip and return that transcript.
|
||||
|
||||
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
|
||||
the ASR segment's OWN text field, on the assumption that the segment's
|
||||
timestamps and its transcribed text agree. They routinely don't — Whisper
|
||||
(and friends) frequently drift on segment boundaries: a trailing word
|
||||
audible in `[start, end]` but missing from `text`, or vice versa. When the
|
||||
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
|
||||
down and the clone can speak the mismatched reference text itself instead
|
||||
of the target-language text it was given to synthesize (issue #1004).
|
||||
|
||||
Re-transcribing the *actual written clip* guarantees the pair matches by
|
||||
construction — the model doesn't care whether the original ASR text was
|
||||
right, only that ref_text is what's really in ref_audio. `asr_backend` is
|
||||
the caller's already-loaded active backend (duck-typed:
|
||||
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
|
||||
`{"text": ...}`); the model is already warm, so this costs one more short
|
||||
transcribe call, not a fresh load. Falls back to `fallback_text` — never
|
||||
raises — so a re-transcribe failure is a strict no-op, never a regression
|
||||
from the original (matching) behavior.
|
||||
"""
|
||||
if asr_backend is None:
|
||||
return fallback_text
|
||||
try:
|
||||
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
|
||||
text = " ".join(
|
||||
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
|
||||
).strip()
|
||||
return text or fallback_text
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
|
||||
ref_audio_path, e,
|
||||
)
|
||||
return fallback_text
|
||||
|
||||
|
||||
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
|
||||
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
|
||||
|
||||
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
|
||||
or per-segment `seg_clones` from `extract_segment_refs`) into the single
|
||||
executor round-trip the caller submits to the GPU pool, rather than one
|
||||
dispatch per reference. Mutates and returns `clones` for a convenient
|
||||
call-and-reassign at the call site.
|
||||
"""
|
||||
for entry in clones.values():
|
||||
entry["ref_text"] = refine_ref_text(
|
||||
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
|
||||
)
|
||||
return clones
|
||||
|
||||
|
||||
# ── Internals ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -143,12 +143,19 @@ class TTSBackend(ABC):
|
||||
supports_voice_design: bool = False
|
||||
|
||||
#: Whether this engine already emits mastered, studio-grade audio and should
|
||||
#: therefore skip the shared apply_mastering() chain (Compressor + Reverb,
|
||||
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
|
||||
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
|
||||
#: 48 kHz) set this True so their clean output isn't pumped/reverbed. Loudness
|
||||
#: 48 kHz) set this True so their clean output isn't pumped. Loudness
|
||||
#: normalisation is applied regardless — it's a benign peak scale.
|
||||
applies_own_mastering: bool = False
|
||||
|
||||
#: Whether this engine can clone an arbitrary voice from reference audio
|
||||
#: (`ref_audio=`), as opposed to only offering a fixed set of preset
|
||||
#: voices. Default True — most engines clone. Dub/batch gate on this
|
||||
#: (issue #312 class) before committing to a job that needs it, instead
|
||||
#: of silently falling back to OmniVoice or mis-cloning per segment.
|
||||
supports_cloning: bool = True
|
||||
|
||||
#: GPU/accelerator targets the engine can run on. Surfaced via the
|
||||
#: Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) so users can
|
||||
#: tell at a glance which engines will use their hardware. Defaults to
|
||||
@@ -603,6 +610,7 @@ class KittenTTSBackend(TTSBackend):
|
||||
display_name = "KittenTTS (English, 8 preset voices, CPU realtime)"
|
||||
# KittenTTS ships as an ONNX CPU graph; no CUDA/MPS path today.
|
||||
gpu_compat = ("cpu",)
|
||||
supports_cloning = False # fixed preset voices only; ref_audio is ignored
|
||||
|
||||
PRESET_VOICES = [
|
||||
"expr-voice-2-m", "expr-voice-2-f",
|
||||
@@ -683,6 +691,54 @@ class KittenTTSBackend(TTSBackend):
|
||||
# ── MLX-Audio (mac-ARM engine multiplexer) ──────────────────────────────────
|
||||
|
||||
|
||||
# #977: Kokoro's own ALIASES table (mlx_audio.tts.models.kokoro.pipeline) only
|
||||
# recognizes ISO-ish tokens ("en", "es", "fr-fr", "pt-br", …) — it has no idea
|
||||
# what a full language name is. OmniVoice's `language` kwarg is normally a
|
||||
# full display name from frontend/src/languages.json (e.g. "Dutch",
|
||||
# "Spanish"), forwarded verbatim by the frontend and by
|
||||
# `OmniVoiceBackend.generate()`. Translate the subset Kokoro actually
|
||||
# supports to the ISO token its own ALIASES expects; a caller that already
|
||||
# passes an ISO code (or one of Kokoro's own single-letter codes) is
|
||||
# resolved unchanged by `resolve_kokoro_lang_code()` below.
|
||||
_KOKORO_ISO_BY_FULL_NAME = {
|
||||
"english": "en",
|
||||
"spanish": "es",
|
||||
"french": "fr",
|
||||
"hindi": "hi",
|
||||
"italian": "it",
|
||||
"portuguese": "pt",
|
||||
"japanese": "ja",
|
||||
"chinese": "zh",
|
||||
}
|
||||
|
||||
|
||||
def resolve_kokoro_lang_code(language: str) -> str:
|
||||
"""Map a full language name / ISO code to Kokoro's single-letter
|
||||
`lang_code`, against the AUTHORITATIVE table read from the installed
|
||||
mlx-audio package (never a hardcoded guess — the vendored table is the
|
||||
only source of truth and can change across mlx-audio versions).
|
||||
|
||||
Raises ``ValueError`` for anything Kokoro doesn't support, naming what
|
||||
it *does* support — instead of forwarding a bogus code into Kokoro's
|
||||
`assert lang_code in LANG_CODES`, which crashes with an unreadable
|
||||
``(lang_code, LANG_CODES)`` tuple/dict repr (#977).
|
||||
"""
|
||||
from mlx_audio.tts.models.kokoro.pipeline import ALIASES, LANG_CODES
|
||||
|
||||
key = language.strip().lower()
|
||||
iso = _KOKORO_ISO_BY_FULL_NAME.get(key, key)
|
||||
code = ALIASES.get(iso, iso)
|
||||
if code not in LANG_CODES:
|
||||
supported = ", ".join(sorted(name.title() for name in _KOKORO_ISO_BY_FULL_NAME))
|
||||
raise ValueError(
|
||||
f"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) doesn't "
|
||||
f"support language={language!r}. Kokoro supports: {supported}. "
|
||||
f"Pick one of those, leave language as 'Auto', or switch to a "
|
||||
f"multilingual engine (e.g. OmniVoice) for other languages."
|
||||
)
|
||||
return code
|
||||
|
||||
|
||||
class MLXAudioBackend(TTSBackend):
|
||||
"""Blaizzy/mlx-audio — Apple-Silicon-only wrapper over 14+ TTS engines
|
||||
(Kokoro, CSM, Dia, Qwen3-TTS, Chatterbox, MeloTTS, OuteTTS, Spark,
|
||||
@@ -721,7 +777,16 @@ class MLXAudioBackend(TTSBackend):
|
||||
def __init__(self):
|
||||
self._model = None
|
||||
self._sr = 24000 # most mlx-audio engines emit 24 kHz mono
|
||||
key = os.environ.get("OMNIVOICE_MLX_AUDIO_MODEL", self.DEFAULT_MODEL_KEY)
|
||||
# Env var > persisted UI choice (#981 — Settings → Engines curated-
|
||||
# model picker) > default. Mirrors active_backend_id()'s resolution
|
||||
# order exactly so power-users can still pin a model without the UI
|
||||
# silently undoing it.
|
||||
from core import prefs
|
||||
key = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=self.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
# Accept either a curated key ("kokoro") or a full HF repo id
|
||||
# ("mlx-community/Kokoro-82M-bf16") — flexibility for power users.
|
||||
self._model_id = self.CURATED_MODELS.get(key, key)
|
||||
@@ -759,6 +824,18 @@ class MLXAudioBackend(TTSBackend):
|
||||
# silently ignores languages it doesn't know.
|
||||
return ["multi"]
|
||||
|
||||
@property
|
||||
def supports_cloning(self) -> bool:
|
||||
"""Model-dependent — this adapter multiplexes 7+ curated models and
|
||||
only some take a reference-audio speaker prompt. `generate()` passes
|
||||
`ref_audio` through when present (~kwargs below) but silently retries
|
||||
without it on a TypeError, so an engine picked for cloning that's
|
||||
actually running Kokoro/Qwen3-TTS/etc. would clone nothing. Of the
|
||||
curated set, only CSM (`mlx-community/csm-1b-8bit`) is confirmed to
|
||||
accept a reference prompt — default False for every other model,
|
||||
curated or user-supplied, until positively confirmed."""
|
||||
return self._model_id == self.CURATED_MODELS.get("csm")
|
||||
|
||||
def _ensure_loaded(self):
|
||||
if self._model is not None:
|
||||
return
|
||||
@@ -772,6 +849,7 @@ class MLXAudioBackend(TTSBackend):
|
||||
|
||||
voice = kw.get("voice")
|
||||
ref_audio = kw.get("ref_audio")
|
||||
ref_text = kw.get("ref_text")
|
||||
language = kw.get("language")
|
||||
speed = float(kw.get("speed", 1.0))
|
||||
|
||||
@@ -782,7 +860,31 @@ class MLXAudioBackend(TTSBackend):
|
||||
kwargs = {"text": text, "speed": speed}
|
||||
if voice: kwargs["voice"] = voice
|
||||
if ref_audio: kwargs["ref_audio"] = ref_audio
|
||||
if language: kwargs["lang_code"] = language[:2].lower()
|
||||
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
|
||||
# AND ref_text are present — with ref_text missing, its context list
|
||||
# stays empty and indexing into it raises an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio,
|
||||
# instead of ever attempting the clone. Community-diagnosed (#1012).
|
||||
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
|
||||
if language and language != "Auto":
|
||||
if self._model_id == self.CURATED_MODELS.get("kokoro"):
|
||||
# Kokoro's vendored pipeline hard-asserts `lang_code` against
|
||||
# its own single-letter table — a bogus code crashes with an
|
||||
# unreadable AssertionError instead of failing cleanly
|
||||
# (#977). Resolve against the authoritative installed table
|
||||
# instead of guessing via `language[:2]`.
|
||||
kwargs["lang_code"] = resolve_kokoro_lang_code(language)
|
||||
else:
|
||||
# `lang_code`-as-2-letter-truncation is Kokoro's own
|
||||
# convention, not mlx-audio's in general — other curated
|
||||
# models either ignore unrecognized kwargs (CSM/Dia/OuteTTS
|
||||
# accept **kwargs and drop it) or expect something else
|
||||
# entirely (Qwen3-TTS's own docstring: "lang_code: Language
|
||||
# code (auto, chinese, english, etc.)" — a full name, not a
|
||||
# 2-letter code). Kokoro's strict validation doesn't apply to
|
||||
# them, so don't reject a language that's valid for whatever
|
||||
# model is actually active.
|
||||
kwargs["lang_code"] = language[:2].lower()
|
||||
|
||||
pieces = []
|
||||
try:
|
||||
@@ -1111,6 +1213,7 @@ class SherpaOnnxBackend(TTSBackend):
|
||||
# Sherpa-ONNX uses the onnxruntime providers — CPU is the universal
|
||||
# baseline; CUDA provider is available on Linux/Windows installs.
|
||||
gpu_compat = ("cuda", "cpu")
|
||||
supports_cloning = False # VITS speaker-id only; no ref_audio support
|
||||
|
||||
def __init__(self):
|
||||
self._tts = None
|
||||
@@ -1366,6 +1469,21 @@ _SETUP_SNIPPETS: dict[str, str] = {
|
||||
}
|
||||
|
||||
|
||||
# Short, readable labels for mlx-audio's curated models (#981) — surfaced in
|
||||
# the Settings → Engines model picker so users see more than a bare key.
|
||||
# Single-sourced here rather than on MLXAudioBackend.CURATED_MODELS itself so
|
||||
# the class dict stays a plain key → repo-id map (what __init__ needs).
|
||||
_MLX_AUDIO_MODEL_LABELS: dict[str, str] = {
|
||||
"kokoro": "Kokoro (default, fast)",
|
||||
"csm": "CSM (voice cloning)",
|
||||
"qwen3-tts": "Qwen3-TTS (voice design)",
|
||||
"dia": "Dia",
|
||||
"chatterbox": "Chatterbox",
|
||||
"melotts": "MeloTTS (lightweight)",
|
||||
"outetts": "OuteTTS",
|
||||
}
|
||||
|
||||
|
||||
def list_backends() -> list[dict]:
|
||||
"""Enumerate every registered backend with its availability state.
|
||||
|
||||
@@ -1449,6 +1567,22 @@ def list_backends() -> list[dict]:
|
||||
# effective_device / routing_status / routing_reason (scrubbed):
|
||||
**routing_fields(gpu_compat, caps),
|
||||
})
|
||||
# #981: mlx-audio multiplexes 7+ curated models behind one backend id
|
||||
# — surface the roster + the currently-active pick so Settings can
|
||||
# render a model picker instead of always defaulting to Kokoro.
|
||||
# mlx-audio ONLY; every other backend loads a single fixed model.
|
||||
if bid == "mlx-audio":
|
||||
from core import prefs
|
||||
active_model = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=cls.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
out[-1]["curated_models"] = [
|
||||
{"key": key, "label": _MLX_AUDIO_MODEL_LABELS.get(key, key), "repo_id": repo_id}
|
||||
for key, repo_id in cls.CURATED_MODELS.items()
|
||||
]
|
||||
out[-1]["active_model_id"] = active_model
|
||||
return out
|
||||
|
||||
|
||||
@@ -1458,6 +1592,29 @@ def get_backend_class(backend_id: str) -> type[TTSBackend]:
|
||||
return _REGISTRY[backend_id]
|
||||
|
||||
|
||||
def cloning_capable_engine_ids() -> list[str]:
|
||||
"""Engine ids that support reference-audio voice cloning — used to build
|
||||
an actionable error when the active engine can't (dub/batch gating).
|
||||
|
||||
Iterates the same registry ``list_backends()`` uses, via ``.items()`` so
|
||||
lazy entries resolve through ``_LazyRegistry``'s snapshot-safe iteration
|
||||
(see ``_LazyRegistry.__iter__``) exactly like every other registry scan
|
||||
in this module.
|
||||
|
||||
A class-level ``getattr`` on a *property* returns the descriptor object
|
||||
itself (always truthy) rather than its computed value — so a
|
||||
model-dependent adapter like ``MLXAudioBackend`` (only some of its 7+
|
||||
curated models can clone) would always show up here regardless of which
|
||||
model is actually configured. Excluded rather than falsely recommended:
|
||||
``isinstance(..., bool)`` is False for a descriptor, True for a plain
|
||||
class attribute.
|
||||
"""
|
||||
return [
|
||||
bid for bid, cls in _REGISTRY.items()
|
||||
if isinstance((v := getattr(cls, "supports_cloning", True)), bool) and v
|
||||
]
|
||||
|
||||
|
||||
def active_routing() -> dict | None:
|
||||
"""Routing verdict for the currently-active TTS engine, or ``None`` if it
|
||||
can't be determined (no engine / probe failure).
|
||||
@@ -1526,15 +1683,21 @@ def active_backend_id() -> str:
|
||||
# call the outgoing engine's unload() before switching.
|
||||
_active_instance: "TTSBackend | None" = None
|
||||
_active_instance_id: "str | None" = None
|
||||
# mlx-audio multiplexes 7+ curated models behind one backend id — a model-only
|
||||
# switch (same "mlx-audio" id, different curated model) must also invalidate
|
||||
# the cache, or picking a different model in Settings has no effect until the
|
||||
# app restarts (#981). Only meaningful when _active_instance_id == "mlx-audio".
|
||||
_active_mlx_model_key: "str | None" = None
|
||||
|
||||
|
||||
def reset_active_backend() -> None:
|
||||
"""Unload + clear the cached active backend. For app shutdown and tests.
|
||||
Idempotent and best-effort — a raising unload() never propagates."""
|
||||
global _active_instance, _active_instance_id
|
||||
global _active_instance, _active_instance_id, _active_mlx_model_key
|
||||
inst = _active_instance
|
||||
_active_instance = None
|
||||
_active_instance_id = None
|
||||
_active_mlx_model_key = None
|
||||
if inst is not None:
|
||||
try:
|
||||
inst.unload()
|
||||
@@ -1552,13 +1715,32 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
``model=`` (caller already holds a loaded model), we return a fresh view
|
||||
over the shared singleton rather than caching it — but a switch *away from*
|
||||
a different engine still triggers that engine's unload().
|
||||
|
||||
For mlx-audio specifically, the backend id alone doesn't capture *which*
|
||||
curated model is loaded (#981) — so we also track the resolved model key
|
||||
and treat a model-only change as a switch, reusing the exact same
|
||||
unload-and-reconstruct path as an id switch.
|
||||
"""
|
||||
global _active_instance, _active_instance_id
|
||||
global _active_instance, _active_instance_id, _active_mlx_model_key
|
||||
bid = active_backend_id()
|
||||
|
||||
# Switching engines: release the outgoing one first. Best-effort so a bad
|
||||
# unload() can never block the switch.
|
||||
if _active_instance is not None and _active_instance_id != bid:
|
||||
mlx_model_key = None
|
||||
if bid == "mlx-audio":
|
||||
from core import prefs
|
||||
mlx_model_key = prefs.resolve(
|
||||
"mlx_audio_model_id",
|
||||
env="OMNIVOICE_MLX_AUDIO_MODEL",
|
||||
default=MLXAudioBackend.DEFAULT_MODEL_KEY,
|
||||
)
|
||||
|
||||
# Switching engines (or, for mlx-audio, switching curated models): release
|
||||
# the outgoing one first. Best-effort so a bad unload() can never block
|
||||
# the switch.
|
||||
switching = _active_instance is not None and (
|
||||
_active_instance_id != bid
|
||||
or (bid == "mlx-audio" and mlx_model_key != _active_mlx_model_key)
|
||||
)
|
||||
if switching:
|
||||
try:
|
||||
_active_instance.unload()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
@@ -1566,6 +1748,7 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
type(_active_instance).__name__, exc)
|
||||
_active_instance = None
|
||||
_active_instance_id = None
|
||||
_active_mlx_model_key = None
|
||||
|
||||
cls = get_backend_class(bid)
|
||||
if cls is OmniVoiceBackend and model is not None:
|
||||
@@ -1577,9 +1760,82 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
|
||||
if _active_instance is None or _active_instance_id != bid:
|
||||
_active_instance = OmniVoiceBackend(model=model) if cls is OmniVoiceBackend else cls()
|
||||
_active_instance_id = bid
|
||||
_active_mlx_model_key = mlx_model_key
|
||||
return _active_instance
|
||||
|
||||
|
||||
# ── Shared generation-time engine resolution (issue #312 class) ───────────
|
||||
#
|
||||
# dub_generate.py and batch.py used to call services.model_manager.get_model()
|
||||
# directly, hardcoding OmniVoice regardless of the engine selected in
|
||||
# Settings → Engines — a SILENT fallback: pick VoxCPM2, dub anyway with
|
||||
# OmniVoice, no error. This is the single resolution path both routers now
|
||||
# call instead, mirroring generation.py's /generate resolution (engine id →
|
||||
# is_available() → routing gate) plus a voice-cloning capability gate that
|
||||
# /generate doesn't need (OmniVoice's native path always clones).
|
||||
|
||||
|
||||
async def resolve_generation_backend(
|
||||
*, require_cloning: bool = False, cloning_purpose: str = "dubbing",
|
||||
) -> TTSBackend:
|
||||
"""Resolve + validate the active TTS engine for a generation call.
|
||||
|
||||
Returns the live backend instance (:func:`get_active_tts_backend`) —
|
||||
cached, and properly unload()ed on an engine switch. Raises ``ValueError``
|
||||
with an actionable message (never silently falls back to OmniVoice) when:
|
||||
|
||||
* the configured engine id is unknown (bad env var / stale pref),
|
||||
* the engine reports itself unavailable (``is_available()``),
|
||||
* the engine needs an accelerator this host lacks and has no CPU path
|
||||
(``routing_status == "unavailable"``),
|
||||
* ``require_cloning`` is True and the resolved backend can't clone
|
||||
from reference audio (``supports_cloning`` False) — checked on the
|
||||
live *instance*, not the class, so a model-dependent adapter like
|
||||
MLX-Audio (Kokoro vs. CSM) is judged by what's actually loaded.
|
||||
"""
|
||||
engine_id = active_backend_id()
|
||||
try:
|
||||
backend_cls = get_backend_class(engine_id)
|
||||
except ValueError as e:
|
||||
raise ValueError(
|
||||
f"Active TTS engine '{engine_id}' is not a recognized backend ({e}). "
|
||||
"Check Settings → Engines or the OMNIVOICE_TTS_BACKEND env var."
|
||||
) from e
|
||||
|
||||
try:
|
||||
ok, msg = backend_cls.is_available()
|
||||
except Exception as exc: # noqa: BLE001 — surface as an actionable ValueError
|
||||
ok, msg = False, f"{type(exc).__name__}: {exc}"
|
||||
if not ok:
|
||||
raise ValueError(f"TTS engine '{engine_id}' is not available: {_mask_hf_tokens(msg)}")
|
||||
|
||||
from core.device_caps import detect_host_caps
|
||||
from services.engine_routing import resolve_routing
|
||||
routing = resolve_routing(getattr(backend_cls, "gpu_compat", ("cpu",)), detect_host_caps())
|
||||
if routing["routing_status"] == "unavailable":
|
||||
raise ValueError(routing["routing_reason"])
|
||||
|
||||
_model = None
|
||||
if backend_cls is OmniVoiceBackend:
|
||||
# OmniVoice needs its model pre-loaded before construction: called
|
||||
# from an async context, OmniVoiceBackend._ensure_loaded() refuses to
|
||||
# bootstrap its own event loop (see its docstring) — same reason
|
||||
# generation.py's /generate special-cases this backend.
|
||||
from services.model_manager import get_model
|
||||
_model = await get_model()
|
||||
backend = get_active_tts_backend(model=_model)
|
||||
|
||||
if require_cloning and not getattr(backend, "supports_cloning", True):
|
||||
raise ValueError(
|
||||
f"The active TTS engine '{engine_id}' doesn't support voice cloning, "
|
||||
f"so {cloning_purpose} can't preserve speaker voices. Switch to one "
|
||||
f"of: {', '.join(cloning_capable_engine_ids())} in Settings → "
|
||||
"Engines, or use OmniVoice for this job."
|
||||
)
|
||||
|
||||
return backend
|
||||
|
||||
|
||||
# ── PEP 562 lazy attribute re-export ───────────────────────────────────────
|
||||
#
|
||||
# Allows ``from services.tts_backend import IndexTTS2Backend`` to keep
|
||||
|
||||
@@ -82,8 +82,12 @@ If `huggingface.co` is slow or blocked, point the client at a mirror:
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
```
|
||||
|
||||
Set it as an environment variable (or in **Settings → environment**) before
|
||||
downloading. Caveats:
|
||||
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets
|
||||
included), or as an environment variable before launching. On first run, the
|
||||
setup wizard offers the same mirror quick-pick right on the system-check
|
||||
screen when the endpoint is unreachable — the network check is a warning, not
|
||||
a blocker, so an offline or firewalled machine can still finish setup once
|
||||
models are available (mirror, or manual download below). Caveats:
|
||||
|
||||
- A mirror serves the **classic** download path, **not Xet** — you lose
|
||||
chunk-dedup and Xet's parallel fetch, but you gain reachability. On the
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# OmniVoice Studio — OpenAI-Compatible Remote ASR
|
||||
|
||||
A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own
|
||||
Whisper API — today, without waiting on `transformers` to ship a direct
|
||||
Qwen3-ASR integration (tracked separately). Unlike every other ASR engine,
|
||||
this one runs no model locally: it's a pure network client that calls any
|
||||
server exposing an OpenAI-compatible `POST /v1/audio/transcriptions`
|
||||
endpoint.
|
||||
|
||||
## Setup
|
||||
|
||||
No install step — configure it directly:
|
||||
|
||||
1. Open **Settings → Models** and find **OpenAI-compatible ASR (remote
|
||||
server)**.
|
||||
2. Set **Server URL** to your server's base URL (e.g.
|
||||
`http://localhost:8000/v1` for a local Qwen3-ASR/FunASR server, or
|
||||
`https://api.openai.com/v1` for OpenAI's own API).
|
||||
3. Set **Model** to whatever your server expects (`whisper-1` for OpenAI's
|
||||
API; check your self-hosted server's docs otherwise).
|
||||
4. **API key** is optional — many self-hosted servers accept requests
|
||||
without one. Set it if your server requires auth, or if you're using
|
||||
OpenAI's own API.
|
||||
5. Activate the engine in **Settings → Engines** — click **Use** on
|
||||
**OpenAI-compatible ASR** in the ASR Engines table (the same picker TTS
|
||||
engines have). Power users can pin it instead by setting
|
||||
`OMNIVOICE_ASR_BACKEND=openai-compat-asr` before launching — the env var
|
||||
always wins over the Settings pick.
|
||||
|
||||
## Response format
|
||||
|
||||
The backend prefers `response_format=verbose_json` for real per-segment
|
||||
timestamps (OpenAI's API and most compatible servers support it) and falls
|
||||
back to plain text automatically if your server rejects that format. Neither
|
||||
path returns word-level timestamps — that's not part of this API.
|
||||
|
||||
## Privacy note
|
||||
|
||||
Unlike every other ASR engine in OmniVoice, audio sent through this backend
|
||||
leaves your machine — to whatever server you configured. If that's a
|
||||
self-hosted server on your own network, nothing leaves your control; if
|
||||
it's a third-party API (OpenAI's, or someone else's), review their data
|
||||
handling before sending anything sensitive.
|
||||
@@ -74,6 +74,8 @@ asr_engines:
|
||||
readme: FunASR
|
||||
- id: sherpa-onnx-asr
|
||||
readme: "**sherpa-onnx** (live dictation)"
|
||||
- id: openai-compat-asr
|
||||
readme: "**OpenAI-compatible** ⚠️ remote"
|
||||
|
||||
# Doc files that must exist (the install path users are sent to).
|
||||
docs:
|
||||
|
||||
@@ -56,6 +56,8 @@ Priority: `duration` > `speed`.
|
||||
| `preprocess_prompt` | bool | True | Whether to apply preprocessing to the voice-clone prompt audio (remove long silences in reference audio, add punctuation in the end of reference text). |
|
||||
| `postprocess_output` | bool | True | Apply post-processing to generated audio (remove long silences). |
|
||||
|
||||
> **Tip — reference-clip quality transfers.** Zero-shot cloning mirrors the acoustics of the reference clip, not just the voice: a clip recorded in an echoey room clones echoey. Record dry and close-mic for clean output. No effect preset adds reverb unless you choose one that declares it (Cinematic, Warm).
|
||||
|
||||
## Long-Form Generation
|
||||
|
||||
To support stable long-form speech generation with low VRAM consumption, the text is automatically split into smaller segments when the estimated duration of the generated speech exceeds `audio_chunk_duration`, with each segment producing approximately `audio_chunk_duration` seconds of audio. This approach allows the model to accept arbitrarily long text and generate arbitrarily long speech with near-constant VRAM consumption.
|
||||
|
||||
+31
-15
@@ -5,13 +5,17 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Using the AppImage or .deb
|
||||
### Using the AppImage
|
||||
|
||||
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
|
||||
a Tauri / WebKitGTK app.
|
||||
- **~10 GB free disk** for the app, its Python environment, and model weights.
|
||||
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
|
||||
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
|
||||
- Optional: **yt-dlp** for downloading YouTube/video clips directly in the
|
||||
Voice Gallery and Dub tabs — `sudo apt install yt-dlp` (Debian/Ubuntu),
|
||||
`sudo dnf install yt-dlp` (Fedora), or `sudo pacman -S yt-dlp` (Arch).
|
||||
Without it those downloads fail; everything else works fine.
|
||||
|
||||
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
|
||||
by the app itself on first launch. No toolchain needed.
|
||||
@@ -73,12 +77,15 @@ No FUSE? Use `--appimage-extract-and-run`:
|
||||
./OmniVoice.Studio_*.AppImage --appimage-extract-and-run
|
||||
```
|
||||
|
||||
## Install (.deb)
|
||||
## .deb package
|
||||
|
||||
```bash
|
||||
sudo apt install ./OmniVoice.Studio_*.amd64.deb
|
||||
omnivoice-studio
|
||||
```
|
||||
Not currently published: `.deb` bundling is disabled in the release pipeline
|
||||
because of a `tauri-cli` bug (`Failed to create control scripts`) — see the
|
||||
comment in `.github/workflows/release.yml` for the tracking note. The
|
||||
AppImage above is the supported Linux install path until a `tauri-cli`
|
||||
version resolves it. `apt install`-able `.deb`s shipped before v0.3 (see
|
||||
[.deb ffprobe conflict](#deb-ffprobe-conflict) below) if you're upgrading
|
||||
from one of those.
|
||||
|
||||
The desktop app uses these canonical paths (kept in sync with
|
||||
`scripts/desktop-prod.sh` by the docs-drift CI gate):
|
||||
@@ -190,26 +197,35 @@ Three ways to opt in, in order of preference:
|
||||
with an AMD GPU but no ROCm runtime it stays offered-but-unselected — install
|
||||
ROCm first (or continue on CPU). Choosing ROCm makes the bootstrap reinstall
|
||||
`torch`/`torchaudio` from the ROCm wheel index
|
||||
(`https://download.pytorch.org/whl/rocm6.2` by default) right after the
|
||||
dependency sync.
|
||||
(`https://download.pytorch.org/whl/rocm6.4` by default) right after the
|
||||
dependency sync — matched to the app's pinned `torch==2.8.0` (the rocm6.2
|
||||
index only ever published up to torch 2.5.1, so it silently failed the
|
||||
reinstall and left the CPU-only CUDA build in place).
|
||||
|
||||
**2. Environment variable (existing installs / headless).** Set
|
||||
`OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
|
||||
the same ROCm reinstall. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
|
||||
index when you need a different ROCm version
|
||||
([pytorch.org](https://pytorch.org/get-started/locally/) lists available
|
||||
wheels). If the reinstall fails (network, unsupported card), OmniVoice keeps
|
||||
the default torch build and warns instead of breaking the install.
|
||||
index when you need a different ROCm version — e.g. AMD publishes newer
|
||||
driver-matched builds (7.2.x) at `repo.radeon.com` as a `--find-links` page
|
||||
rather than a PyPI-style index:
|
||||
```bash
|
||||
uv pip install --reinstall torch==2.8.0 torchaudio==2.8.0 \
|
||||
--find-links https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2.4/
|
||||
```
|
||||
run that manually if you want a specific ROCm point release; the
|
||||
`OMNIVOICE_TORCH_INDEX` env var only accepts a PEP 503 index URL, not a
|
||||
find-links page. If the reinstall fails (network, unsupported card), OmniVoice
|
||||
keeps the default torch build and warns instead of breaking the install.
|
||||
|
||||
**3. Manual wheel swap (fallback).** Replace torch with the ROCm wheel
|
||||
**after** the first-run install populates the venv:
|
||||
|
||||
```bash
|
||||
# From the project directory (source install), into OmniVoice's uv venv.
|
||||
# Current stable is ROCm 6.2 — match your installed ROCm/driver version
|
||||
# (https://pytorch.org/get-started/locally/ lists available wheels).
|
||||
# Matches the app's torch==2.8.0 pin — a different ROCm point release
|
||||
# (e.g. rocm6.2, rocm7.x) may not carry that exact torch build.
|
||||
uv pip install --reinstall torch torchaudio \
|
||||
--index-url https://download.pytorch.org/whl/rocm6.2
|
||||
--index-url https://download.pytorch.org/whl/rocm6.4
|
||||
```
|
||||
|
||||
Once a ROCm build of PyTorch is in the venv, detection is automatic —
|
||||
|
||||
@@ -242,6 +242,15 @@ for the dedicated CosyVoice path.
|
||||
|
||||
**Linked issue:** [#55](https://github.com/debpalash/OmniVoice-Studio/issues/55)
|
||||
|
||||
**Same class, ASR side:** the `nemo-parakeet` ASR engine has the identical
|
||||
problem and currently has **no safe install path** at all — `nemo_toolkit[asr]`
|
||||
hard-pins `transformers>=4.57,<4.58`, which is unsatisfiable alongside
|
||||
OmniVoice's own `transformers>=5.3` requirement. Installing it into the
|
||||
shared venv breaks the backend outright. Do not `pip install nemo_toolkit`
|
||||
into OmniVoice's environment; if you want to try it, use a separate Python
|
||||
environment. Isolated-venv support for this engine (matching CosyVoice/
|
||||
dots-tts) is tracked in [#974](https://github.com/debpalash/OmniVoice-Studio/issues/974).
|
||||
|
||||
## 12. CUDA PyTorch wheel download fails on first run
|
||||
|
||||
**Symptom:** first-run setup stops at **Installing dependencies** with a failure
|
||||
@@ -314,8 +323,10 @@ order:
|
||||
files are a common false-positive quarantine), then re-enable.
|
||||
- **Connection** — use a stable, direct connection; pause any VPN; avoid
|
||||
corporate/school networks.
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, set a
|
||||
mirror **before** launching and relaunch:
|
||||
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, pick a
|
||||
mirror in-app (**Settings → Models → Hugging Face mirror**, or the quick-pick
|
||||
the first-run system check offers when the endpoint is unreachable), or set
|
||||
it as an env var before launching and relaunch:
|
||||
- macOS/Linux: `export HF_ENDPOINT=https://hf-mirror.com`
|
||||
- Windows (PowerShell): `[Environment]::SetEnvironmentVariable("HF_ENDPOINT","https://hf-mirror.com","User")`
|
||||
|
||||
@@ -434,6 +445,41 @@ quit OmniVoice Studio, delete the folder below, then start the app again.
|
||||
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\com.debpalash.omnivoice-studio\EBWebView"
|
||||
```
|
||||
|
||||
## 16. macOS: microphone permission never prompts, OmniVoice never appears in System Settings
|
||||
|
||||
**Symptom:** clicking record shows "Microphone access denied. macOS: open
|
||||
System Settings → Privacy & Security → Microphone and enable OmniVoice" —
|
||||
but OmniVoice never appears in that list, so there's nothing to enable.
|
||||
`NSMicrophoneUsageDescription` is present in the app's `Info.plist`, and
|
||||
resetting the permission (`tccutil reset Microphone
|
||||
com.debpalash.omnivoice-studio`) followed by a relaunch changes nothing — no
|
||||
system prompt ever appears.
|
||||
|
||||
**Cause:** the app bundle was missing the Hardened Runtime *entitlement* for
|
||||
microphone access. An earlier revision of this section blamed an upstream
|
||||
Tauri/WebKit limitation — that was wrong (a community contributor,
|
||||
[@MahdiHedhli](https://github.com/MahdiHedhli), read the sources more
|
||||
carefully and found the real gap). wry's `WKUIDelegate` already grants the
|
||||
WebKit-layer media-capture request; but Tauri's macOS bundler enables
|
||||
Hardened Runtime by default, and Hardened Runtime blocks microphone hardware
|
||||
access unless `com.apple.security.device.audio-input` is present in the
|
||||
signed binary's entitlements — regardless of `Info.plist`'s
|
||||
`NSMicrophoneUsageDescription` (that only supplies the prompt *text*).
|
||||
Without the entitlement, macOS's TCC layer never registers a request, which
|
||||
is exactly why the app never appears in the System Settings list.
|
||||
|
||||
**Fix:** ships in the release after v0.3.12 (the bundle now carries
|
||||
`src-tauri/entitlements.plist` — [#1016](https://github.com/debpalash/OmniVoice-Studio/pull/1016),
|
||||
contributed by the same person who diagnosed it). Update and live recording
|
||||
works, with a normal macOS permission prompt on first use.
|
||||
|
||||
**Workaround on older builds (≤ v0.3.12):** record your voice sample in any
|
||||
other app (Voice Memos, QuickTime, etc.) and upload the resulting file in
|
||||
OmniVoice instead of using live recording — upload-based cloning is
|
||||
unaffected and works normally.
|
||||
|
||||
**Linked issue:** [#1013](https://github.com/debpalash/OmniVoice-Studio/issues/1013)
|
||||
|
||||
## Dub: "translation engine needs the optional … package"
|
||||
|
||||
**Symptom:** in the Dub tab, translating fails with e.g. *"The 'google'
|
||||
|
||||
@@ -77,6 +77,28 @@ Download the latest MSI from the
|
||||
run it, follow the wizard. The shortcut lands in the Start menu as
|
||||
**OmniVoice Studio**.
|
||||
|
||||
### Installing to a different drive
|
||||
|
||||
<a id="install-other-drive"></a>
|
||||
|
||||
The wizard's **directory picker** lets you install the app to any **local**
|
||||
drive (D:, E:, …). Two caveats:
|
||||
|
||||
- **Mapped network drives (Z: → a share) are not supported** — this is a
|
||||
Windows Installer limitation, not an OmniVoice bug: MSI custom actions run
|
||||
as a service account that doesn't see per-user drive mappings, so the
|
||||
install fails or rolls back. Install to a local drive instead.
|
||||
- The install location only moves the ~200 MB app itself. The big data
|
||||
(models, voices, projects — tens of GB) lives in the **data directory**,
|
||||
which you move independently: **Settings → Storage → Models directory**
|
||||
in-app, or `OMNIVOICE_DATA_DIR` / [Portable mode](#portable-install) for
|
||||
the whole data tree.
|
||||
|
||||
If an install to a local non-C: drive fails anyway, capture a log with
|
||||
`msiexec /i OmniVoice*.msi /L*V install.log` and
|
||||
[open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) with it
|
||||
— that log shows exactly which step rolled back.
|
||||
|
||||
## Portable install (Windows)
|
||||
|
||||
<a id="portable-install"></a>
|
||||
|
||||
@@ -6,7 +6,7 @@ Today the longform renderer (Audiobook + Stories) applies a **single-pass** `lou
|
||||
|
||||
Upgrade to **two-pass** `loudnorm`: a first **measure** pass (`print_format=json`, output to `-f null -`) parses the clip's `input_i / input_tp / input_lra / input_thresh / target_offset`, then a second **apply** pass feeds those measured values back as `measured_*` + `offset` + `linear=true`. This lands the output accurately on the preset target. The change is a **runner enhancement** layered over the existing pure builders — the pure `build_loudnorm_filter()` and `LOUDNESS_PRESETS` stay; we add a measure-filter builder, a measured-apply-filter builder, a JSON parser, a measure-cmd argv builder, and an async two-pass orchestrator that runs in `_render_longform_sse` (`backend/api/routers/audiobook.py:345`) between the chapter renders and the final mux. Loudness stays **opt-in** (`loudness: None` default on both `AudiobookRequest` `:151` and `LongformRenderRequest` `:510`), so default cross-platform behavior is unchanged.
|
||||
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard EQ/Compressor/Reverb chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard highpass/Compressor chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "omnivoice-studio",
|
||||
"version": "0.3.11",
|
||||
"version": "0.3.14",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"type": "module",
|
||||
|
||||
Generated
+2
-1
@@ -2941,7 +2941,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.11"
|
||||
version = "0.3.14"
|
||||
dependencies = [
|
||||
"arboard",
|
||||
"dirs-next",
|
||||
@@ -2969,6 +2969,7 @@ dependencies = [
|
||||
"walkdir",
|
||||
"webkit2gtk",
|
||||
"webview2-com",
|
||||
"windows 0.61.3",
|
||||
"windows-core 0.61.2",
|
||||
"zip 2.4.2",
|
||||
]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "omnivoice-studio"
|
||||
version = "0.3.11"
|
||||
version = "0.3.14"
|
||||
description = "OmniVoice Studio – AI voice cloning & dubbing desktop app"
|
||||
authors = ["Debpalash"]
|
||||
license = "AGPL-3.0-only"
|
||||
@@ -71,6 +71,13 @@ zip = { version = "2", default-features = false, features = ["deflate"] }
|
||||
# Versions match what wry already locks — no new native code is pulled in.
|
||||
webview2-com = "0.38"
|
||||
windows-core = "0.61"
|
||||
# Raw HWND access (GetWindowLongPtrW/SetWindowLongPtrW/ShowWindow) to mark the
|
||||
# dictation pill WS_EX_NOACTIVATE so showing it doesn't steal Win32 foreground
|
||||
# activation (#982 — Windows counterpart of #287's macOS focus-steal fix).
|
||||
# Version pinned to match what `tauri` itself already resolves to (0.61.x) so
|
||||
# `WebviewWindow::hwnd()`'s return type and our syscalls share the exact same
|
||||
# `HWND` type — no second copy of the crate enters the dependency graph.
|
||||
windows = { version = "0.61", features = ["Win32_Foundation", "Win32_UI_WindowsAndMessaging"] }
|
||||
|
||||
[target.'cfg(unix)'.dependencies]
|
||||
libc = "0.2"
|
||||
|
||||
@@ -22,9 +22,27 @@ HERE="$(dirname -- "$(readlink -f -- "$0")")"
|
||||
# Sourced by AppRun.test.sh — keep this function pure so unit tests can stub
|
||||
# `pkg-config`, source the file, call _detect_webkit_workaround, and inspect
|
||||
# the resulting environment without exec'ing the binary.
|
||||
#
|
||||
# Version source (#961 follow-up): the WebKitGTK that actually RUNS is the
|
||||
# BUNDLED copy (LD_LIBRARY_PATH below puts $HERE/usr/lib first) — NOT the
|
||||
# host's. Asking the host's pkg-config therefore reads the wrong number
|
||||
# whenever host and bundle diverge (e.g. a user who builds from source has
|
||||
# dev packages installed, so pkg-config answers with their system's healthy
|
||||
# 2.48 while the bundle runs an older lib — skipping a workaround the running
|
||||
# library needs). inject-apprun.sh stamps the bundled version into
|
||||
# .bundled-webkitgtk-version at build time, where it is knowable by
|
||||
# construction; the host pkg-config path survives only as a fallback for
|
||||
# bundles predating the stamp. OMNIVOICE_APPRUN_WK_MARKER exists for the
|
||||
# unit tests to point at a fixture marker.
|
||||
_detect_webkit_workaround() {
|
||||
local wk_version="0.0"
|
||||
if command -v pkg-config >/dev/null 2>&1; then
|
||||
local marker="${OMNIVOICE_APPRUN_WK_MARKER:-$HERE/.bundled-webkitgtk-version}"
|
||||
if [ -r "$marker" ]; then
|
||||
# Empty/unreadable marker content → "0.0" (unknown) → fail-safe workaround,
|
||||
# same philosophy as the missing-pkg-config branch below.
|
||||
wk_version="$(cat "$marker" 2>/dev/null | tr -d '[:space:]')"
|
||||
[ -n "$wk_version" ] || wk_version="0.0"
|
||||
elif command -v pkg-config >/dev/null 2>&1; then
|
||||
wk_version="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|
||||
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|
||||
|| echo "0.0")"
|
||||
|
||||
@@ -72,6 +72,56 @@ run_case "2.46 (broken)" "2.46.1" "1"
|
||||
run_case "2.48 (healthy)" "2.48.0" "unset"
|
||||
run_case "pkg-config absent" "0.0" "1" "no"
|
||||
|
||||
# ── Bundled-version marker cases (#961 follow-up) ───────────────────────────
|
||||
# inject-apprun.sh stamps the bundle's actual WebKitGTK version into
|
||||
# .bundled-webkitgtk-version at build time; AppRun must prefer that marker
|
||||
# over the host's pkg-config (which reports the SYSTEM version — wrong
|
||||
# whenever it diverges from the bundled copy, e.g. on a machine with newer
|
||||
# dev packages installed).
|
||||
|
||||
run_marker_case() {
|
||||
local label="$1" marker_content="$2" pkg_output="$3" expected="$4"
|
||||
local marker_file
|
||||
marker_file="$(mktemp)"
|
||||
printf '%s\n' "$marker_content" > "$marker_file"
|
||||
|
||||
local actual
|
||||
actual=$(
|
||||
bash -c '
|
||||
set +e
|
||||
pkg_output="'"$pkg_output"'"
|
||||
export OMNIVOICE_APPRUN_WK_MARKER="'"$marker_file"'"
|
||||
|
||||
pkg-config() { echo "$pkg_output"; }
|
||||
export -f pkg-config
|
||||
|
||||
exec() { :; }
|
||||
export -f exec
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
source "'"$THIS_DIR"'/AppRun" >/dev/null 2>&1 || true
|
||||
echo "${WEBKIT_DISABLE_COMPOSITING_MODE:-unset}"
|
||||
'
|
||||
)
|
||||
rm -f "$marker_file"
|
||||
|
||||
if [[ "$actual" == "$expected" ]]; then
|
||||
echo "PASS [$label]"
|
||||
PASS_COUNT=$((PASS_COUNT + 1))
|
||||
else
|
||||
echo "FAIL [$label]: expected '$expected' got '$actual'" >&2
|
||||
FAIL_COUNT=$((FAIL_COUNT + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
# Marker says broken → workaround applies, even though host pkg-config says healthy.
|
||||
run_marker_case "marker 2.46 beats host 2.48" "2.46.1" "2.48.0" "1"
|
||||
# Marker says healthy → no workaround, even though host pkg-config says broken
|
||||
# (the exact #961 inversion: from-source user with old system lib, new bundle).
|
||||
run_marker_case "marker 2.48 beats host 2.44" "2.48.0" "2.44.3" "unset"
|
||||
# Empty marker → treated as unknown → fail-safe workaround.
|
||||
run_marker_case "empty marker fails safe" "" "2.48.0" "1"
|
||||
|
||||
echo
|
||||
echo "─── AppRun test summary: $PASS_COUNT pass / $FAIL_COUNT fail ───"
|
||||
if [[ $FAIL_COUNT -ne 0 ]]; then
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<!--
|
||||
Tauri's macOS bundle defaults `hardenedRuntime` to true. Hardened
|
||||
Runtime blocks camera/microphone hardware access unless the matching
|
||||
entitlement is present here — regardless of Info.plist's
|
||||
NSMicrophoneUsageDescription and regardless of wry's own WKUIDelegate
|
||||
already granting the request at the WebKit/JS layer
|
||||
(WryWebViewUIDelegate::request_media_capture_permission unconditionally
|
||||
calls WKPermissionDecision::Grant). Without this entitlement, TCC
|
||||
never even registers a request for the app — nothing shows up in
|
||||
System Settings → Privacy & Security → Microphone to enable, because
|
||||
the OS never saw a legitimately-entitled process ask.
|
||||
-->
|
||||
<key>com.apple.security.device.audio-input</key>
|
||||
<true/>
|
||||
|
||||
<!--
|
||||
Matches Info.plist's forward-looking NSCameraUsageDescription — no
|
||||
current feature uses the camera, but ship the entitlement now so a
|
||||
future getUserMedia({video: true}) call doesn't hit this same bug.
|
||||
-->
|
||||
<key>com.apple.security.device.camera</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -704,9 +704,14 @@ fn sync_failure_is_torch_download(tail: &str) -> bool {
|
||||
|| (low.contains("torch") && (low.contains("failed to download") || low.contains("failed to fetch")))
|
||||
}
|
||||
|
||||
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124). ROCm 6.2 is
|
||||
/// the current stable wheel set; overridable via OMNIVOICE_TORCH_INDEX.
|
||||
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.2";
|
||||
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124).
|
||||
/// ROCm 6.4, not 6.2: the app's pinned `torch==2.8.0` (pyproject.toml) has no
|
||||
/// build on the rocm6.2 index (it tops out at 2.5.1), so that index silently
|
||||
/// failed the reinstall and left the default CUDA build in place — which runs
|
||||
/// on CPU on an AMD GPU (#972). rocm6.4 carries a matching 2.8.0 build.
|
||||
/// Overridable via OMNIVOICE_TORCH_INDEX (e.g. a `--find-links` URL for
|
||||
/// distro-matched ROCm builds torch's own index doesn't carry).
|
||||
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.4";
|
||||
|
||||
/// `uv pip install` args that replace the default CUDA torch build with the AMD
|
||||
/// ROCm wheel (#124). Opt-in (gated on OMNIVOICE_TORCH_VARIANT=rocm by the
|
||||
@@ -1817,7 +1822,10 @@ mod tests {
|
||||
assert!(args.iter().any(|a| a == "torch"));
|
||||
assert!(args.iter().any(|a| a == "torchaudio"));
|
||||
let i = args.iter().position(|a| a == "--index-url").expect("has --index-url");
|
||||
assert!(args[i + 1].contains("rocm6.2"), "default index is the rocm6.2 wheel set");
|
||||
// rocm6.4, not rocm6.2: rocm6.2's index tops out at torch 2.5.1 and
|
||||
// can't satisfy the app's torch==2.8.0 pin (#972) — a regression to
|
||||
// rocm6.2 here would silently resurrect the CPU-fallback bug.
|
||||
assert!(args[i + 1].contains("rocm6.4"), "default index is the rocm6.4 wheel set (matches torch==2.8.0)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
+153
-12
@@ -79,9 +79,18 @@ pub const TRAY_ICON_RECORDING: &[u8] = include_bytes!("../icons/tray-recording.p
|
||||
// applies on top.
|
||||
// - Linux (WebKitGTK): media-stream must be enabled per-WebView and the
|
||||
// permission request answered programmatically.
|
||||
// - macOS (WKWebView): nothing to do here — wry grants media-capture to the
|
||||
// app origin and the user-visible consent is the system TCC prompt driven
|
||||
// by NSMicrophoneUsageDescription in src-tauri/Info.plist.
|
||||
// - macOS (WKWebView): nothing to do here in code — wry's own WKUIDelegate
|
||||
// (WryWebViewUIDelegate::request_media_capture_permission) already grants
|
||||
// every media-capture request unconditionally at the WebKit/JS layer. But
|
||||
// that alone isn't sufficient (#1013): Tauri's macOS bundle defaults
|
||||
// `hardenedRuntime` to true, and Hardened Runtime blocks camera/microphone
|
||||
// hardware access unless the matching entitlement is present — without it,
|
||||
// TCC never even registers a request, so the app never appears in System
|
||||
// Settings → Privacy & Security → Microphone for the user to enable. See
|
||||
// src-tauri/entitlements.plist (wired in via tauri.conf.json's
|
||||
// bundle.macOS.entitlements) for the actual grant; NSMicrophoneUsageDescription
|
||||
// in Info.plist only supplies the *prompt text* TCC shows, it doesn't
|
||||
// substitute for the entitlement.
|
||||
|
||||
/// True for origins the app itself serves: the Tauri custom-protocol origin
|
||||
/// in production and the Vite dev server / loopback in `tauri dev`.
|
||||
@@ -205,6 +214,120 @@ mod media_permission_tests {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Windows: dictation pill must never take foreground focus (#982) ────────
|
||||
//
|
||||
// Windows counterpart of #287 (macOS auto-paste — don't steal focus). The
|
||||
// pill is `.always_on_top(true).skip_taskbar(true)` and is documented above
|
||||
// (see `grant_webview_media_permissions`) as "deliberately unfocused so the
|
||||
// auto-paste lands in the target app" — true on macOS, but on Windows,
|
||||
// showing an always-on-top top-level window gives it Win32 foreground
|
||||
// activation by default (ordinary Windows window-manager behavior; macOS
|
||||
// doesn't force-activate a shown window the same way). Nothing marked the
|
||||
// pill non-activating, so on Windows it stole foreground on every show —
|
||||
// the synthesized Ctrl+V from `simulate_paste` landed back in the pill
|
||||
// instead of the app the user was dictating into, and because the pill
|
||||
// wrongly held focus for the whole session the target app never got it back
|
||||
// until the pill's auto-dismiss timer eventually hid it.
|
||||
//
|
||||
// Two pieces, both required (verified by reading how `.show()` is used at
|
||||
// the call sites below — several are followed by an explicit `set_focus()`
|
||||
// that would fight the style bit on its own):
|
||||
// 1. WS_EX_NOACTIVATE on the HWND, applied once right after creation, so
|
||||
// the OS never grants this window foreground activation implicitly.
|
||||
// 2. `ShowWindow(SW_SHOWNOACTIVATE)` in place of `WebviewWindow::show()` at
|
||||
// the pill's dictation-trigger call sites, and the explicit
|
||||
// `set_focus()` calls at those same sites are skipped on Windows (the
|
||||
// same way they already are on macOS below).
|
||||
//
|
||||
// The flag math (`with_noactivate_style`) is a plain function so it's
|
||||
// unit-testable on every platform — the actual Win32 syscalls that use it
|
||||
// are Windows-only and can't run under `cargo test` on a non-Windows runner.
|
||||
|
||||
/// `WS_EX_NOACTIVATE` (winuser.h: `#define WS_EX_NOACTIVATE 0x08000000L`).
|
||||
/// Hardcoded rather than imported from the `windows` crate so `with_noactivate_style`
|
||||
/// below stays free of the Windows-only dependency and is testable everywhere.
|
||||
/// Only consumed by Windows-only code (or the platform-agnostic test module
|
||||
/// below) — `#[allow(dead_code)]` elsewhere, same as `is_app_origin` above.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
const WS_EX_NOACTIVATE_BIT: isize = 0x0800_0000;
|
||||
|
||||
/// OR `WS_EX_NOACTIVATE` into an existing extended window style, preserving
|
||||
/// every other bit already set (topmost, layered, etc. — the pill's
|
||||
/// `always_on_top(true)` sets one of these). Pure so it's unit-testable
|
||||
/// without a real HWND. See module comment above for why this exists.
|
||||
#[cfg_attr(not(windows), allow(dead_code))]
|
||||
fn with_noactivate_style(current_ex_style: isize) -> isize {
|
||||
current_ex_style | WS_EX_NOACTIVATE_BIT
|
||||
}
|
||||
|
||||
/// Mark the pill's HWND `WS_EX_NOACTIVATE`, once, right after creation — this
|
||||
/// holds for every later `.show()` regardless of call site (belt-and-braces
|
||||
/// alongside `show_pill_noactivate` below, which some call sites also need
|
||||
/// because they pair `.show()` with an explicit `set_focus()`).
|
||||
#[cfg(target_os = "windows")]
|
||||
fn mark_pill_noactivate(win: &tauri::WebviewWindow) {
|
||||
use windows::Win32::UI::WindowsAndMessaging::{
|
||||
GetWindowLongPtrW, SetWindowLongPtrW, GWL_EXSTYLE,
|
||||
};
|
||||
let Ok(hwnd) = win.hwnd() else {
|
||||
log::warn!("pill: could not resolve HWND to apply WS_EX_NOACTIVATE (#982)");
|
||||
return;
|
||||
};
|
||||
unsafe {
|
||||
let current = GetWindowLongPtrW(hwnd, GWL_EXSTYLE);
|
||||
SetWindowLongPtrW(hwnd, GWL_EXSTYLE, with_noactivate_style(current));
|
||||
}
|
||||
}
|
||||
|
||||
/// Show the pill without granting it foreground activation. Used instead of
|
||||
/// `WebviewWindow::show()` at the pill's dictation-trigger call sites on
|
||||
/// Windows — `.show()` maps to plain `ShowWindow(SW_SHOW)`, which relies on
|
||||
/// the NOACTIVATE style alone to suppress activation; `SW_SHOWNOACTIVATE` is
|
||||
/// the explicit, documented way to show a window without activating it and
|
||||
/// costs nothing extra now that the style bit is also set (#982).
|
||||
#[cfg(target_os = "windows")]
|
||||
fn show_pill_noactivate(win: &tauri::WebviewWindow) {
|
||||
use windows::Win32::UI::WindowsAndMessaging::{ShowWindow, SW_SHOWNOACTIVATE};
|
||||
let Ok(hwnd) = win.hwnd() else {
|
||||
log::warn!("pill: could not resolve HWND for non-activating show (#982)");
|
||||
return;
|
||||
};
|
||||
unsafe {
|
||||
let _ = ShowWindow(hwnd, SW_SHOWNOACTIVATE);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod pill_noactivate_tests {
|
||||
use super::{with_noactivate_style, WS_EX_NOACTIVATE_BIT};
|
||||
|
||||
#[test]
|
||||
fn adds_noactivate_bit_without_clobbering_existing_style() {
|
||||
// Stand-in for whatever bits the pill's always_on_top/skip_taskbar
|
||||
// window already carries (e.g. WS_EX_TOPMOST = 0x00000008) —
|
||||
// NOACTIVATE must be added on top, never replace them.
|
||||
let topmost = 0x0000_0008isize;
|
||||
let updated = with_noactivate_style(topmost);
|
||||
assert_eq!(
|
||||
updated & WS_EX_NOACTIVATE_BIT,
|
||||
WS_EX_NOACTIVATE_BIT,
|
||||
"NOACTIVATE bit must be set"
|
||||
);
|
||||
assert_eq!(updated & topmost, topmost, "pre-existing style bits must survive");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn idempotent_if_already_noactivate() {
|
||||
assert_eq!(with_noactivate_style(WS_EX_NOACTIVATE_BIT), WS_EX_NOACTIVATE_BIT);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn matches_documented_win32_value() {
|
||||
// winuser.h: #define WS_EX_NOACTIVATE 0x08000000L
|
||||
assert_eq!(WS_EX_NOACTIVATE_BIT, 0x0800_0000);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Tauri entry ───────────────────────────────────────────────────────────
|
||||
|
||||
#[cfg_attr(mobile, tauri::mobile_entry_point)]
|
||||
@@ -334,9 +457,15 @@ pub fn run() {
|
||||
.skip_taskbar(true)
|
||||
.center()
|
||||
.build();
|
||||
if let Err(e) = result {
|
||||
if let Err(e) = &result {
|
||||
log::error!("Failed to create widget window: {e:?}");
|
||||
}
|
||||
// Windows: mark the pill non-activating right away so it holds
|
||||
// for every later `.show()` regardless of call site (#982).
|
||||
#[cfg(target_os = "windows")]
|
||||
if let Ok(win) = &result {
|
||||
mark_pill_noactivate(win);
|
||||
}
|
||||
}
|
||||
|
||||
app.manage(AppFlags {
|
||||
@@ -370,12 +499,17 @@ pub fn run() {
|
||||
if win.move_window(Position::BottomCenter).is_err() {
|
||||
let _ = win.center();
|
||||
}
|
||||
// Windows: show without granting foreground activation
|
||||
// (#982) — `.show()` on other platforms is unaffected.
|
||||
#[cfg(target_os = "windows")]
|
||||
show_pill_noactivate(&win);
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
let _ = win.show();
|
||||
// Don't steal focus on macOS: the simulated ⌘V from
|
||||
// simulate_paste() must land in the app the user is
|
||||
// dictating into — focusing the widget would swallow
|
||||
// it (#287).
|
||||
#[cfg(not(target_os = "macos"))]
|
||||
// Don't steal focus on macOS or Windows: the simulated
|
||||
// ⌘V/Ctrl+V from simulate_paste() must land in the app
|
||||
// the user is dictating into — focusing the widget would
|
||||
// swallow it (#287 macOS, #982 Windows).
|
||||
#[cfg(not(any(target_os = "macos", target_os = "windows")))]
|
||||
let _ = win.set_focus();
|
||||
}
|
||||
let _ = app_handle.emit("tray-dictate", ());
|
||||
@@ -526,7 +660,9 @@ pub fn run() {
|
||||
// + focus the widget BEFORE emitting tray-dictate so
|
||||
// the user sees the pill instead of silent recording.
|
||||
// Positioning mirrors the global-shortcut handler:
|
||||
// bottom-center (WhisperFlow style).
|
||||
// bottom-center (WhisperFlow style). Windows skips the
|
||||
// focus (and uses a non-activating show) for the same
|
||||
// reason the global-shortcut handler does — see #982.
|
||||
if let Some(win) = app.get_webview_window("widget") {
|
||||
if win.is_visible().unwrap_or(false) {
|
||||
let _ = app.emit("tray-dictate-stop", ());
|
||||
@@ -534,8 +670,13 @@ pub fn run() {
|
||||
if win.move_window(Position::BottomCenter).is_err() {
|
||||
let _ = win.center();
|
||||
}
|
||||
let _ = win.show();
|
||||
let _ = win.set_focus();
|
||||
#[cfg(target_os = "windows")]
|
||||
show_pill_noactivate(&win);
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
{
|
||||
let _ = win.show();
|
||||
let _ = win.set_focus();
|
||||
}
|
||||
let _ = app.emit("tray-dictate", ());
|
||||
}
|
||||
} else {
|
||||
|
||||
@@ -85,7 +85,8 @@
|
||||
],
|
||||
"macOS": {
|
||||
"minimumSystemVersion": "12.0",
|
||||
"signingIdentity": "-"
|
||||
"signingIdentity": "-",
|
||||
"entitlements": "entitlements.plist"
|
||||
}
|
||||
},
|
||||
"plugins": {
|
||||
|
||||
@@ -46,8 +46,15 @@ export async function listEngines(): Promise<AllEnginesResponse> {
|
||||
export async function selectEngine(
|
||||
family: EngineFamily,
|
||||
backendId: string,
|
||||
modelId?: string,
|
||||
): Promise<SelectEngineResponse> {
|
||||
return apiPost<SelectEngineResponse>('/engines/select', { family, backend_id: backendId });
|
||||
return apiPost<SelectEngineResponse>('/engines/select', {
|
||||
family,
|
||||
backend_id: backendId,
|
||||
// Only mlx-audio's curated-model picker (#981) sets this — omit
|
||||
// entirely rather than send `undefined`/null for every other call site.
|
||||
...(modelId ? { model_id: modelId } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -40,6 +40,19 @@ interface EngineBackend {
|
||||
effective_device?: EffectiveDevice;
|
||||
routing_status?: RoutingStatus;
|
||||
routing_reason?: string | null;
|
||||
// #981 — mlx-audio ONLY: it multiplexes 7+ curated models behind one
|
||||
// backend id, so its entry also carries the roster + current pick so
|
||||
// Settings can render a model picker. Absent on every other backend.
|
||||
curated_models?: CuratedModel[];
|
||||
active_model_id?: string;
|
||||
}
|
||||
|
||||
// #981 — one of mlx-audio's curated models (see backend
|
||||
// MLXAudioBackend.CURATED_MODELS / _MLX_AUDIO_MODEL_LABELS).
|
||||
export interface CuratedModel {
|
||||
key: string;
|
||||
label: string;
|
||||
repo_id: string;
|
||||
}
|
||||
|
||||
interface EngineFamilyResponse {
|
||||
|
||||
@@ -17,7 +17,7 @@ import { useTranslation } from 'react-i18next';
|
||||
import { listEngines, getEngineHealth, selfTestEngine } from '../api/engines';
|
||||
import { copyText } from '../utils/copyText';
|
||||
import { ChevronRight } from 'lucide-react';
|
||||
import { Badge, Button, Segmented, Table } from '../ui';
|
||||
import { Badge, Button, Segmented, Select, Table } from '../ui';
|
||||
import { cn } from '@/lib/utils';
|
||||
import SupertonicLicenseDialog from './SupertonicLicenseDialog';
|
||||
|
||||
@@ -63,12 +63,18 @@ function reasonMentionsLicense(reason) {
|
||||
*
|
||||
* Props:
|
||||
* - family: 'tts' | 'asr' | 'llm' default 'tts'
|
||||
* - onSelect?: (family, backendId) => Promise<void> optional — when
|
||||
* provided, a "Use" button appears next to "Test engine" for
|
||||
* - onSelect?: (family, backendId, modelId?) => Promise<void> optional —
|
||||
* when provided, a "Use" button appears next to "Test engine" for
|
||||
* available, non-active rows. Lets the matrix double as an engine
|
||||
* picker so Settings doesn't need a parallel table.
|
||||
* picker so Settings doesn't need a parallel table. The optional third
|
||||
* arg is set only by mlx-audio's curated-model picker (#981).
|
||||
* - activeId?: string the currently-active backend id for this
|
||||
* family. Used to render the "active" badge.
|
||||
* - showFamilyTabs?: boolean default true. When false, the matrix is
|
||||
* pinned to `family` — no TTS/ASR/LLM switcher, and the header names
|
||||
* the family ("ASR Engines") instead of the generic matrix title.
|
||||
* Settings → Engines stacks one pinned matrix per family so the ASR
|
||||
* and LLM pickers are visible instead of tucked behind a tab.
|
||||
*/
|
||||
const FAMILY_META = {
|
||||
tts: { label: 'TTS', icon: Cpu },
|
||||
@@ -139,6 +145,10 @@ function normalizeEntry(entry) {
|
||||
effective_device: entry.effective_device || null,
|
||||
routing_status: entry.routing_status || null,
|
||||
routing_reason: entry.routing_reason || null,
|
||||
// #981 — mlx-audio ONLY: the curated-model roster + current pick.
|
||||
// null/absent on every other backend, which never renders a picker.
|
||||
curated_models: Array.isArray(entry.curated_models) ? entry.curated_models : null,
|
||||
active_model_id: entry.active_model_id || null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -153,8 +163,10 @@ export default function EngineCompatibilityMatrix({
|
||||
family = 'tts',
|
||||
onSelect = null,
|
||||
activeId = null,
|
||||
// Test-friendly overrides — let the RTL suite mock the API layer
|
||||
// without resorting to module-level vi.mock incantations.
|
||||
showFamilyTabs = true,
|
||||
// Injectable API layer — lets the RTL suite mock it without module-level
|
||||
// vi.mock incantations, and lets EnginesTab share one in-flight
|
||||
// GET /engines across its stacked per-family matrices.
|
||||
apiListEngines = listEngines,
|
||||
apiGetEngineHealth = getEngineHealth,
|
||||
apiSelfTestEngine = selfTestEngine,
|
||||
@@ -291,6 +303,18 @@ export default function EngineCompatibilityMatrix({
|
||||
setTimeout(() => setCopiedId((c) => (c === id ? null : c)), 1500);
|
||||
}, []);
|
||||
|
||||
// #981 — mlx-audio's curated-model picker. Reuses the same onSelect the
|
||||
// "Use" button calls, with the curated model key as the optional third
|
||||
// arg, then reloads so active_model_id reflects the new pick immediately.
|
||||
const changeModel = useCallback(
|
||||
async (id, modelId) => {
|
||||
if (!onSelect || !modelId) return;
|
||||
await onSelect(activeFamily, id, modelId);
|
||||
reload();
|
||||
},
|
||||
[onSelect, activeFamily, reload],
|
||||
);
|
||||
|
||||
const COLUMNS = [
|
||||
{ key: 'name', label: t('engines.matrixTitle').split(' ')[0] || 'Engine', flex: 3 },
|
||||
{ key: 'status', label: t('engines.status'), width: 130, align: 'center' },
|
||||
@@ -330,12 +354,19 @@ export default function EngineCompatibilityMatrix({
|
||||
// TTS-05: the license dialog registered for the engine awaiting acceptance
|
||||
// (or null). Capitalized so JSX renders it as a component below.
|
||||
const LicenseDialog = licenseDialogFor ? LICENSE_DIALOGS[licenseDialogFor] : null;
|
||||
// Pinned mode: the header names the family (with its icon) since there is
|
||||
// no switcher to say which family this table is.
|
||||
const familyMeta = FAMILY_META[activeFamily] || FAMILY_META.tts;
|
||||
const TitleIcon = showFamilyTabs ? Layers : familyMeta.icon;
|
||||
|
||||
return (
|
||||
<section className="engine-matrix flex flex-col gap-[var(--space-3,8px)]">
|
||||
<header className="engine-matrix__head flex items-center justify-between gap-[12px]">
|
||||
<h3 className="engine-matrix__title inline-flex items-center gap-[6px] m-0 text-[13px] font-semibold text-[color:var(--chrome-fg,currentColor)]">
|
||||
<Layers size={14} /> {t('engines.matrixTitle')}
|
||||
<TitleIcon size={14} />{' '}
|
||||
{showFamilyTabs
|
||||
? t('engines.matrixTitle')
|
||||
: t('engines.familyMatrixTitle', { family: familyMeta.label })}
|
||||
</h3>
|
||||
<Button
|
||||
size="sm"
|
||||
@@ -348,7 +379,7 @@ export default function EngineCompatibilityMatrix({
|
||||
</Button>
|
||||
</header>
|
||||
|
||||
{families.length > 1 && (
|
||||
{showFamilyTabs && families.length > 1 && (
|
||||
<Segmented
|
||||
size="sm"
|
||||
value={activeFamily}
|
||||
@@ -413,6 +444,35 @@ export default function EngineCompatibilityMatrix({
|
||||
<code className="engine-matrix__id font-mono text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
|
||||
{b.id}
|
||||
</code>
|
||||
{/* #981 — mlx-audio multiplexes 7+ curated models behind this
|
||||
one backend id (Kokoro, CSM, OuteTTS, …); without this
|
||||
picker there's no way to load anything but the default
|
||||
(Kokoro) even after downloading a different model's
|
||||
weights in Settings → Models. Disabled while the row
|
||||
itself isn't available/selectable, matching the "Use"
|
||||
button's gating. */}
|
||||
{b.curated_models && b.curated_models.length > 0 && (
|
||||
<div className="engine-matrix__model-picker flex items-center gap-[6px] mt-[2px]">
|
||||
<span className="text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
|
||||
{t('engines.curatedModelLabel')}
|
||||
</span>
|
||||
<Select
|
||||
size="sm"
|
||||
className="w-auto min-w-[150px]"
|
||||
value={b.active_model_id || ''}
|
||||
disabled={!onSelect || !b.available}
|
||||
onChange={(e) => changeModel(b.id, e.target.value)}
|
||||
aria-label={t('engines.curatedModelAria', { engine: b.display_name })}
|
||||
data-testid={`curated-model-select-${b.id}`}
|
||||
>
|
||||
{b.curated_models.map((m) => (
|
||||
<option key={m.key} value={m.key}>
|
||||
{m.label}
|
||||
</option>
|
||||
))}
|
||||
</Select>
|
||||
</div>
|
||||
)}
|
||||
{/* For available rows, show install_hint inline (one line — usually
|
||||
a parenthetical like "(bundled — no extra install needed)").
|
||||
For unavailable rows, collapse reason + install_hint + last_error
|
||||
|
||||
@@ -376,6 +376,13 @@ export default function Header({
|
||||
<div className="flex flex-col gap-[1px] min-w-0">
|
||||
<span className="text-[12px] text-[var(--color-fg)] font-medium">
|
||||
{m.name}
|
||||
{/* Resident-but-not-routed engine (e.g. OmniVoice still in
|
||||
VRAM after switching to another backend) — say so. */}
|
||||
{m.is_active_engine === false && (
|
||||
<span className="ml-[6px] text-[10px] font-normal text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
|
||||
{t('header.model_not_active')}
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<span className="text-[10px] text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
|
||||
{m.device} {m.vram_mb > 0 ? `· ${m.vram_mb.toFixed(0)} MB` : ''}
|
||||
|
||||
@@ -73,9 +73,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
[t],
|
||||
);
|
||||
|
||||
const donateLabel = t('donate.pill', { defaultValue: 'Support OmniVoice' });
|
||||
const donateActive = mode === 'donate';
|
||||
|
||||
// `nav-rail` is retained purely as the layout hook the (out-of-scope)
|
||||
// `.app-container > .nav-rail` grid rules position by; all visual styling now
|
||||
// lives in the utilities below. Border flips to the inner edge when on the right.
|
||||
@@ -84,17 +81,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
? '[border-left:1px_solid_var(--chrome-border)]'
|
||||
: '[border-right:1px_solid_var(--chrome-border)]';
|
||||
|
||||
// Quiet "Support" pill (was `.rail-btn.donate-pill`): neutral at rest, warms to
|
||||
// the accent on hover/active.
|
||||
const donateState = donateActive
|
||||
? 'text-[var(--chrome-accent)] bg-[var(--chrome-accent-bg)] [border:1px_solid_var(--chrome-accent-border)]'
|
||||
: 'bg-transparent text-[var(--chrome-fg-dim)] [border:1px_solid_transparent] hover:bg-[color-mix(in_srgb,var(--chrome-accent)_10%,transparent)] hover:text-[var(--chrome-accent)]';
|
||||
const heartBase =
|
||||
'text-[16px] leading-none [transition:filter_0.16s,opacity_0.16s,transform_0.16s] group-hover:[transform:scale(1.1)] motion-reduce:[transition:none] motion-reduce:group-hover:[transform:none]';
|
||||
const heartState = donateActive
|
||||
? 'opacity-100 [filter:grayscale(0)]'
|
||||
: 'opacity-75 [filter:grayscale(0.55)] group-hover:opacity-100 group-hover:[filter:grayscale(0)]';
|
||||
|
||||
return (
|
||||
<aside
|
||||
className={`nav-rail z-50 flex select-none flex-col items-center gap-[6px] bg-[var(--chrome-bg)] py-[8px] ${asideBorder}`}
|
||||
@@ -111,19 +97,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
|
||||
))}
|
||||
</div>
|
||||
<div className="flex flex-col items-center gap-[4px]">
|
||||
{/* Quiet "Support" pill — warms to the accent on hover, opens the
|
||||
donate page. Sits with the footer nav (Settings / flip). (#007) */}
|
||||
<button
|
||||
onClick={() => setMode('donate')}
|
||||
title={donateLabel}
|
||||
aria-label={donateLabel}
|
||||
className={`${RAIL_BTN_BASE} ${donateState}`}
|
||||
>
|
||||
<span className={`${heartBase} ${heartState}`} aria-hidden="true">
|
||||
🩷
|
||||
</span>
|
||||
<span className={railLabelCls(side)}>{donateLabel}</span>
|
||||
</button>
|
||||
{footerItems.map((it) => (
|
||||
<RailBtn
|
||||
key={it.id}
|
||||
|
||||
@@ -351,7 +351,14 @@ function WaveformTimeline(
|
||||
console.warn('WebKit audio decode not supported, using media element directly');
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
// Don't rely solely on the 'ready' event firing again for this
|
||||
// recovery load — the play button stayed permanently disabled
|
||||
// when it didn't (the waveform still rendered from the peaks, so
|
||||
// there was no visible sign anything was wrong). Confirm
|
||||
// readiness explicitly once this load settles either way.
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setReady(true);
|
||||
}
|
||||
@@ -372,7 +379,12 @@ function WaveformTimeline(
|
||||
})
|
||||
.then((audioBuffer) => {
|
||||
const channelData = audioBuffer.getChannelData(0);
|
||||
ws.load(undefined, [channelData], audioBuffer.duration);
|
||||
// Same explicit-readiness guard as the NotSupportedError branch
|
||||
// above — don't depend on the 'ready' event re-firing for this
|
||||
// manually-decoded recovery load.
|
||||
Promise.resolve(ws.load(undefined, [channelData], audioBuffer.duration))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
})
|
||||
.catch((decodeErr) => {
|
||||
// HTTP 404 on the companion audio means the source file is
|
||||
@@ -391,7 +403,9 @@ function WaveformTimeline(
|
||||
console.warn('Audio decode fallback failed, loading with empty peaks:', decodeErr);
|
||||
try {
|
||||
const emptyPeaks = new Float32Array(1000).fill(0);
|
||||
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
|
||||
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
|
||||
.then(() => setReady(true))
|
||||
.catch(() => setReady(true));
|
||||
} catch (_) {
|
||||
setLoadError(true);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
// Regression guard: the dub editor's play button stayed permanently disabled
|
||||
// (disabled={!ready}) whenever the initial WaveSurfer decode failed and the
|
||||
// component fell back to a peaks-only ws.load(undefined, [peaks], duration)
|
||||
// call — the waveform still rendered from those peaks (so nothing looked
|
||||
// visibly broken), but `ready` was only ever set from the 'ready' event
|
||||
// re-firing on that recovery load, which this component's own error-handling
|
||||
// code never actually confirmed. Each fallback load must now explicitly
|
||||
// confirm readiness once it settles, instead of assuming the event fires.
|
||||
//
|
||||
// Driving WaveSurfer + a real decode-failure/recovery sequence through jsdom
|
||||
// is brittle (see WaveformTimeline.unlock.test.js), so this is a
|
||||
// source-level contract guard, same house pattern: every `ws.load(undefined,
|
||||
// ...)` recovery call inside the `ws.on('error', ...)` handler must be
|
||||
// followed by an explicit setReady(true) confirmation.
|
||||
|
||||
const src = readFileSync(
|
||||
path.resolve(process.cwd(), 'src/components/WaveformTimeline.jsx'),
|
||||
'utf8',
|
||||
);
|
||||
|
||||
describe('WaveformTimeline error-recovery ready confirmation', () => {
|
||||
it("confirms readiness explicitly after every fallback ws.load() call, not just via the 'ready' event", () => {
|
||||
const errorHandler = /ws\.on\('error', \(err\) => \{([\s\S]*?)\n \}\);/.exec(src)?.[1];
|
||||
expect(errorHandler, "ws.on('error', ...) handler not found").toBeTruthy();
|
||||
|
||||
// Every recovery load in this handler passes peaks explicitly
|
||||
// (`ws.load(undefined, [...], ...)`) — each occurrence must be
|
||||
// immediately confirmed ready via a .then()/.catch() pair (or an
|
||||
// unconditional setReady in a synchronous catch), not left to hope the
|
||||
// 'ready' event re-fires on its own.
|
||||
const loadCalls = [...errorHandler.matchAll(/ws\.load\(undefined, \[[^\]]*\][^)]*\)/g)];
|
||||
expect(loadCalls.length).toBeGreaterThanOrEqual(3);
|
||||
|
||||
for (const match of loadCalls) {
|
||||
const tail = errorHandler.slice(match.index, match.index + 220);
|
||||
expect(tail, `no readiness confirmation after: ${match[0]}`).toMatch(/setReady\(true\)/);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -147,6 +147,13 @@ export default function DesignMethodPanel({
|
||||
{Object.entries(CATEGORIES).map(([key, options]) => {
|
||||
const many = options.length > 6;
|
||||
const optLabel = (val) => {
|
||||
// #983: a profile/localStorage-restored vdStates can carry a
|
||||
// partial shape (missing category keys) — val is then undefined
|
||||
// here even though the 'Auto' check above only catches the
|
||||
// literal sentinel. Guard before .replace() rather than crash;
|
||||
// 'Auto' matches how the rest of the component (the ternary
|
||||
// above, the chip/select fallbacks) treats an unset category.
|
||||
if (typeof val !== 'string' || !val) return 'Auto';
|
||||
const tKey = `clone.opt_${val.replace(/[ -]/g, '_')}`;
|
||||
const tl = t(tKey);
|
||||
return tl !== tKey ? tl : val;
|
||||
@@ -180,9 +187,13 @@ export default function DesignMethodPanel({
|
||||
aria-label={t(`clone.cat_${key}`)}
|
||||
>
|
||||
{options.map((opt, i) => {
|
||||
const optTKey = `clone.opt_${opt.replace(/[ -]/g, '_')}`;
|
||||
// `opt` is always a hardcoded CATEGORIES string today,
|
||||
// never undefined — guarded anyway for consistency with
|
||||
// the identical pattern above (#983).
|
||||
const safeOpt = typeof opt === 'string' && opt ? opt : 'Auto';
|
||||
const optTKey = `clone.opt_${safeOpt.replace(/[ -]/g, '_')}`;
|
||||
const optTl = t(optTKey);
|
||||
const optLabel = optTl !== optTKey ? optTl : opt;
|
||||
const optLabel = optTl !== optTKey ? optTl : safeOpt;
|
||||
const checked = vdStates[key] === opt;
|
||||
// Roving tabindex: the checked chip is the group's
|
||||
// single tab stop (first chip if nothing matches).
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import DesignMethodPanel from './DesignMethodPanel';
|
||||
|
||||
// #983: "Cannot read properties of undefined (reading 'replace')" — the
|
||||
// identity panel crashed whenever vdStates was missing one of the 6
|
||||
// CATEGORIES keys (a design profile saved by an older/foreign client, or a
|
||||
// stale localStorage shape). The label helper called `val.replace(...)` on
|
||||
// an undefined category value. This regression-tests the render guard added
|
||||
// to DesignMethodPanel.jsx directly, independent of the upstream data-shape
|
||||
// fixes in useProfiles.js / useAppData.js / profiles.py.
|
||||
|
||||
// A minimal i18next-compatible mock: returns the defaultValue if given, else
|
||||
// echoes the key back (mirrors i18next's behavior for a missing translation,
|
||||
// which is what `optLabel`'s `tl !== tKey` check relies on).
|
||||
const t = (key, opts) => opts?.defaultValue ?? key;
|
||||
|
||||
function setup(vdStates, props = {}) {
|
||||
return render(
|
||||
<DesignMethodPanel
|
||||
t={t}
|
||||
describeText=""
|
||||
onDescribeChange={vi.fn()}
|
||||
describeMatchedAny={false}
|
||||
describeUnmatched={[]}
|
||||
chipPersonalities={[]}
|
||||
activePersonality={null}
|
||||
applyPersonality={vi.fn()}
|
||||
applyPreset={vi.fn()}
|
||||
identityOpen={true}
|
||||
setIdentityOpen={vi.fn()}
|
||||
identityRecipe="test recipe"
|
||||
vdStates={vdStates}
|
||||
setVdStates={vi.fn()}
|
||||
onChipKeyDown={vi.fn()}
|
||||
showSaveProfile={false}
|
||||
setShowSaveProfile={vi.fn()}
|
||||
profileName=""
|
||||
setProfileName={vi.fn()}
|
||||
handleSaveDesignProfile={vi.fn()}
|
||||
instruct=""
|
||||
language="Auto"
|
||||
{...props}
|
||||
/>,
|
||||
);
|
||||
}
|
||||
|
||||
describe('DesignMethodPanel — #983 partial vdStates crash', () => {
|
||||
it('does not throw when vdStates is missing 5 of the 6 CATEGORIES keys', () => {
|
||||
// Only Gender is set — Age, Pitch, Style, EnglishAccent, ChineseDialect
|
||||
// are all undefined, exercising both the chip-based and <select>-based
|
||||
// ("many" options) render paths.
|
||||
expect(() => setup({ Gender: 'male' })).not.toThrow();
|
||||
});
|
||||
|
||||
it('does not throw when vdStates is a fully empty object', () => {
|
||||
expect(() => setup({})).not.toThrow();
|
||||
});
|
||||
|
||||
it('still renders category labels and the identity recipe with a partial shape', () => {
|
||||
const { container } = setup({ Gender: 'male' });
|
||||
expect(screen.getByText('test recipe')).toBeInTheDocument();
|
||||
// The label text sits alongside a sibling <span> kicker, so assert via
|
||||
// textContent rather than getByText's exact-node matching.
|
||||
expect(container.textContent).toContain('clone.cat_Gender');
|
||||
});
|
||||
});
|
||||
@@ -91,8 +91,13 @@ export default function CommunityZone({
|
||||
name: r.name,
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
flash(t('gallery.use_failed', { defaultValue: 'Could not add that voice.' }));
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
}}
|
||||
onDesign={(item) =>
|
||||
|
||||
@@ -110,7 +110,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const r = await searchYoutube(q, 'import', 10);
|
||||
setResults(r.results || []);
|
||||
} catch (e) {
|
||||
flash(t('gallery.search_failed', { defaultValue: 'Search failed.' }));
|
||||
flash(
|
||||
t('gallery.search_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Search failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
setIsSearching(false);
|
||||
}
|
||||
@@ -149,7 +154,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
await uploadVoiceClip(fd);
|
||||
reload();
|
||||
} catch (err) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: err?.message || String(err),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
if (fileRef.current) fileRef.current.value = '';
|
||||
}
|
||||
@@ -165,7 +175,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
}),
|
||||
);
|
||||
} catch (e) {
|
||||
flash(t('gallery.save_failed', { defaultValue: 'Could not save profile.' }));
|
||||
flash(
|
||||
t('gallery.save_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not save profile: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -179,8 +194,13 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
try {
|
||||
await deleteGalleryVoice(v.id);
|
||||
reload();
|
||||
} catch {
|
||||
/* noop */
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.delete_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not delete: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -191,7 +211,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
const file = new File([blob], `${v.name}.wav`, { type: 'audio/wav' });
|
||||
setTrimming({ voice: v, file });
|
||||
} catch (e) {
|
||||
flash(t('gallery.trim_load_failed', { defaultValue: 'Could not load audio for trimming.' }));
|
||||
flash(
|
||||
t('gallery.trim_load_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not load audio for trimming: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -209,7 +234,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
|
||||
reload();
|
||||
setTrimming(null);
|
||||
} catch (e) {
|
||||
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
|
||||
flash(
|
||||
t('gallery.upload_failed', {
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Upload failed: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* Settings → Models tab → OpenAI-compatible remote ASR panel (#877).
|
||||
*
|
||||
* A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's
|
||||
* own Whisper API — today, without waiting on transformers to ship a direct
|
||||
* Qwen3-ASR integration. Configures the `openai-compat-asr` backend's
|
||||
* base_url/model/api_key; activating it as the active ASR engine still needs
|
||||
* `OMNIVOICE_ASR_BACKEND=openai-compat-asr` (no in-app ASR engine picker
|
||||
* exists yet for any ASR backend — this panel only configures this one).
|
||||
*
|
||||
* Endpoints (loopback-only):
|
||||
* GET /api/settings/asr-openai-compat → {base_url, model, has_key}
|
||||
* PUT /api/settings/asr-openai-compat body {base_url?, model?, api_key?}
|
||||
* ('' clears api_key; omitted/null leaves it unchanged — never returned)
|
||||
*/
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
import { Mic } from 'lucide-react';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { apiJson, apiFetch } from '../../api/client';
|
||||
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
|
||||
import { Button } from '../../ui';
|
||||
|
||||
export default function AsrOpenAICompatPanel() {
|
||||
const { t } = useTranslation();
|
||||
const [baseUrl, setBaseUrl] = useState('');
|
||||
const [model, setModel] = useState('');
|
||||
const [apiKey, setApiKey] = useState('');
|
||||
const [hasKey, setHasKey] = useState(false);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
|
||||
const refresh = useCallback(async () => {
|
||||
setError(null);
|
||||
try {
|
||||
const d = await apiJson('/api/settings/asr-openai-compat');
|
||||
setBaseUrl(d?.base_url || '');
|
||||
setModel(d?.model || '');
|
||||
setHasKey(Boolean(d?.has_key));
|
||||
setApiKey(''); // the key is never returned — the field always starts blank
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatLoadError'));
|
||||
}
|
||||
}, [t]);
|
||||
|
||||
useEffect(() => {
|
||||
refresh();
|
||||
}, [refresh]);
|
||||
|
||||
const save = async () => {
|
||||
setSaving(true);
|
||||
setError(null);
|
||||
try {
|
||||
const res = await apiFetch('/api/settings/asr-openai-compat', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
base_url: baseUrl,
|
||||
model,
|
||||
// Only send api_key when the user actually typed something —
|
||||
// an untouched field must leave the stored key unchanged, not
|
||||
// clear it (the field is always blank on load, so "unchanged"
|
||||
// and "empty" would otherwise be indistinguishable).
|
||||
...(apiKey ? { api_key: apiKey } : {}),
|
||||
}),
|
||||
});
|
||||
const d = await res.json();
|
||||
setBaseUrl(d.base_url || '');
|
||||
setModel(d.model || '');
|
||||
setHasKey(Boolean(d.has_key));
|
||||
setApiKey('');
|
||||
} catch (e) {
|
||||
setError(e?.message || t('models.asrOpenAICompatSaveError'));
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<SettingsSection
|
||||
icon={Mic}
|
||||
title={t('models.asrOpenAICompatTitle')}
|
||||
description={t('models.asrOpenAICompatDescription')}
|
||||
>
|
||||
{error && (
|
||||
<div className="perfpanel__error" role="alert">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatBaseUrlTitle')}
|
||||
hint={t('models.asrOpenAICompatBaseUrlHint')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
placeholder="http://localhost:8000/v1"
|
||||
data-testid="asr-openai-compat-base-url"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatModelTitle')}
|
||||
control={
|
||||
<SettingsInput
|
||||
mono
|
||||
type="text"
|
||||
value={model}
|
||||
onChange={(e) => setModel(e.target.value)}
|
||||
placeholder="whisper-1"
|
||||
data-testid="asr-openai-compat-model"
|
||||
/>
|
||||
}
|
||||
/>
|
||||
|
||||
<SettingRow
|
||||
stack
|
||||
title={t('models.asrOpenAICompatApiKeyTitle')}
|
||||
hint={
|
||||
hasKey ? t('models.asrOpenAICompatKeyConfigured') : t('models.asrOpenAICompatApiKeyHint')
|
||||
}
|
||||
control={
|
||||
<>
|
||||
<SettingsInput
|
||||
mono
|
||||
type="password"
|
||||
value={apiKey}
|
||||
onChange={(e) => setApiKey(e.target.value)}
|
||||
placeholder={hasKey ? '••••••••' : t('models.asrOpenAICompatApiKeyOptional')}
|
||||
data-testid="asr-openai-compat-api-key"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
onClick={save}
|
||||
loading={saving}
|
||||
disabled={saving}
|
||||
data-testid="asr-openai-compat-save"
|
||||
>
|
||||
{t('common.save')}
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
</SettingsSection>
|
||||
);
|
||||
}
|
||||
@@ -1,12 +1,19 @@
|
||||
import React, { useCallback } from 'react';
|
||||
import React, { useCallback, useRef } from 'react';
|
||||
import { toast } from 'react-hot-toast';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
import { addBreadcrumb } from '../../utils/breadcrumbs';
|
||||
import { selectEngine } from '../../api/engines';
|
||||
import { listEngines, selectEngine } from '../../api/engines';
|
||||
import { notifyEngineSelected } from '../../utils/engineSelectToast';
|
||||
import EngineCompatibilityMatrix from '../EngineCompatibilityMatrix';
|
||||
import { SETTINGS_SECTION_SURFACE } from './primitives';
|
||||
|
||||
/** One pinned matrix per family, stacked in this order. ASR used to be
|
||||
* reachable only through the matrix's family tabs, which read as a
|
||||
* TTS-only table — README even promised a Settings ASR picker that
|
||||
* didn't exist (UX gap found during #877). Every family now gets a
|
||||
* visible picker; `OMNIVOICE_*_BACKEND` env vars still win over any pick. */
|
||||
const FAMILIES = ['tts', 'asr', 'llm'];
|
||||
|
||||
export default function EnginesTab() {
|
||||
const { t } = useTranslation();
|
||||
|
||||
@@ -17,10 +24,12 @@ export default function EnginesTab() {
|
||||
//
|
||||
// Review mode (the staged-checkpoint nudges) moved to Settings → General.
|
||||
const onSelect = useCallback(
|
||||
async (family, backendId) => {
|
||||
// modelId is only ever set by mlx-audio's curated-model picker (#981) —
|
||||
// every other call site (the "Use" button) omits it.
|
||||
async (family, backendId, modelId) => {
|
||||
try {
|
||||
addBreadcrumb(`engine:${family}=${backendId}`);
|
||||
const r = await selectEngine(family, backendId);
|
||||
const r = await selectEngine(family, backendId, modelId);
|
||||
// Consume the routing echo: warn (not a bare success) when the pick
|
||||
// lands on a CPU fallback on this host. See notifyEngineSelected.
|
||||
notifyEngineSelected(r, t, family);
|
||||
@@ -31,9 +40,32 @@ export default function EnginesTab() {
|
||||
[t],
|
||||
);
|
||||
|
||||
// The stacked matrices all consume the same GET /engines payload — share
|
||||
// one in-flight request so opening the tab probes every engine once, not
|
||||
// once per family. A per-matrix Refresh after the shared promise settles
|
||||
// still triggers a fresh fetch.
|
||||
const inflightList = useRef(null);
|
||||
const listEnginesShared = useCallback(() => {
|
||||
if (!inflightList.current) {
|
||||
inflightList.current = listEngines().finally(() => {
|
||||
inflightList.current = null;
|
||||
});
|
||||
}
|
||||
return inflightList.current;
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<section className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
|
||||
<EngineCompatibilityMatrix family="tts" onSelect={onSelect} />
|
||||
</section>
|
||||
<>
|
||||
{FAMILIES.map((family) => (
|
||||
<section key={family} className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
|
||||
<EngineCompatibilityMatrix
|
||||
family={family}
|
||||
showFamilyTabs={false}
|
||||
onSelect={onSelect}
|
||||
apiListEngines={listEnginesShared}
|
||||
/>
|
||||
</section>
|
||||
))}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import React from 'react';
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
|
||||
// Keep toast side-channels out of the test (timers, portals).
|
||||
vi.mock('react-hot-toast', () => ({
|
||||
default: { error: vi.fn(), success: vi.fn() },
|
||||
toast: Object.assign(vi.fn(), { error: vi.fn(), success: vi.fn() }),
|
||||
}));
|
||||
|
||||
vi.mock('../../api/engines', () => ({
|
||||
listEngines: vi.fn(),
|
||||
selectEngine: vi.fn(),
|
||||
getEngineHealth: vi.fn(),
|
||||
selfTestEngine: vi.fn(),
|
||||
}));
|
||||
|
||||
import { listEngines, selectEngine } from '../../api/engines';
|
||||
import EnginesTab from './EnginesTab';
|
||||
|
||||
function entry(id, name) {
|
||||
return {
|
||||
id,
|
||||
display_name: name,
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
};
|
||||
}
|
||||
|
||||
const ENGINES = {
|
||||
tts: { active: 'omnivoice', backends: [entry('omnivoice', 'OmniVoice (test)')] },
|
||||
asr: {
|
||||
active: 'whisperx',
|
||||
backends: [
|
||||
entry('whisperx', 'WhisperX (test)'),
|
||||
entry('openai-compat-asr', 'OpenAI-compatible ASR (test)'),
|
||||
],
|
||||
},
|
||||
llm: { active: 'off', backends: [entry('off', 'Off (test)')] },
|
||||
};
|
||||
|
||||
describe('EnginesTab', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
listEngines.mockResolvedValue(ENGINES);
|
||||
});
|
||||
|
||||
it('renders a pinned picker per family — TTS, ASR and LLM all visible at once', async () => {
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
|
||||
// One named section per family (the ASR picker used to be tucked behind
|
||||
// a family tab inside a single TTS-titled matrix — no picker to find).
|
||||
expect(screen.getByText('TTS Engines')).toBeInTheDocument();
|
||||
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
|
||||
expect(screen.getByText('LLM Engines')).toBeInTheDocument();
|
||||
// Pinned matrices render no family switcher.
|
||||
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
|
||||
});
|
||||
|
||||
it('the stacked matrices share one GET /engines on mount', async () => {
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
expect(listEngines).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('clicking Use on an ASR engine selects it with family="asr"', async () => {
|
||||
selectEngine.mockResolvedValue({
|
||||
family: 'asr',
|
||||
active: 'openai-compat-asr',
|
||||
env_override: false,
|
||||
routing_status: 'cpu_only',
|
||||
effective_device: 'cpu',
|
||||
routing_reason: null,
|
||||
});
|
||||
render(<EnginesTab />);
|
||||
await waitFor(() => screen.getByText('OpenAI-compatible ASR (test)'));
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /use openai-compatible asr \(test\)/i }));
|
||||
await waitFor(() => {
|
||||
expect(selectEngine).toHaveBeenCalledWith('asr', 'openai-compat-asr', undefined);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -8,6 +8,7 @@ import { listExportHistory } from '../api/exports';
|
||||
import { modelStatus as apiModelStatus } from '../api/system';
|
||||
import { useModelStatus } from '../api/hooks';
|
||||
import useRealtimeEvents from './useRealtimeEvents';
|
||||
import { mergeDescribedAttrs } from '../utils/voiceInstruct';
|
||||
|
||||
/**
|
||||
* Encapsulates all data-loading effects, localStorage persistence,
|
||||
@@ -174,7 +175,12 @@ export default function useAppData() {
|
||||
setDefineMethod('design');
|
||||
} else if (saved.mode) setMode(saved.mode);
|
||||
if (saved.defineMethod) setDefineMethod(saved.defineMethod);
|
||||
if (saved.vdStates) setVdStates(saved.vdStates);
|
||||
// #983: legacy localStorage state had no shape validation at all — a
|
||||
// partial/corrupt saved.vdStates crashed DesignMethodPanel on restore.
|
||||
// Mirror useProfiles.js's guard: require a plain object, then complete
|
||||
// it to the full CATEGORIES shape (missing/unknown keys → 'Auto').
|
||||
if (saved.vdStates && typeof saved.vdStates === 'object')
|
||||
setVdStates(mergeDescribedAttrs(saved.vdStates));
|
||||
if (saved.language) setLanguage(saved.language);
|
||||
if (saved.isSidebarCollapsed !== undefined) setIsSidebarCollapsed(saved.isSidebarCollapsed);
|
||||
if (saved.sidebarTab) setSidebarTab(saved.sidebarTab);
|
||||
|
||||
@@ -11,7 +11,11 @@ import { generateSpeech, audioUrlWithCacheBust } from '../api/generate';
|
||||
import { apiFetch } from '../api/client';
|
||||
import { playBlobAudio } from '../utils/media';
|
||||
import { PRESETS } from '../utils/constants';
|
||||
import { instructToFormValue } from '../utils/voiceInstruct';
|
||||
import {
|
||||
instructToFormValue,
|
||||
mergeDescribedAttrs,
|
||||
buildDesignInstruct,
|
||||
} from '../utils/voiceInstruct';
|
||||
import { askConfirm } from '../utils/dialog';
|
||||
import { toast } from 'react-hot-toast';
|
||||
import { recordValueMoment } from '../utils/donationMoments';
|
||||
@@ -55,7 +59,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
const safeBlob = new Blob([arrBuf], { type: refAudio.type });
|
||||
formData.append('ref_audio', safeBlob, refAudio.name || 'profile.wav');
|
||||
formData.append('ref_text', refText);
|
||||
formData.append('instruct', instruct);
|
||||
// #1010: the backend only sanitizes instruct on save for kind='design'
|
||||
// profiles — a clone profile (this call always creates kind='clone')
|
||||
// would silently persist an unsupported free-text instruct and then
|
||||
// 400 every single time it's used to generate. Filter here too.
|
||||
const { instruct: safeInst } = buildDesignInstruct({}, instruct);
|
||||
formData.append('instruct', safeInst);
|
||||
formData.append('language', language);
|
||||
try {
|
||||
await createProfile(formData);
|
||||
@@ -96,7 +105,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
if (profile.kind === 'design' && profile.vd_states) {
|
||||
try {
|
||||
const parsed = JSON.parse(profile.vd_states);
|
||||
if (parsed && typeof parsed === 'object') setVdStates(parsed);
|
||||
// #983: a profile saved by an older/foreign client (or hand-edited)
|
||||
// can carry a partial shape — mergeDescribedAttrs (already used for
|
||||
// the "describe your voice" restore path) guarantees every
|
||||
// CATEGORIES key is present, defaulting missing/unknown ones to
|
||||
// 'Auto', so DesignMethodPanel never sees an undefined category.
|
||||
if (parsed && typeof parsed === 'object') setVdStates(mergeDescribedAttrs(parsed));
|
||||
} catch {
|
||||
/* malformed stored state — sliders keep their current values */
|
||||
}
|
||||
@@ -199,6 +213,25 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
fin_prof = '';
|
||||
}
|
||||
|
||||
// #1010: this instruct string comes straight from segment/preset data,
|
||||
// never through the validator-safe builder — a preset's raw attrs or a
|
||||
// free-text style field can carry phrases outside the active engine's
|
||||
// supported instruct vocabulary, 400ing instead of previewing. Same
|
||||
// client-side guard useTTS.js already applies to the clone path.
|
||||
if (fin_inst) {
|
||||
const { instruct: safeInst, unsupported, duplicates } = buildDesignInstruct({}, fin_inst);
|
||||
if (unsupported.length) {
|
||||
toast(t('tts_errors.ignored_unsupported', { items: unsupported.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
if (duplicates.length) {
|
||||
toast(t('tts_errors.ignored_duplicate', { items: duplicates.join(', ') }), {
|
||||
icon: '⚠️',
|
||||
});
|
||||
}
|
||||
fin_inst = safeInst;
|
||||
}
|
||||
if (fin_prof) formData.append('profile_id', fin_prof);
|
||||
if (fin_inst) formData.append('instruct', fin_inst);
|
||||
const fin_lang = seg.target_lang || dubLang;
|
||||
@@ -240,7 +273,10 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
|
||||
: item.text
|
||||
: '';
|
||||
formData.append('ref_text', extractedText);
|
||||
formData.append('instruct', item.instruct || '');
|
||||
// #1010: same guard as handleSaveProfile — this always creates a
|
||||
// kind='clone' profile, which the backend never sanitizes on save.
|
||||
const { instruct: safeHistInst } = buildDesignInstruct({}, item.instruct || '');
|
||||
formData.append('instruct', safeHistInst);
|
||||
formData.append('language', item.language || 'Auto');
|
||||
if (item.seed !== undefined && item.seed !== null) {
|
||||
formData.append('seed', item.seed);
|
||||
|
||||
@@ -1188,8 +1188,8 @@
|
||||
"no_matches": "No voices match these filters.",
|
||||
"load_more": "Load more",
|
||||
"saved_as_profile": "Added \"{{name}}\" to your voices.",
|
||||
"use_failed": "Could not create that voice — the engine may be loading.",
|
||||
"preview_failed": "Preview unavailable — the voice engine may still be loading.",
|
||||
"use_failed": "Could not create that voice: {{message}}",
|
||||
"preview_failed": "Preview unavailable: {{message}}",
|
||||
"import_explainer": "Paste a URL you have the rights to (or upload a file), trim the part you need, and save it as a voice. You are responsible for the licensing of anything you import.",
|
||||
"import_placeholder": "Paste a video/audio URL, or type to search…",
|
||||
"imported_clip": "Imported clip",
|
||||
@@ -1199,11 +1199,12 @@
|
||||
"no_imports": "Nothing imported yet. Paste a URL above to get started.",
|
||||
"search_results": "{{count}} results",
|
||||
"download_failed": "Download failed: {{msg}}",
|
||||
"search_failed": "Search failed.",
|
||||
"upload_failed": "Upload failed.",
|
||||
"save_failed": "Could not save profile.",
|
||||
"search_failed": "Search failed: {{message}}",
|
||||
"upload_failed": "Upload failed: {{message}}",
|
||||
"save_failed": "Could not save profile: {{message}}",
|
||||
"confirm_delete": "Delete \"{{name}}\"?",
|
||||
"trim_load_failed": "Could not load audio for trimming.",
|
||||
"delete_failed": "Could not delete: {{message}}",
|
||||
"trim_load_failed": "Could not load audio for trimming: {{message}}",
|
||||
"delete": "Delete",
|
||||
"community_empty": "No community voices loaded yet — connect to the internet and reopen, or be the first to submit one.",
|
||||
"community_explainer": "Designed presets and recorded voices shared by the community, loaded from the omnivoice-gallery.",
|
||||
@@ -1291,7 +1292,11 @@
|
||||
"try_dictation": "Try dictation",
|
||||
"next_try_dictation": "Next: Try dictation",
|
||||
"step_aria": "Step {{num}}: {{label}}",
|
||||
"step_completed": "completed"
|
||||
"step_completed": "completed",
|
||||
"mirror_rescue_title": "Can't reach Hugging Face? Use a mirror",
|
||||
"mirror_rescue_hint": "Model downloads switch to the mirror immediately — no restart needed during setup. hf-mirror.com is the community mirror commonly used in China.",
|
||||
"mirror_apply": "Apply & re-check",
|
||||
"mirror_apply_error": "Could not save the mirror setting."
|
||||
},
|
||||
"donate": {
|
||||
"title": "Support",
|
||||
@@ -1549,6 +1554,7 @@
|
||||
"loading": "Loading engines…",
|
||||
"refresh": "Refresh",
|
||||
"matrixTitle": "Engine Compatibility Matrix",
|
||||
"familyMatrixTitle": "{{family}} Engines",
|
||||
"loadFailed": "Failed to load engines: {{message}}",
|
||||
"couldNotLoad": "Could not load engines: {{message}}",
|
||||
"retry": "Retry",
|
||||
@@ -1588,7 +1594,9 @@
|
||||
"routingUnknown": "Unknown",
|
||||
"routingEffectiveChip": "Runs on {{device}} on this machine",
|
||||
"routingCaveatTitle": "GPU selected, but: {{reason}}",
|
||||
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}"
|
||||
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}",
|
||||
"curatedModelLabel": "Model",
|
||||
"curatedModelAria": "Model for {{engine}}"
|
||||
},
|
||||
"errors": {
|
||||
"title": "This tab hit a snag.",
|
||||
@@ -1759,6 +1767,7 @@
|
||||
"flush": "Flush",
|
||||
"loaded_models": "Loaded Models",
|
||||
"no_models": "No models loaded",
|
||||
"model_not_active": "not active — safe to unload",
|
||||
"unload": "Unload",
|
||||
"flush_caches": "Flush caches",
|
||||
"unload_all_flush": "Unload all + flush"
|
||||
@@ -2007,7 +2016,18 @@
|
||||
"mirror_preset_hint": "On a restricted network, route model downloads through a mirror. Leave empty for the official endpoint.",
|
||||
"mirror_restart_note": "Model Store downloads use the new mirror immediately. Only model loads (transformers) pick it up after a restart.",
|
||||
"mirror_load_error": "Failed to load mirror setting",
|
||||
"mirror_save_error": "Failed to save"
|
||||
"mirror_save_error": "Failed to save",
|
||||
"asrOpenAICompatTitle": "OpenAI-compatible ASR (remote server)",
|
||||
"asrOpenAICompatDescription": "Point transcription at Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own API.",
|
||||
"asrOpenAICompatBaseUrlTitle": "Server URL",
|
||||
"asrOpenAICompatBaseUrlHint": "The base URL of an OpenAI-compatible transcription server. To use this engine, also set OMNIVOICE_ASR_BACKEND=openai-compat-asr — there's no in-app engine picker for ASR yet.",
|
||||
"asrOpenAICompatModelTitle": "Model",
|
||||
"asrOpenAICompatApiKeyTitle": "API key",
|
||||
"asrOpenAICompatApiKeyHint": "Optional — many self-hosted servers don't require one.",
|
||||
"asrOpenAICompatApiKeyOptional": "optional",
|
||||
"asrOpenAICompatKeyConfigured": "A key is saved. Leave blank to keep it, or type a new one to replace it.",
|
||||
"asrOpenAICompatLoadError": "Failed to load ASR server setting",
|
||||
"asrOpenAICompatSaveError": "Failed to save"
|
||||
},
|
||||
"enterprise_faq": {
|
||||
"q_internal_tools": "Do I need a license for internal tools?",
|
||||
|
||||
@@ -515,7 +515,9 @@
|
||||
"routingRemote": "远程",
|
||||
"routingUnknown": "未知",
|
||||
"routingEffectiveChip": "在此机器上的 {{device}} 上运行",
|
||||
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}"
|
||||
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}",
|
||||
"curatedModelLabel": "模型",
|
||||
"curatedModelAria": "{{engine}} 的模型"
|
||||
},
|
||||
"capture": {
|
||||
"desc": "全局热键仅在桌面应用中有效。网页界面使用页面内快捷键 <1>Ctrl+Shift+Space</1>(窗口聚焦时可用)。",
|
||||
@@ -1144,7 +1146,11 @@
|
||||
"next_try_dictation": "下一步:试用听写",
|
||||
"step_aria": "第 {{num}} 步:{{label}}",
|
||||
"step_completed": "已完成",
|
||||
"cache_label": "模型缓存"
|
||||
"cache_label": "模型缓存",
|
||||
"mirror_rescue_title": "无法连接 Hugging Face?使用镜像",
|
||||
"mirror_rescue_hint": "模型下载会立即切换到镜像地址,安装过程无需重启。hf-mirror.com 是国内常用的社区镜像。",
|
||||
"mirror_apply": "应用并重新检查",
|
||||
"mirror_apply_error": "镜像设置保存失败。"
|
||||
},
|
||||
"donate": {
|
||||
"title": "支持项目",
|
||||
@@ -1504,6 +1510,7 @@
|
||||
"flush": "冲洗",
|
||||
"loaded_models": "加载模型",
|
||||
"no_models": "没有加载模型",
|
||||
"model_not_active": "非当前引擎 — 可安全卸载",
|
||||
"unload": "卸载",
|
||||
"flush_caches": "刷新缓存",
|
||||
"unload_all_flush": "全部卸载+冲洗"
|
||||
|
||||
@@ -22,6 +22,7 @@ import StoragePanel from '../components/settings/StoragePanel';
|
||||
import StorageTab from '../components/settings/StorageTab';
|
||||
import StorageUsagePanel from '../components/settings/StorageUsagePanel';
|
||||
import HFMirrorPanel from '../components/settings/HFMirrorPanel';
|
||||
import AsrOpenAICompatPanel from '../components/settings/AsrOpenAICompatPanel';
|
||||
import SharingPanel from '../components/settings/SharingPanel';
|
||||
import RemoteBackendPanel from '../components/settings/RemoteBackendPanel';
|
||||
import MCPBindingsPanel from '../components/settings/MCPBindingsPanel';
|
||||
@@ -367,6 +368,7 @@ export default function Settings() {
|
||||
<>
|
||||
<StoragePanel />
|
||||
<HFMirrorPanel />
|
||||
<AsrOpenAICompatPanel />
|
||||
<ModelStoreTab info={info} modelBadge={modelBadge} />
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -3,6 +3,7 @@ import { useTranslation } from 'react-i18next';
|
||||
import { Loader, RotateCw } from 'lucide-react';
|
||||
import { cn } from '@/lib/utils';
|
||||
import { useSetupStatus, usePreflight } from '../api/hooks';
|
||||
import { apiJson, apiFetch } from '../api/client';
|
||||
import WizardLibrary from '../components/WizardLibrary';
|
||||
import HfTokenCard from '../components/HfTokenCard';
|
||||
import DictationDemo from '../components/DictationDemo';
|
||||
@@ -133,6 +134,101 @@ function PreflightPanel({ report, loading, onRecheck }) {
|
||||
);
|
||||
}
|
||||
|
||||
/* ── Mirror rescue — restricted-network escape hatch on step 0 ─────────── */
|
||||
|
||||
/**
|
||||
* Shown when the network preflight check can't reach the Hugging Face
|
||||
* endpoint. Users behind restricted networks (e.g. China, where
|
||||
* huggingface.co is blocked) can't reach Settings yet — the wizard gates the
|
||||
* studio — so the mirror quick-pick has to live right here. PUT /hf-mirror
|
||||
* takes effect immediately for downloads (no restart during setup).
|
||||
*/
|
||||
function MirrorRescue({ onApplied }) {
|
||||
const { t } = useTranslation();
|
||||
const [presets, setPresets] = useState([]);
|
||||
const [url, setUrl] = useState('');
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [error, setError] = useState(null);
|
||||
|
||||
useEffect(() => {
|
||||
let alive = true;
|
||||
apiJson('/api/settings/hf-mirror')
|
||||
.then((d) => {
|
||||
if (!alive) return;
|
||||
setPresets((d?.presets || []).filter((p) => p.url));
|
||||
setUrl(d?.configured || '');
|
||||
})
|
||||
.catch(() => {
|
||||
/* endpoint unavailable — keep the free-text input usable */
|
||||
});
|
||||
return () => {
|
||||
alive = false;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const apply = async (value) => {
|
||||
setSaving(true);
|
||||
setError(null);
|
||||
try {
|
||||
await apiFetch('/api/settings/hf-mirror', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url: value }),
|
||||
});
|
||||
setUrl(value);
|
||||
onApplied();
|
||||
} catch (e) {
|
||||
setError(e?.message || t('setup.mirror_apply_error'));
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="mt-3 flex flex-col gap-1.5 rounded-md border border-border px-3 py-2.5">
|
||||
<span className="text-sm font-semibold">{t('setup.mirror_rescue_title')}</span>
|
||||
<span className="text-xs leading-snug text-fg-muted">{t('setup.mirror_rescue_hint')}</span>
|
||||
{error && (
|
||||
<span className="text-xs text-danger" role="alert">
|
||||
{error}
|
||||
</span>
|
||||
)}
|
||||
<div className="mt-1 flex flex-wrap items-center gap-2">
|
||||
{presets.map((p) => (
|
||||
<Button
|
||||
key={p.url}
|
||||
variant="preset"
|
||||
size="sm"
|
||||
disabled={saving}
|
||||
onClick={() => apply(p.url)}
|
||||
data-testid={`wizard-mirror-${p.url}`}
|
||||
>
|
||||
{p.label}
|
||||
</Button>
|
||||
))}
|
||||
<input
|
||||
type="text"
|
||||
value={url}
|
||||
onChange={(e) => setUrl(e.target.value)}
|
||||
placeholder="https://hf-mirror.com"
|
||||
className="min-w-[220px] flex-1 rounded border border-border bg-transparent px-2 py-1 font-mono text-xs text-fg"
|
||||
data-testid="wizard-mirror-url"
|
||||
/>
|
||||
<Button
|
||||
variant="subtle"
|
||||
size="sm"
|
||||
loading={saving}
|
||||
disabled={saving || !url.trim()}
|
||||
onClick={() => apply(url.trim())}
|
||||
data-testid="wizard-mirror-apply"
|
||||
>
|
||||
{t('setup.mirror_apply')}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
/* ── LED stepper rail ──────────────────────────────────────────────────── */
|
||||
|
||||
function StepperNav({ step, maxUnlockedStep, onStep }) {
|
||||
@@ -241,6 +337,10 @@ export default function SetupWizard({ onReady }) {
|
||||
|
||||
const modelsReady = !!status?.models_ready;
|
||||
const preflightOk = !!pre?.ok;
|
||||
// Offer the mirror quick-pick whenever the HF endpoint probe didn't pass —
|
||||
// the wizard is the only surface these users can reach (Settings is gated
|
||||
// behind setup), so the escape hatch must live here.
|
||||
const networkDown = (pre?.checks || []).some((c) => c.id === 'network' && c.status !== 'pass');
|
||||
|
||||
const cachePath = status?.hf_cache_dir || '~/.cache/huggingface';
|
||||
|
||||
@@ -288,6 +388,7 @@ export default function SetupWizard({ onReady }) {
|
||||
<div className="flex min-h-0 flex-auto flex-col gap-3" key="step-0">
|
||||
<div className="fr-rise min-h-0 flex-1 overflow-y-auto" style={{ '--rise': 1 }}>
|
||||
<PreflightPanel report={pre} loading={preLoading} onRecheck={recheckPreflight} />
|
||||
{networkDown && <MirrorRescue onApplied={recheckPreflight} />}
|
||||
</div>
|
||||
<div
|
||||
className="fr-rise flex shrink-0 items-center justify-between gap-4 border-t border-border pt-3"
|
||||
|
||||
@@ -138,7 +138,8 @@ export default function VoiceGallery() {
|
||||
stopPlayback();
|
||||
flash(
|
||||
t('gallery.preview_failed', {
|
||||
defaultValue: 'Preview unavailable — the voice engine may still be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Preview unavailable: {{message}}',
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
@@ -222,7 +223,8 @@ export default function VoiceGallery() {
|
||||
} catch (e) {
|
||||
flash(
|
||||
t('gallery.use_failed', {
|
||||
defaultValue: 'Could not create that voice — the engine may be loading.',
|
||||
message: e?.message || String(e),
|
||||
defaultValue: 'Could not create that voice: {{message}}',
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -702,4 +702,189 @@ describe('EngineCompatibilityMatrix', () => {
|
||||
// KittenTTS in the fixture carries no setup_snippet → no snippet block.
|
||||
expect(screen.queryByTestId('setup-snippet-kittentts')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
// ── #981 — mlx-audio curated-model picker ───────────────────────────────
|
||||
function mlxAudioResponse({ activeModelId = 'kokoro' } = {}) {
|
||||
return {
|
||||
tts: {
|
||||
active: 'mlx-audio',
|
||||
backends: [
|
||||
{
|
||||
id: 'mlx-audio',
|
||||
display_name: 'MLX-Audio (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['mps', 'cpu'],
|
||||
curated_models: [
|
||||
{
|
||||
key: 'kokoro',
|
||||
label: 'Kokoro (default, fast)',
|
||||
repo_id: 'mlx-community/Kokoro-82M-bf16',
|
||||
},
|
||||
{ key: 'csm', label: 'CSM (voice cloning)', repo_id: 'mlx-community/csm-1b-8bit' },
|
||||
{
|
||||
key: 'outetts',
|
||||
label: 'OuteTTS',
|
||||
repo_id: 'mlx-community/Llama-OuteTTS-1.0-1B-4bit',
|
||||
},
|
||||
],
|
||||
active_model_id: activeModelId,
|
||||
},
|
||||
],
|
||||
},
|
||||
asr: { active: '', backends: [] },
|
||||
llm: { active: 'off', backends: [] },
|
||||
};
|
||||
}
|
||||
|
||||
it('renders the curated-model picker for mlx-audio, pre-selected to the active model', async () => {
|
||||
const apiListEngines = vi
|
||||
.fn()
|
||||
.mockResolvedValue(mlxAudioResponse({ activeModelId: 'outetts' }));
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={vi.fn()}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
const select = screen.getByTestId('curated-model-select-mlx-audio');
|
||||
expect(select).toHaveValue('outetts');
|
||||
// All curated models are offered as options.
|
||||
expect(
|
||||
within(select).getByRole('option', { name: 'Kokoro (default, fast)' }),
|
||||
).toBeInTheDocument();
|
||||
expect(within(select).getByRole('option', { name: 'CSM (voice cloning)' })).toBeInTheDocument();
|
||||
expect(within(select).getByRole('option', { name: 'OuteTTS' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('picking a different curated model calls onSelect with the model key and refreshes', async () => {
|
||||
let activeModelId = 'kokoro';
|
||||
const apiListEngines = vi.fn(async () => mlxAudioResponse({ activeModelId }));
|
||||
const onSelect = vi.fn(async (_family, _id, modelId) => {
|
||||
activeModelId = modelId;
|
||||
});
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={onSelect}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
const select = screen.getByTestId('curated-model-select-mlx-audio');
|
||||
fireEvent.change(select, { target: { value: 'csm' } });
|
||||
|
||||
await waitFor(() => {
|
||||
expect(onSelect).toHaveBeenCalledWith('tts', 'mlx-audio', 'csm');
|
||||
});
|
||||
// Reloaded after the pick — matrix reflects the new active_model_id.
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('curated-model-select-mlx-audio')).toHaveValue('csm');
|
||||
});
|
||||
expect(apiListEngines.mock.calls.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it('does not render a curated-model picker for engines without curated_models', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
onSelect={vi.fn()}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('OmniVoice (test)'));
|
||||
expect(screen.queryByTestId(/curated-model-select-/)).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('disables the curated-model picker when no onSelect is provided', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(mlxAudioResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('MLX-Audio (test)'));
|
||||
expect(screen.getByTestId('curated-model-select-mlx-audio')).toBeDisabled();
|
||||
});
|
||||
|
||||
// ── showFamilyTabs={false} — pinned per-family mount (Settings → Engines) ─
|
||||
function multiFamilyResponse() {
|
||||
return {
|
||||
tts: {
|
||||
active: 'omnivoice',
|
||||
backends: [
|
||||
{
|
||||
id: 'omnivoice',
|
||||
display_name: 'OmniVoice (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
},
|
||||
],
|
||||
},
|
||||
asr: {
|
||||
active: 'whisperx',
|
||||
backends: [
|
||||
{
|
||||
id: 'whisperx',
|
||||
display_name: 'WhisperX (test)',
|
||||
available: true,
|
||||
reason: null,
|
||||
install_hint: null,
|
||||
last_error: null,
|
||||
isolation_mode: 'in-process',
|
||||
gpu_compat: ['cpu'],
|
||||
},
|
||||
],
|
||||
},
|
||||
llm: { active: 'off', backends: [] },
|
||||
};
|
||||
}
|
||||
|
||||
it('pins to the given family and hides the TTS/ASR/LLM switcher when showFamilyTabs is false', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="asr"
|
||||
showFamilyTabs={false}
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('WhisperX (test)'));
|
||||
// Pinned header names the family instead of the generic matrix title…
|
||||
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
|
||||
// …the TTS family never leaks into the pinned table…
|
||||
expect(screen.queryByText('OmniVoice (test)')).not.toBeInTheDocument();
|
||||
// …and there is no family switcher to wander off to.
|
||||
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
|
||||
});
|
||||
|
||||
it('keeps the family switcher by default (standalone mounts unchanged)', async () => {
|
||||
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
|
||||
render(
|
||||
<EngineCompatibilityMatrix
|
||||
family="tts"
|
||||
apiListEngines={apiListEngines}
|
||||
apiGetEngineHealth={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
await waitFor(() => screen.getByText('OmniVoice (test)'));
|
||||
expect(screen.getByText('Engine Compatibility Matrix')).toBeInTheDocument();
|
||||
expect(document.querySelectorAll('.engine-matrix__tab-family').length).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
// Regression guard: VoiceGallery/CommunityZone/ImportsZone catch blocks used
|
||||
// to discard the real error and show a hardcoded, often-wrong generic guess
|
||||
// (e.g. "the engine may be loading" on ANY failure, including ones that had
|
||||
// nothing to do with loading). Fixed to interpolate the real `e.message`
|
||||
// (already a clean, user-facing string from api/client.js's ApiError),
|
||||
// matching the `{{message}}` convention used everywhere else in this file.
|
||||
// This test only pins the i18n keys, not the call sites, deliberately: it's
|
||||
// a cheap net against reverting to a hardcoded string, not a full behavior test.
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import en from '../i18n/locales/en.json';
|
||||
|
||||
describe('gallery error messages interpolate the real error', () => {
|
||||
const keys = [
|
||||
'use_failed',
|
||||
'preview_failed',
|
||||
'search_failed',
|
||||
'upload_failed',
|
||||
'save_failed',
|
||||
'delete_failed',
|
||||
'trim_load_failed',
|
||||
];
|
||||
|
||||
it.each(keys)('gallery.%s contains {{message}}', (key) => {
|
||||
expect(en.gallery[key]).toContain('{{message}}');
|
||||
});
|
||||
});
|
||||
@@ -5,7 +5,20 @@
|
||||
const MAX = 500;
|
||||
const buf = [];
|
||||
|
||||
// Tauri's own internal IPC fallback (#975): on some Windows configurations
|
||||
// the custom-protocol IPC probe fails once at startup and Tauri logs this
|
||||
// exact console.warn before silently — and successfully — falling back to
|
||||
// postMessage + WebSocket. It's benign and fires at most once per launch,
|
||||
// but as a captured console.warn it spuriously flips the Logs footer's
|
||||
// Frontend pill to "1 warning" on every affected launch. Filtered at the
|
||||
// capture source (not the display layer) so it never enters the ring
|
||||
// buffer or a copied diagnostic dump either.
|
||||
const BENIGN_WARNING_PREFIXES = ['IPC custom protocol failed'];
|
||||
|
||||
function push(level, args) {
|
||||
if (level === 'warn' && typeof args[0] === 'string') {
|
||||
if (BENIGN_WARNING_PREFIXES.some((p) => args[0].startsWith(p))) return;
|
||||
}
|
||||
const msg = Array.from(args)
|
||||
.map((a) => {
|
||||
if (a instanceof Error) return `${a.name}: ${a.message}${a.stack ? '\n' + a.stack : ''}`;
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import { beforeEach, describe, expect, it } from 'vitest';
|
||||
import { clearFrontendLogs, getFrontendLogs, installConsoleCapture } from './consoleBuffer';
|
||||
|
||||
describe('consoleBuffer', () => {
|
||||
// installConsoleCapture() wraps console.* exactly once per page load (a
|
||||
// module-level `installed` guard) — it must NOT be re-installed or have
|
||||
// console.warn restored between tests, or later tests run against the
|
||||
// un-wrapped original. Install once; only the ring buffer resets per test.
|
||||
installConsoleCapture();
|
||||
|
||||
beforeEach(() => {
|
||||
clearFrontendLogs();
|
||||
});
|
||||
|
||||
it('captures an ordinary warning', () => {
|
||||
console.warn('something genuinely worth seeing');
|
||||
expect(getFrontendLogs().some((l) => l.msg.includes('something genuinely worth seeing'))).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("#975: filters Tauri's benign IPC-fallback warning out of the captured buffer", () => {
|
||||
console.warn(
|
||||
'IPC custom protocol failed, Tauri will now use the postMessage interface instead',
|
||||
);
|
||||
expect(getFrontendLogs().some((l) => l.msg.includes('IPC custom protocol failed'))).toBe(false);
|
||||
});
|
||||
|
||||
it('does not filter a different warning that merely mentions IPC', () => {
|
||||
// Prefix match, not a substring match — only Tauri's exact known message
|
||||
// is suppressed; anything else that happens to mention "IPC" is not.
|
||||
console.warn('some other IPC warning entirely');
|
||||
expect(getFrontendLogs().some((l) => l.msg.includes('some other IPC warning entirely'))).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -52,6 +52,22 @@ describe('buildDesignInstruct', () => {
|
||||
expect(unsupported).toEqual(['sôi nổi']);
|
||||
});
|
||||
|
||||
it('clone path (#980): RTL/non-Latin script (Hebrew) is dropped like any other unsupported free-text, not a crash', () => {
|
||||
// #612 covered Latin-script-with-diacritics (Vietnamese); #980 was the same
|
||||
// failure mode with a right-to-left script — a name typed into the Style
|
||||
// field must degrade the same way (dropped client-side + unsupported bucket),
|
||||
// never round-trip to the backend's 400.
|
||||
const { instruct, unsupported } = buildDesignInstruct({}, 'שמואל');
|
||||
expect(instruct).toBe('');
|
||||
expect(unsupported).toEqual(['שמואל']);
|
||||
});
|
||||
|
||||
it('clone path (#980): RTL script mixed with a valid tag keeps the tag, drops the rest', () => {
|
||||
const { instruct, unsupported } = buildDesignInstruct({}, 'whisper, שמואל');
|
||||
expect(instruct).toBe('whisper');
|
||||
expect(unsupported).toEqual(['שמואל']);
|
||||
});
|
||||
|
||||
it('buckets a valid tag outranked by a dropdown as a duplicate, not unsupported (#114)', () => {
|
||||
const { instruct, unsupported, duplicates } = buildDesignInstruct(
|
||||
{ Pitch: 'low pitch' },
|
||||
|
||||
+10
-1
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "omnivoice"
|
||||
version = "0.3.11"
|
||||
version = "0.3.14"
|
||||
description = "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models"
|
||||
readme = "README.md"
|
||||
# Free and open-source under the GNU Affero General Public License v3 (see
|
||||
@@ -154,6 +154,15 @@ dependencies = [
|
||||
# httpx imports it lazily inside try/except, so PyInstaller's tracer
|
||||
# misses it and frozen installers would stay broken without the entry.
|
||||
"socksio>=1.0",
|
||||
# OS trust store for TLS (#976). Users behind a corporate/antivirus proxy
|
||||
# that TLS-inspects traffic get a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE]"
|
||||
# on every model install — the TCP connection succeeds, but the proxy's
|
||||
# re-signed certificate is trusted by the OS (Windows CryptoAPI/SChannel)
|
||||
# and not by Python's bundled `certifi` CA list. `truststore` patches
|
||||
# `ssl.SSLContext` to verify against the OS trust store instead. Pure-
|
||||
# Python, MIT, PyPA-maintained, zero transitive deps — same class of fix
|
||||
# as socksio above, identical on macOS/Windows/Linux.
|
||||
"truststore>=0.9",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
||||
@@ -42,6 +42,23 @@ for stage_base in "$STAGE_BASE_RELEASE" "$STAGE_BASE_DEBUG"; do
|
||||
echo "inject-apprun: replacing AppRun in $appdir"
|
||||
cp -f "$APPRUN_SRC" "$appdir/AppRun"
|
||||
chmod 755 "$appdir/AppRun"
|
||||
# Stamp the bundled WebKitGTK version (#961 follow-up). The AppImage
|
||||
# bundles THIS build host's libwebkit2gtk, so the host's pkg-config
|
||||
# answer here is the version the shipped bundle will actually run —
|
||||
# knowable by construction at bundle time, unknowable reliably at
|
||||
# runtime (a user's pkg-config reports their SYSTEM's version, which
|
||||
# LD_LIBRARY_PATH overrides with the bundled copy). AppRun's workaround
|
||||
# auto-detection reads this marker first and only falls back to host
|
||||
# pkg-config when the marker is absent (bundles predating the stamp).
|
||||
wk_bundled="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|
||||
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|
||||
|| echo "")"
|
||||
if [ -n "$wk_bundled" ]; then
|
||||
printf '%s\n' "$wk_bundled" > "$appdir/.bundled-webkitgtk-version"
|
||||
echo "inject-apprun: stamped bundled WebKitGTK version: $wk_bundled"
|
||||
else
|
||||
echo "inject-apprun: WARNING — could not read the bundled WebKitGTK version (pkg-config missing?); AppRun will use its runtime fallback" >&2
|
||||
fi
|
||||
found=1
|
||||
fi
|
||||
done
|
||||
|
||||
@@ -247,6 +247,213 @@ def test_select_llm_never_routing_gated(fresh_app, monkeypatch):
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
|
||||
# ── ASR selection via /engines/select (Settings → Engines ASR picker) ──────
|
||||
#
|
||||
# The ASR family was always wired in _FAMILIES on paper, but no UI called it
|
||||
# and nothing exercised it — the Settings picker now does. Lock the contract:
|
||||
# a pick persists to prefs["asr_backend"], `OMNIVOICE_ASR_BACKEND` still wins
|
||||
# over the pick, and unknown / not-ready ids are 400s.
|
||||
|
||||
|
||||
def _register_fake_asr(asr_mod, engine_id, *, available=True):
|
||||
"""Register a light in-process ASR stub (CPU-only so a forced-CPU host
|
||||
routes it `cpu_only`, never `unavailable`). Returns (cls, restore_fn)."""
|
||||
_avail = available
|
||||
|
||||
class _FakeASR(asr_mod.ASRBackend):
|
||||
id = engine_id
|
||||
display_name = f"Fake {engine_id}"
|
||||
gpu_compat = ("cpu",)
|
||||
|
||||
@classmethod
|
||||
def is_available(cls):
|
||||
return (True, "ready") if _avail else (False, "deps missing (test)")
|
||||
|
||||
def transcribe(self, audio_path, *, word_timestamps=True):
|
||||
raise NotImplementedError
|
||||
|
||||
saved = dict(asr_mod._REGISTRY)
|
||||
asr_mod._REGISTRY[engine_id] = _FakeASR
|
||||
|
||||
def restore():
|
||||
asr_mod._REGISTRY.clear()
|
||||
asr_mod._REGISTRY.update(saved)
|
||||
|
||||
return _FakeASR, restore
|
||||
|
||||
|
||||
def test_select_asr_persists_pref_and_echoes_active(fresh_app, monkeypatch):
|
||||
from core import prefs as _prefs
|
||||
from services import asr_backend as asr_mod
|
||||
|
||||
_force_cpu_host(monkeypatch)
|
||||
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
|
||||
_, restore = _register_fake_asr(asr_mod, "fake-asr")
|
||||
try:
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select", json={"family": "asr", "backend_id": "fake-asr"})
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["family"] == "asr"
|
||||
assert body["active"] == "fake-asr"
|
||||
assert body["env_override"] is False
|
||||
assert _prefs.get("asr_backend") == "fake-asr"
|
||||
finally:
|
||||
restore()
|
||||
|
||||
|
||||
def test_select_asr_env_var_still_wins(fresh_app, monkeypatch):
|
||||
"""CRITICAL backward-compat: an existing `OMNIVOICE_ASR_BACKEND` pin keeps
|
||||
winning over a Settings pick — the pick persists to prefs (for when the
|
||||
pin is lifted) but the active id stays the env value, and the response
|
||||
says so via env_override."""
|
||||
from core import prefs as _prefs
|
||||
from services import asr_backend as asr_mod
|
||||
|
||||
_force_cpu_host(monkeypatch)
|
||||
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "pytorch-whisper")
|
||||
_, restore = _register_fake_asr(asr_mod, "fake-asr-pinned")
|
||||
try:
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select", json={"family": "asr", "backend_id": "fake-asr-pinned"})
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["env_override"] is True
|
||||
assert body["active"] == "pytorch-whisper" # env wins
|
||||
assert _prefs.get("asr_backend") == "fake-asr-pinned"
|
||||
finally:
|
||||
restore()
|
||||
|
||||
|
||||
def test_select_asr_unknown_backend_is_400(fresh_app):
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select", json={"family": "asr", "backend_id": "nope-not-real"})
|
||||
assert r.status_code == 400
|
||||
assert "Unknown asr backend" in r.json()["detail"]
|
||||
|
||||
|
||||
def test_select_asr_unavailable_backend_is_400(fresh_app, monkeypatch):
|
||||
from services import asr_backend as asr_mod
|
||||
|
||||
_force_cpu_host(monkeypatch)
|
||||
_, restore = _register_fake_asr(asr_mod, "fake-asr-down", available=False)
|
||||
try:
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select", json={"family": "asr", "backend_id": "fake-asr-down"})
|
||||
assert r.status_code == 400
|
||||
assert "not ready" in r.json()["detail"]
|
||||
finally:
|
||||
restore()
|
||||
|
||||
|
||||
def test_get_engines_asr_family_shape(fresh_app):
|
||||
"""GET /engines/asr — the ASR picker's data source: active id + one row
|
||||
per registered backend with availability, reasons and install hints."""
|
||||
r = _client(fresh_app).get("/engines/asr")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert isinstance(body["active"], str) and body["active"]
|
||||
by_id = {b["id"]: b for b in body["backends"]}
|
||||
assert {"whisperx", "faster-whisper", "openai-compat-asr"}.issubset(by_id)
|
||||
# Install hints power the picker's tooltips (parity with TTS).
|
||||
assert by_id["openai-compat-asr"]["install_hint"]
|
||||
for entry in by_id.values():
|
||||
missing = _REQUIRED_KEYS - entry.keys()
|
||||
assert not missing, f"asr entry {entry['id']!r} missing: {missing}"
|
||||
|
||||
|
||||
# ── #981 — mlx-audio curated-model selection via /engines/select ───────────
|
||||
#
|
||||
# mlx-audio multiplexes 7+ curated models behind one backend id. Before this
|
||||
# fix there was NO way anywhere in the UI/API to pick which curated model
|
||||
# actually loads — it always defaulted to Kokoro even if the user had
|
||||
# downloaded e.g. Llama-OuteTTS via Settings → Models.
|
||||
|
||||
|
||||
def _make_mlx_audio_available(monkeypatch):
|
||||
"""mlx-audio is Apple-Silicon-gated; force is_available()=True + a
|
||||
CPU-friendly host so the routing gate doesn't block these tests on
|
||||
non-mac CI runners."""
|
||||
from services import tts_backend as tts_mod
|
||||
monkeypatch.setattr(
|
||||
tts_mod.MLXAudioBackend, "is_available",
|
||||
classmethod(lambda cls: (True, "ready")),
|
||||
)
|
||||
_force_cpu_host(monkeypatch)
|
||||
|
||||
|
||||
def test_select_mlx_audio_unknown_model_id_is_400(fresh_app, monkeypatch):
|
||||
_make_mlx_audio_available(monkeypatch)
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select",
|
||||
json={"family": "tts", "backend_id": "mlx-audio", "model_id": "not-a-real-model"},
|
||||
)
|
||||
assert r.status_code == 400
|
||||
assert "Unknown mlx-audio model" in r.json()["detail"]
|
||||
|
||||
|
||||
def test_select_mlx_audio_curated_key_persists(fresh_app, monkeypatch):
|
||||
from core import prefs as _prefs
|
||||
_make_mlx_audio_available(monkeypatch)
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select",
|
||||
json={"family": "tts", "backend_id": "mlx-audio", "model_id": "outetts"},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert _prefs.get("mlx_audio_model_id") == "outetts"
|
||||
assert _prefs.get("tts_backend") == "mlx-audio"
|
||||
|
||||
|
||||
def test_select_mlx_audio_raw_repo_id_accepted(fresh_app, monkeypatch):
|
||||
"""MLXAudioBackend already tolerates a raw HF repo id, not just a
|
||||
curated key (tts_backend.py ~733) — the API must too."""
|
||||
from core import prefs as _prefs
|
||||
_make_mlx_audio_available(monkeypatch)
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select",
|
||||
json={
|
||||
"family": "tts", "backend_id": "mlx-audio",
|
||||
"model_id": "mlx-community/Some-Other-Model-4bit",
|
||||
},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert _prefs.get("mlx_audio_model_id") == "mlx-community/Some-Other-Model-4bit"
|
||||
|
||||
|
||||
def test_select_mlx_audio_without_model_id_does_not_touch_pref(fresh_app, monkeypatch):
|
||||
"""Selecting mlx-audio without a model_id (e.g. an older frontend) must
|
||||
leave any existing mlx_audio_model_id pref untouched."""
|
||||
from core import prefs as _prefs
|
||||
_prefs.set_("mlx_audio_model_id", "csm")
|
||||
_make_mlx_audio_available(monkeypatch)
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select", json={"family": "tts", "backend_id": "mlx-audio"})
|
||||
assert r.status_code == 200, r.text
|
||||
assert _prefs.get("mlx_audio_model_id") == "csm"
|
||||
|
||||
|
||||
def test_select_model_id_ignored_for_non_mlx_audio_backend(fresh_app):
|
||||
"""model_id is only meaningful for mlx-audio; picking a different TTS
|
||||
backend with a model_id set must not persist a stray pref."""
|
||||
from core import prefs as _prefs
|
||||
r = _client(fresh_app).post(
|
||||
"/engines/select",
|
||||
json={"family": "tts", "backend_id": "omnivoice", "model_id": "kokoro"},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
assert _prefs.get("mlx_audio_model_id") is None
|
||||
|
||||
|
||||
def test_engines_response_curated_models_only_on_mlx_audio(fresh_app):
|
||||
client = _client(fresh_app)
|
||||
body = client.get("/engines").json()
|
||||
by_id = {b["id"]: b for b in body["tts"]["backends"]}
|
||||
assert "curated_models" in by_id["mlx-audio"]
|
||||
assert "active_model_id" in by_id["mlx-audio"]
|
||||
assert "curated_models" not in by_id["omnivoice"]
|
||||
assert "active_model_id" not in by_id["omnivoice"]
|
||||
|
||||
|
||||
# ── /engines/{id}/health round-trip ────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
@@ -141,7 +141,10 @@ def test_list_backends_resilient(registry_sandbox):
|
||||
|
||||
|
||||
def test_list_backends_shape(registry_sandbox):
|
||||
"""Every entry must contain exactly the documented keys — no more, no less."""
|
||||
"""Every entry must contain exactly the documented keys — no more, no
|
||||
less — EXCEPT mlx-audio, which also carries `curated_models` +
|
||||
`active_model_id` (#981): it multiplexes 7+ curated models behind one
|
||||
backend id, so the Settings picker needs the roster + current pick."""
|
||||
out = list_backends()
|
||||
# `gpu_compat` joined the documented shape in Plan 02-04 alongside the
|
||||
# Engine Compatibility Matrix UI (ENGINE-06). The three routing keys
|
||||
@@ -154,14 +157,51 @@ def test_list_backends_shape(registry_sandbox):
|
||||
# Copy-paste env-var line for path-gated opt-in engines (None otherwise).
|
||||
"setup_snippet",
|
||||
}
|
||||
mlx_audio_extra = {"curated_models", "active_model_id"}
|
||||
for entry in out:
|
||||
assert set(entry.keys()) == required, (
|
||||
expected = required | mlx_audio_extra if entry["id"] == "mlx-audio" else required
|
||||
assert set(entry.keys()) == expected, (
|
||||
f"entry {entry.get('id')} has wrong keys: "
|
||||
f"missing {required - entry.keys()}, "
|
||||
f"extra {entry.keys() - required}"
|
||||
f"missing {expected - entry.keys()}, "
|
||||
f"extra {entry.keys() - expected}"
|
||||
)
|
||||
|
||||
|
||||
def test_mlx_audio_curated_models_roster(registry_sandbox):
|
||||
"""#981 — mlx-audio's entry carries the curated-model roster + the
|
||||
currently-active pick, so Settings can render a model picker instead of
|
||||
always silently defaulting to Kokoro."""
|
||||
out = {entry["id"]: entry for entry in list_backends()}
|
||||
entry = out["mlx-audio"]
|
||||
assert entry["active_model_id"] == "kokoro" # DEFAULT_MODEL_KEY, no prefs set
|
||||
keys = {m["key"] for m in entry["curated_models"]}
|
||||
assert keys == set(tts_backend.MLXAudioBackend.CURATED_MODELS)
|
||||
for m in entry["curated_models"]:
|
||||
assert set(m.keys()) == {"key", "label", "repo_id"}
|
||||
assert m["repo_id"] == tts_backend.MLXAudioBackend.CURATED_MODELS[m["key"]]
|
||||
assert m["label"] # non-empty, readable
|
||||
|
||||
|
||||
def test_mlx_audio_active_model_id_reflects_prefs(registry_sandbox, monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
|
||||
_prefs.set_("mlx_audio_model_id", "outetts")
|
||||
out = {entry["id"]: entry for entry in list_backends()}
|
||||
assert out["mlx-audio"]["active_model_id"] == "outetts"
|
||||
|
||||
|
||||
def test_curated_models_not_present_on_other_backends(registry_sandbox):
|
||||
"""Only mlx-audio multiplexes multiple models behind one backend id — no
|
||||
other entry should carry curated_models/active_model_id."""
|
||||
out = list_backends()
|
||||
for entry in out:
|
||||
if entry["id"] == "mlx-audio":
|
||||
continue
|
||||
assert "curated_models" not in entry
|
||||
assert "active_model_id" not in entry
|
||||
|
||||
|
||||
def test_isolation_mode_in_process_vs_subprocess(registry_sandbox):
|
||||
"""SubprocessBackend subclasses get isolation_mode='subprocess'; others 'in-process'."""
|
||||
registry_sandbox["fake-sub"] = FakeSubBackend
|
||||
|
||||
@@ -37,6 +37,38 @@ if not os.environ.get("OMNIVOICE_ENV_FILE"):
|
||||
|
||||
|
||||
import pytest
|
||||
import warnings as _warnings
|
||||
|
||||
|
||||
# ── torch default-dtype isolation (CI flaky trio) ───────────────────────────
|
||||
# Three tests (test_effects_chain / test_generation_audio_guard /
|
||||
# test_persona_bundle) fail intermittently on CI — never locally — with
|
||||
# signatures that all trace to one cause: a leaked
|
||||
# `torch.set_default_dtype(torch.float16)` from some earlier test. The
|
||||
# smoking gun is test_generation_audio_guard's observed value
|
||||
# 0.0999755859375, which is exactly float16(0.1): `torch.tensor([0.1, …])`
|
||||
# built under a leaked fp16 default. The same leak collapses
|
||||
# test_effects_chain's preset differences into identical quantized outputs,
|
||||
# and hands test_persona_bundle's soundfile writer fp16 data libsndfile
|
||||
# can't encode. The polluter only executes on CI-Linux (it never reproduces
|
||||
# on macOS), so rather than chase it blind, this guard makes the whole leak
|
||||
# class impossible — same philosophy as the LLM-state guard below — and
|
||||
# names the offender in CI output when it fires, so it CAN be chased.
|
||||
@pytest.fixture(autouse=True)
|
||||
def _torch_default_dtype_guard(request):
|
||||
yield
|
||||
torch = sys.modules.get("torch")
|
||||
if torch is None:
|
||||
return
|
||||
if torch.get_default_dtype() is not torch.float32:
|
||||
_warnings.warn(
|
||||
f"{request.node.nodeid} leaked torch default dtype "
|
||||
f"{torch.get_default_dtype()} — resetting to float32. This is "
|
||||
f"the polluter behind the CI flaky trio; fix it at the source.",
|
||||
stacklevel=1,
|
||||
)
|
||||
torch.set_default_dtype(torch.float32)
|
||||
|
||||
|
||||
# ── LLM-provider state isolation (issue #878) ──────────────────────────────
|
||||
# LLM provider selection is process-global three ways: env vars (the
|
||||
|
||||
Vendored
+2
@@ -18,6 +18,7 @@ DELETE /profiles/{profile_id}/consent
|
||||
DELETE /projects/{project_id}
|
||||
DELETE /pronunciation/{entry_id}
|
||||
GET /api/mcp/bindings
|
||||
GET /api/settings/asr-openai-compat
|
||||
GET /api/settings/changelog
|
||||
GET /api/settings/db-backup
|
||||
GET /api/settings/dictation-refinement
|
||||
@@ -216,6 +217,7 @@ POST /v1/audio/transcriptions
|
||||
POST /watermark/detect
|
||||
POST /watermark/settings
|
||||
PUT /api/mcp/bindings
|
||||
PUT /api/settings/asr-openai-compat
|
||||
PUT /api/settings/dictation-refinement
|
||||
PUT /api/settings/hf-mirror
|
||||
PUT /api/settings/llm-endpoint
|
||||
|
||||
@@ -61,3 +61,19 @@ test('empty / missing attrs yields all-Auto', () => {
|
||||
for (const cat of ALL_CATS) assert.equal(out[cat], 'Auto');
|
||||
}
|
||||
});
|
||||
|
||||
test('#983 — a partial vdStates shape (as restored from a saved profile or '
|
||||
+ 'localStorage) is completed to all 6 CATEGORIES keys', () => {
|
||||
// Mirrors the exact partial shape from issue #983: only Gender survives
|
||||
// (e.g. a design profile saved by an older client, or a hand-edited
|
||||
// payload), the other 5 category keys are simply absent from the object.
|
||||
// useProfiles.js/useAppData.js now run any restored vd_states through this
|
||||
// helper before calling setVdStates, so DesignMethodPanel's render never
|
||||
// sees an undefined category value.
|
||||
const out = mergeDescribedAttrs({ Gender: 'male' });
|
||||
assert.deepEqual(Object.keys(out).sort(), [...ALL_CATS].sort());
|
||||
assert.equal(out.Gender, 'male');
|
||||
for (const cat of ALL_CATS) {
|
||||
if (cat !== 'Gender') assert.equal(out[cat], 'Auto');
|
||||
}
|
||||
});
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Run the AppImage AppRun launcher's shell unit tests under pytest.
|
||||
|
||||
AppRun.test.sh existed but was wired into NO CI job — the launcher's
|
||||
workaround auto-detection (which decides whether shipped Linux builds get
|
||||
WEBKIT_DISABLE_COMPOSITING_MODE) could regress silently. This wrapper rides
|
||||
the standard "Tests (backend + frontend)" gate instead of needing its own
|
||||
workflow step. Covers the #961 follow-up too: the build-time
|
||||
.bundled-webkitgtk-version marker must beat the host's pkg-config answer.
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
_REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
_SCRIPT = os.path.join(_REPO, "frontend", "src-tauri", "appimage", "AppRun.test.sh")
|
||||
|
||||
|
||||
@pytest.mark.skipif(shutil.which("bash") is None, reason="bash not available")
|
||||
def test_apprun_shell_suite_passes():
|
||||
proc = subprocess.run(
|
||||
["bash", _SCRIPT], capture_output=True, text=True, timeout=120,
|
||||
)
|
||||
assert proc.returncode == 0, (
|
||||
f"AppRun.test.sh failed (exit {proc.returncode}):\n"
|
||||
f"{proc.stdout}\n{proc.stderr}"
|
||||
)
|
||||
assert "0 fail" in proc.stdout
|
||||
@@ -0,0 +1,239 @@
|
||||
"""Generic OpenAI-compatible ASR backend (#877) — a path to Qwen3-ASR,
|
||||
FunASR/SenseVoice self-hosted servers, or OpenAI's own Whisper API, today,
|
||||
without waiting on transformers to ship a direct Qwen3-ASR integration.
|
||||
|
||||
settings_store backed by in-memory dicts, OpenAI client faked at the SDK
|
||||
boundary (no network) — house convention, same as test_llm_providers_router.py:
|
||||
direct handler calls, no TestClient, so the loopback auth guard isn't in play.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
|
||||
|
||||
os.environ.setdefault("OMNIVOICE_MODEL", "test")
|
||||
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
|
||||
|
||||
_HAS_OPENAI = __import__("importlib").util.find_spec("openai") is not None
|
||||
pytestmark = pytest.mark.skipif(not _HAS_OPENAI, reason="openai package not installed")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ss(monkeypatch):
|
||||
"""services.settings_store, resolved fresh (no module-level import — see
|
||||
asr_mod's docstring for why staleness across sys.modules reimports is a
|
||||
real risk in this suite) and patched to in-memory dicts (no SQLite)."""
|
||||
from services import settings_store as _ss
|
||||
|
||||
text: dict[str, str] = {}
|
||||
secrets: dict[str, str] = {}
|
||||
monkeypatch.setattr(_ss, "get_text", lambda k, default=None: text.get(k, default))
|
||||
monkeypatch.setattr(_ss, "set_text", lambda k, v: text.__setitem__(k, v))
|
||||
monkeypatch.setattr(_ss, "get_secret", lambda n: secrets.get(n))
|
||||
monkeypatch.setattr(
|
||||
_ss, "set_secret", lambda n, v: secrets.__setitem__(n, v) if v else secrets.pop(n, None)
|
||||
)
|
||||
monkeypatch.setattr(_ss, "list_secret_names", lambda: list(secrets))
|
||||
return _ss
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def asr_mod(ss, monkeypatch):
|
||||
"""services.asr_backend with settings_store in-memory (no SQLite).
|
||||
|
||||
Resolved via importlib.import_module INSIDE the fixture (not a top-level
|
||||
`import` in this file) so it's the module object actually live in
|
||||
sys.modules at test-run time — other test files in this ~2400-test suite
|
||||
pop+reimport shared service modules (services.model_manager,
|
||||
services.tts_backend), and a module-level import captured once at file
|
||||
COLLECTION time can go stale by the time an individual test in this file
|
||||
finally runs, hours of test-order later. A collection-time reference
|
||||
calling .set_text() and a fixture-time reference reading via .get_text()
|
||||
can silently be two different module objects — the write and the read
|
||||
land in different in-memory dicts, and the test fails with no obvious
|
||||
cause. Every test below takes `ss` as a fixture (not a module-level
|
||||
`from services import settings_store`) for the same reason.
|
||||
"""
|
||||
for var in ("ASR_OPENAI_COMPAT_BASE_URL", "ASR_OPENAI_COMPAT_MODEL", "ASR_OPENAI_COMPAT_API_KEY"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
import importlib
|
||||
return importlib.import_module("services.asr_backend")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def settings_mod(asr_mod):
|
||||
"""api.routers.settings sharing the same monkeypatched settings_store."""
|
||||
import importlib
|
||||
return importlib.import_module("api.routers.settings")
|
||||
|
||||
|
||||
def _fake_openai_transcribe(monkeypatch, *, verbose_ok=True, response=None, raise_exc=None):
|
||||
"""Fake openai.OpenAI whose audio.transcriptions.create() either returns
|
||||
a canned response or raises. verbose_ok=False simulates a minimal server
|
||||
that rejects response_format="verbose_json" on the first call, forcing
|
||||
the plain-json fallback."""
|
||||
captured_kwargs = []
|
||||
calls = []
|
||||
|
||||
class _FakeClient:
|
||||
def __init__(self, **kwargs):
|
||||
captured_kwargs.append(kwargs)
|
||||
self.audio = types.SimpleNamespace(
|
||||
transcriptions=types.SimpleNamespace(create=self._create)
|
||||
)
|
||||
|
||||
def _create(self, **kw):
|
||||
calls.append(kw)
|
||||
if raise_exc is not None:
|
||||
raise raise_exc
|
||||
if kw.get("response_format") == "verbose_json" and not verbose_ok:
|
||||
raise RuntimeError("response_format not supported")
|
||||
return response
|
||||
|
||||
import openai
|
||||
monkeypatch.setattr(openai, "OpenAI", _FakeClient)
|
||||
return captured_kwargs, calls
|
||||
|
||||
|
||||
# ── is_available() gating ───────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_unavailable_without_base_url(asr_mod):
|
||||
ok, msg = asr_mod.OpenAICompatASRBackend.is_available()
|
||||
assert ok is False
|
||||
assert "Settings" in msg
|
||||
|
||||
|
||||
def test_available_once_base_url_configured(asr_mod, ss):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
ok, _ = asr_mod.OpenAICompatASRBackend.is_available()
|
||||
assert ok is True
|
||||
|
||||
|
||||
# ── response adaptation ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_transcribe_adapts_verbose_json_segments(asr_mod, ss, monkeypatch, tmp_path):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
|
||||
class _Seg:
|
||||
def model_dump(self):
|
||||
return {"text": "hello world", "start": 0.0, "end": 1.5}
|
||||
|
||||
resp = types.SimpleNamespace(segments=[_Seg()], language="en")
|
||||
_fake_openai_transcribe(monkeypatch, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ") # content is never read by the fake client
|
||||
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert out["language"] == "en"
|
||||
assert out["segments"] == [{"text": "hello world", "start": 0.0, "end": 1.5, "words": []}]
|
||||
assert out["chunks"] == [{"text": "hello world", "timestamp": (0.0, 1.5)}]
|
||||
|
||||
|
||||
def test_transcribe_falls_back_to_plain_text_when_verbose_json_rejected(asr_mod, ss, monkeypatch, tmp_path):
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
resp = types.SimpleNamespace(text="plain text only", segments=None, language=None)
|
||||
_captured, calls = _fake_openai_transcribe(monkeypatch, verbose_ok=False, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert len(calls) == 2 # verbose_json attempt, then the plain fallback
|
||||
assert calls[0]["response_format"] == "verbose_json"
|
||||
assert calls[1]["response_format"] == "json"
|
||||
assert out["segments"] == [{"text": "plain text only", "start": 0.0, "end": None, "words": []}]
|
||||
assert out["language"] == "en" # default when the server doesn't report one
|
||||
|
||||
|
||||
def test_transcribe_network_failure_does_not_leak_raw_exception(asr_mod, ss, monkeypatch, tmp_path):
|
||||
"""Mirrors the #977 convention: a raw SDK/httpx exception must never reach
|
||||
the caller unformatted — only a clean, actionable RuntimeError."""
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
_fake_openai_transcribe(monkeypatch, raise_exc=ConnectionError("connection refused"))
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
with pytest.raises(RuntimeError) as ei:
|
||||
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
msg = str(ei.value)
|
||||
assert "localhost:8080" in msg
|
||||
assert "ConnectionError" in msg
|
||||
|
||||
|
||||
def test_client_disables_sdk_retries(asr_mod, ss, monkeypatch, tmp_path):
|
||||
"""max_retries=0 — mirrors llm_skills.resolve_skill_client: a slow/rate-
|
||||
limited server retrying inside the SDK would blow past the caller's own
|
||||
bounded timeout expectation for a single transcribe call."""
|
||||
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
|
||||
resp = types.SimpleNamespace(text="ok", segments=None, language="en")
|
||||
captured_kwargs, _ = _fake_openai_transcribe(monkeypatch, response=resp)
|
||||
|
||||
audio = tmp_path / "seg.wav"
|
||||
audio.write_bytes(b"RIFF....WAVEfmt ")
|
||||
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
|
||||
assert captured_kwargs[0]["max_retries"] == 0
|
||||
|
||||
|
||||
# ── settings endpoints ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_get_default_empty(settings_mod):
|
||||
st = settings_mod.get_asr_openai_compat()
|
||||
assert st == {"base_url": "", "model": "whisper-1", "has_key": False}
|
||||
|
||||
|
||||
def test_put_persists_and_never_echoes_the_key(settings_mod):
|
||||
st = settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(
|
||||
base_url="http://localhost:8080/v1/", model="qwen3-asr", api_key="sk-test-123",
|
||||
)
|
||||
)
|
||||
assert st["base_url"] == "http://localhost:8080/v1" # trailing slash trimmed
|
||||
assert st["model"] == "qwen3-asr"
|
||||
assert st["has_key"] is True
|
||||
assert "sk-test-123" not in str(st) # the key never round-trips
|
||||
|
||||
st2 = settings_mod.get_asr_openai_compat()
|
||||
assert st2 == st
|
||||
|
||||
|
||||
def test_empty_api_key_clears_it(settings_mod):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(api_key="sk-test-123")
|
||||
)
|
||||
assert settings_mod.get_asr_openai_compat()["has_key"] is True
|
||||
|
||||
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key=""))
|
||||
assert settings_mod.get_asr_openai_compat()["has_key"] is False
|
||||
|
||||
|
||||
def test_none_fields_leave_existing_values_unchanged(settings_mod):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="http://localhost:8080/v1", model="qwen3-asr")
|
||||
)
|
||||
# A save that only touches api_key must not clobber base_url/model.
|
||||
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key="sk-abc"))
|
||||
st = settings_mod.get_asr_openai_compat()
|
||||
assert st["base_url"] == "http://localhost:8080/v1"
|
||||
assert st["model"] == "qwen3-asr"
|
||||
assert st["has_key"] is True
|
||||
|
||||
|
||||
def test_rejects_a_base_url_without_scheme(settings_mod):
|
||||
from fastapi import HTTPException
|
||||
with pytest.raises(HTTPException):
|
||||
settings_mod.set_asr_openai_compat(
|
||||
settings_mod._ASROpenAICompatBody(base_url="localhost:8080/v1")
|
||||
)
|
||||
|
||||
|
||||
def test_registered_in_backend_list(asr_mod):
|
||||
assert "openai-compat-asr" in asr_mod._REGISTRY
|
||||
assert asr_mod._REGISTRY["openai-compat-asr"] is asr_mod.OpenAICompatASRBackend
|
||||
assert "openai-compat-asr" in asr_mod._INSTALL_HINTS
|
||||
@@ -0,0 +1,345 @@
|
||||
"""Issue #312 class — dub generation and batch TTS must honor the active
|
||||
engine selection (Settings → Engines) instead of silently falling back to
|
||||
OmniVoice via services.model_manager.get_model(), and must refuse — with an
|
||||
actionable error naming alternatives — instead of mis-cloning when the
|
||||
active engine can't do reference-audio voice cloning.
|
||||
|
||||
Covers:
|
||||
- `cloning_capable_engine_ids()` excludes the fixed-preset-voice engines
|
||||
(kittentts, supertonic3, sherpa-onnx) and includes the cloning ones.
|
||||
- /dub/generate: a non-cloning active engine fails the whole job with one
|
||||
actionable message (never falls back to OmniVoice, never mis-clones
|
||||
per segment).
|
||||
- /dub/generate: a cloning-capable non-OmniVoice active engine actually
|
||||
runs the request (proves the engine selection is honored, not ignored).
|
||||
- batch: an unpinned voice_id runs fine on a non-cloning active engine;
|
||||
a pinned voice_id on the same engine fails fast, before any TTS runs.
|
||||
- `applies_own_mastering` still skips the shared mastering chain for both
|
||||
pipelines (mirrors test_generate_engine.py's coverage of the same knob
|
||||
for /generate).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import importlib
|
||||
import os
|
||||
|
||||
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
|
||||
|
||||
import pytest
|
||||
import torch
|
||||
from fastapi import HTTPException
|
||||
|
||||
from schemas.requests import DubRequest, DubSegment
|
||||
|
||||
|
||||
def _tts_mod():
|
||||
"""Resolve services.tts_backend at RUN time — see test_generate_engine.py's
|
||||
docstring for why (sys.modules pre-pollution across the collected suite)."""
|
||||
return importlib.import_module("services.tts_backend")
|
||||
|
||||
|
||||
def _make_fake_engine(engine_id, *, supports_cloning=True, available=True,
|
||||
own_mastering=False, gpu_compat=("cpu",)):
|
||||
tb = _tts_mod()
|
||||
# Class-body assignment can't read the same name from the enclosing
|
||||
# function scope (class bodies don't close over locals) — alias first,
|
||||
# matching test_generate_engine.py's _make_fake_engine convention.
|
||||
_cloning, _mastering, _compat = supports_cloning, own_mastering, gpu_compat
|
||||
|
||||
class _FakeEngine(tb.TTSBackend):
|
||||
id = engine_id
|
||||
display_name = f"Fake {engine_id} (test)"
|
||||
supports_cloning = _cloning
|
||||
applies_own_mastering = _mastering
|
||||
gpu_compat = _compat
|
||||
calls: list = []
|
||||
|
||||
@property
|
||||
def sample_rate(self) -> int:
|
||||
return 24000
|
||||
|
||||
@property
|
||||
def supported_languages(self) -> list[str]:
|
||||
return ["multi"]
|
||||
|
||||
@classmethod
|
||||
def is_available(cls):
|
||||
if available:
|
||||
return True, "ready"
|
||||
return False, "fake engine deliberately unavailable (test)"
|
||||
|
||||
def generate(self, text, **kw) -> torch.Tensor:
|
||||
type(self).calls.append((text, kw))
|
||||
return torch.zeros(1, 24000)
|
||||
|
||||
return _FakeEngine
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_registry(monkeypatch):
|
||||
"""Register a fake engine in the REAL registry so resolve_generation_backend
|
||||
exercises the actual lookup/is_available/routing/cloning chain, not a stub.
|
||||
Resets the MM2-01 active-backend cache before/after (see
|
||||
tests/test_mm2_lifecycle.py's convention) so one test's cached instance
|
||||
can't leak into the next."""
|
||||
tb = _tts_mod()
|
||||
tb.reset_active_backend()
|
||||
registered: list[str] = []
|
||||
|
||||
def _register(engine_id, **kw):
|
||||
cls = _make_fake_engine(engine_id, **kw)
|
||||
tb._REGISTRY[engine_id] = cls
|
||||
registered.append(engine_id)
|
||||
return cls
|
||||
|
||||
yield _register
|
||||
|
||||
tb.reset_active_backend()
|
||||
for engine_id in registered:
|
||||
tb._REGISTRY.pop(engine_id, None)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def no_omnivoice_model_manager(monkeypatch):
|
||||
"""Fail loudly if resolution falls back to OmniVoice's get_model() path."""
|
||||
import services.model_manager as mm
|
||||
|
||||
async def _boom():
|
||||
raise AssertionError(
|
||||
"services.model_manager.get_model() was called — engine "
|
||||
"selection was silently ignored (#312 class)"
|
||||
)
|
||||
|
||||
monkeypatch.setattr(mm, "get_model", _boom)
|
||||
|
||||
|
||||
# ── cloning_capable_engine_ids() ────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_cloning_capable_engine_ids_excludes_fixed_voice_engines():
|
||||
tb = _tts_mod()
|
||||
ids = set(tb.cloning_capable_engine_ids())
|
||||
assert ids.isdisjoint({"kittentts", "supertonic3", "sherpa-onnx"})
|
||||
assert {"omnivoice", "voxcpm2", "cosyvoice", "gpt-sovits"}.issubset(ids)
|
||||
|
||||
|
||||
def test_cloning_capable_engine_ids_excludes_model_dependent_adapters():
|
||||
# MLXAudioBackend.supports_cloning is an instance @property (only some of
|
||||
# its curated models can clone) — a class-level getattr() returns the
|
||||
# property descriptor itself, which is truthy, so a naive check would
|
||||
# always recommend "mlx-audio" even when the configured model is Kokoro
|
||||
# (can't clone). Must be excluded from the suggestion list rather than
|
||||
# falsely recommended.
|
||||
tb = _tts_mod()
|
||||
assert isinstance(
|
||||
vars(tb.MLXAudioBackend).get("supports_cloning"), property
|
||||
), "this test assumes MLXAudioBackend.supports_cloning is a property"
|
||||
assert "mlx-audio" not in set(tb.cloning_capable_engine_ids())
|
||||
|
||||
|
||||
# ── /dub/generate/{job_id} ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dub_job_env(monkeypatch, tmp_path):
|
||||
"""Minimal hermetic environment for `dg.dub_generate()` — same stub set as
|
||||
test_smart_fit_generate.py's fixture, but WITHOUT patching
|
||||
resolve_generation_backend, so the real registry + capability gate run."""
|
||||
import api.routers.dub_generate as dg
|
||||
|
||||
job = {"duration": 2.0, "dubbed_tracks": {}, "speaker_clones": {}}
|
||||
job_dir = tmp_path / "jobX"
|
||||
job_dir.mkdir()
|
||||
|
||||
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
|
||||
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
|
||||
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
|
||||
monkeypatch.setattr(
|
||||
dg, "dub_seg_path",
|
||||
lambda job_id, seg_id: str(job_dir / f"seg_{seg_id}.wav"),
|
||||
)
|
||||
monkeypatch.setattr(dg, "rvc_is_enabled", lambda: False)
|
||||
monkeypatch.setattr(dg, "embed_watermark", lambda wav, sr: wav)
|
||||
monkeypatch.setattr(dg, "apply_mastering", lambda a, sample_rate=None: a)
|
||||
monkeypatch.setattr(dg, "get_effect_chain", lambda preset: None)
|
||||
monkeypatch.setattr(dg, "apply_effects_chain", lambda a, **k: a)
|
||||
monkeypatch.setattr(dg, "normalize_audio", lambda a, target_dBFS=None: a)
|
||||
|
||||
class _StubTaskManager:
|
||||
def is_cancelled(self, task_id):
|
||||
return False
|
||||
|
||||
async def add_task(self, task_id, task_type, func, *args, **kwargs):
|
||||
async for _ in func(*args):
|
||||
pass
|
||||
|
||||
monkeypatch.setattr(dg, "task_manager", _StubTaskManager())
|
||||
return dg, job
|
||||
|
||||
|
||||
def _one_seg_request():
|
||||
return DubRequest(
|
||||
segments=[DubSegment(start=0.0, end=1.0, text="hola")],
|
||||
segment_ids=["0"], language="Auto", language_code="es", num_step=4,
|
||||
)
|
||||
|
||||
|
||||
def test_dub_generate_fails_fast_for_non_cloning_engine(
|
||||
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
|
||||
):
|
||||
"""Active engine can't clone → the job fails once, up front, with an
|
||||
actionable message naming alternatives — never a silent OmniVoice run."""
|
||||
dg, job = dub_job_env
|
||||
fake_registry("fake-nonclone", supports_cloning=False)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-nonclone")
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
|
||||
|
||||
assert exc_info.value.status_code == 400
|
||||
detail = exc_info.value.detail
|
||||
assert "fake-nonclone" in detail
|
||||
assert "voice cloning" in detail
|
||||
assert "omnivoice" in detail # names a real alternative
|
||||
|
||||
|
||||
def test_dub_generate_uses_selected_cloning_engine_not_omnivoice(
|
||||
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
|
||||
):
|
||||
"""A cloning-capable non-OmniVoice engine actually runs the segment."""
|
||||
dg, job = dub_job_env
|
||||
fake = fake_registry("fake-clone", supports_cloning=True)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-clone")
|
||||
|
||||
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
|
||||
|
||||
assert len(fake.calls) == 1
|
||||
assert fake.calls[0][0] == "hola"
|
||||
assert "es" in job["dubbed_tracks"]
|
||||
|
||||
|
||||
def test_dub_generate_respects_applies_own_mastering(
|
||||
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
|
||||
):
|
||||
dg, job = dub_job_env
|
||||
fake = fake_registry("fake-studio", supports_cloning=True, own_mastering=True)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-studio")
|
||||
|
||||
mastering_calls = []
|
||||
monkeypatch.setattr(
|
||||
dg, "apply_mastering",
|
||||
lambda a, sample_rate=None: mastering_calls.append(1) or a,
|
||||
)
|
||||
|
||||
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
|
||||
|
||||
assert len(fake.calls) == 1
|
||||
assert mastering_calls == [] # studio engine's own mastering is not double-applied
|
||||
|
||||
|
||||
# ── batch ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def batch_job_env(monkeypatch, tmp_path):
|
||||
import api.routers.batch as b
|
||||
|
||||
monkeypatch.setattr(b, "DATA_DIR", str(tmp_path))
|
||||
|
||||
async def _fake_run_transcribe_guarded(pool, fn, what=None):
|
||||
# Bypass real ASR entirely — the engine-selection gate under test
|
||||
# runs right after transcription, before translate/generate.
|
||||
return (
|
||||
[{"id": "s0", "start": 0.0, "end": 1.0, "text": "hola",
|
||||
"text_original": "hola"}],
|
||||
"en",
|
||||
)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"services.asr_backend.run_transcribe_guarded",
|
||||
_fake_run_transcribe_guarded,
|
||||
)
|
||||
|
||||
def _fake_subprocess_run(cmd, *a, **kw):
|
||||
class _Result:
|
||||
stdout = b""
|
||||
stderr = b"Duration: 00:00:02.00, start: 0.000000, bitrate: 1000 kb/s\n"
|
||||
|
||||
return _Result()
|
||||
|
||||
monkeypatch.setattr("subprocess.run", _fake_subprocess_run)
|
||||
monkeypatch.setattr("services.ffmpeg_utils.find_ffmpeg", lambda: "ffmpeg")
|
||||
|
||||
def _make_job(job_id, *, voice_id=None):
|
||||
return {
|
||||
"id": job_id,
|
||||
"status": "running",
|
||||
"filename": "in.mp4",
|
||||
"video_path": str(tmp_path / "in.mp4"),
|
||||
"langs": ["en"], # == source_lang → translation stage is a no-op
|
||||
"voice_id": voice_id,
|
||||
"preserve_bg": True,
|
||||
"created_at": 0.0,
|
||||
"started_at": None,
|
||||
"finished_at": None,
|
||||
"error": None,
|
||||
"progress": None,
|
||||
}
|
||||
|
||||
return b, _make_job
|
||||
|
||||
|
||||
def test_batch_unpinned_voice_succeeds_on_noncloning_engine(
|
||||
batch_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
|
||||
):
|
||||
"""No voice_id pinned → any active engine (cloning-capable or not) is fine."""
|
||||
b, make_job = batch_job_env
|
||||
fake = fake_registry("fake-batch-nonclone", supports_cloning=False)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-nonclone")
|
||||
|
||||
job = make_job("jobA", voice_id=None)
|
||||
asyncio.run(b._run_batch_pipeline("jobA", job))
|
||||
|
||||
assert len(fake.calls) == 1
|
||||
assert "en" in job.get("outputs", {})
|
||||
|
||||
|
||||
def test_batch_pinned_voice_fails_fast_on_noncloning_engine(
|
||||
batch_job_env, fake_registry, monkeypatch,
|
||||
):
|
||||
"""voice_id pinned + a non-cloning active engine → fail before any TTS
|
||||
runs, with the same actionable message shape as the dub gate."""
|
||||
b, make_job = batch_job_env
|
||||
fake = fake_registry("fake-batch-nonclone2", supports_cloning=False)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-nonclone2")
|
||||
|
||||
job = make_job("jobB", voice_id="some-voice-id")
|
||||
with pytest.raises(ValueError) as exc_info:
|
||||
asyncio.run(b._run_batch_pipeline("jobB", job))
|
||||
|
||||
detail = str(exc_info.value)
|
||||
assert "fake-batch-nonclone2" in detail
|
||||
assert "voice cloning" in detail
|
||||
assert not fake.calls # never reached generate
|
||||
|
||||
|
||||
def test_batch_respects_applies_own_mastering(
|
||||
batch_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
|
||||
):
|
||||
b, make_job = batch_job_env
|
||||
fake = fake_registry("fake-batch-studio", supports_cloning=True, own_mastering=True)
|
||||
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-studio")
|
||||
|
||||
mastering_calls = []
|
||||
import services.audio_dsp as audio_dsp
|
||||
monkeypatch.setattr(
|
||||
audio_dsp, "apply_mastering",
|
||||
lambda a, sample_rate=None: mastering_calls.append(1) or a,
|
||||
)
|
||||
|
||||
job = make_job("jobC", voice_id=None)
|
||||
asyncio.run(b._run_batch_pipeline("jobC", job))
|
||||
|
||||
assert len(fake.calls) == 1
|
||||
assert mastering_calls == []
|
||||
@@ -124,14 +124,32 @@ class _RefCapturingModel:
|
||||
return [torch.full((1, int(0.5 * SR)), 0.1)]
|
||||
|
||||
|
||||
class _FakeBackend:
|
||||
"""Adapts the list-returning fake model above to the TTSBackend.generate()
|
||||
contract (a single tensor, not a list) that resolve_generation_backend()
|
||||
now hands dub_generate.py (issue #312 class)."""
|
||||
|
||||
applies_own_mastering = False
|
||||
|
||||
def __init__(self, model):
|
||||
self._model = model
|
||||
|
||||
@property
|
||||
def sample_rate(self):
|
||||
return self._model.sampling_rate
|
||||
|
||||
def generate(self, *a, **kw):
|
||||
return self._model.generate(*a, **kw)[0]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def patched_generate(monkeypatch, tmp_path):
|
||||
import api.routers.dub_generate as dg
|
||||
|
||||
model = _RefCapturingModel()
|
||||
|
||||
async def _fake_get_model():
|
||||
return model
|
||||
async def _fake_resolve_generation_backend(**kwargs):
|
||||
return _FakeBackend(model)
|
||||
|
||||
job = {
|
||||
"duration": 6.0,
|
||||
@@ -148,7 +166,7 @@ def patched_generate(monkeypatch, tmp_path):
|
||||
job_dir = tmp_path / "jobX"
|
||||
job_dir.mkdir()
|
||||
|
||||
monkeypatch.setattr(dg, "get_model", _fake_get_model)
|
||||
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
|
||||
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
|
||||
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
|
||||
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
|
||||
|
||||
@@ -113,6 +113,66 @@ def test_tts_unknown_backend_raises():
|
||||
tts_backend.get_backend_class("not-a-real-one")
|
||||
|
||||
|
||||
# ── #981 — MLX-Audio curated-model selection ─────────────────────────────
|
||||
#
|
||||
# MLXAudioBackend.__init__ used to resolve its active model ONLY from
|
||||
# OMNIVOICE_MLX_AUDIO_MODEL, invisible to Settings and unchangeable without
|
||||
# restarting the process with an env var set. It must now mirror
|
||||
# active_backend_id()'s env > prefs > default resolution.
|
||||
|
||||
|
||||
def test_mlx_audio_model_id_resolves_via_prefs(monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
|
||||
_prefs.set_("mlx_audio_model_id", "outetts")
|
||||
be = tts_backend.MLXAudioBackend()
|
||||
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["outetts"]
|
||||
|
||||
|
||||
def test_mlx_audio_model_id_env_overrides_prefs(monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
_prefs.set_("mlx_audio_model_id", "outetts")
|
||||
monkeypatch.setenv("OMNIVOICE_MLX_AUDIO_MODEL", "csm")
|
||||
be = tts_backend.MLXAudioBackend()
|
||||
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["csm"]
|
||||
|
||||
|
||||
def test_mlx_audio_model_id_defaults_to_kokoro(monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
|
||||
be = tts_backend.MLXAudioBackend()
|
||||
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["kokoro"]
|
||||
|
||||
|
||||
def test_get_active_tts_backend_reconstructs_on_mlx_model_switch(monkeypatch, tmp_path):
|
||||
"""A curated-model-only change (same backend id 'mlx-audio') must
|
||||
invalidate the cached instance too — otherwise picking a different
|
||||
curated model in Settings has no effect until an app restart."""
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
|
||||
monkeypatch.delenv("OMNIVOICE_TTS_BACKEND", raising=False)
|
||||
_prefs.set_("tts_backend", "mlx-audio")
|
||||
_prefs.set_("mlx_audio_model_id", "kokoro")
|
||||
tts_backend.reset_active_backend()
|
||||
try:
|
||||
be1 = tts_backend.get_active_tts_backend()
|
||||
assert be1._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["kokoro"]
|
||||
# Same instance on a repeat call with nothing changed (still cached).
|
||||
assert tts_backend.get_active_tts_backend() is be1
|
||||
|
||||
_prefs.set_("mlx_audio_model_id", "outetts")
|
||||
be2 = tts_backend.get_active_tts_backend()
|
||||
assert be2 is not be1
|
||||
assert be2._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["outetts"]
|
||||
finally:
|
||||
tts_backend.reset_active_backend()
|
||||
_prefs.set_("tts_backend", "omnivoice")
|
||||
|
||||
|
||||
# ── ASR ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -134,6 +194,53 @@ def test_asr_env_override(monkeypatch):
|
||||
assert asr_backend.active_backend_id() == "pytorch-whisper"
|
||||
|
||||
|
||||
# ── ASR selection resolution (Settings → Engines ASR picker) ────────────────
|
||||
# Same env > prefs > auto-detect contract as TTS. The env var MUST keep
|
||||
# winning so existing `OMNIVOICE_ASR_BACKEND` pins don't change behavior now
|
||||
# that the Settings picker writes the prefs key.
|
||||
|
||||
|
||||
def test_asr_active_backend_prefs_fallback(monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
|
||||
_prefs.set_("asr_backend", "moonshine")
|
||||
assert asr_backend.active_backend_id() == "moonshine"
|
||||
# Env var must beat prefs.
|
||||
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "pytorch-whisper")
|
||||
assert asr_backend.active_backend_id() == "pytorch-whisper"
|
||||
|
||||
|
||||
def test_asr_auto_detects_when_no_env_no_prefs(monkeypatch, tmp_path):
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
|
||||
assert asr_backend.active_backend_id() in {
|
||||
"whisperx", "faster-whisper", "mlx-whisper", "pytorch-whisper",
|
||||
}
|
||||
|
||||
|
||||
def test_get_active_asr_backend_follows_prefs_switch_without_restart(monkeypatch, tmp_path):
|
||||
"""#981 class (fixed on the TTS side): a Settings pick must take effect on
|
||||
the next transcribe, not after an app restart. get_active_asr_backend()
|
||||
re-resolves the id per call, so a prefs write switches immediately."""
|
||||
from core import prefs as _prefs
|
||||
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
|
||||
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
|
||||
_prefs.set_("asr_backend", "pytorch-whisper")
|
||||
assert isinstance(
|
||||
asr_backend.get_active_asr_backend(), asr_backend.PyTorchWhisperBackend)
|
||||
_prefs.set_("asr_backend", "moonshine")
|
||||
assert isinstance(
|
||||
asr_backend.get_active_asr_backend(), asr_backend.MoonshineASRBackend)
|
||||
|
||||
|
||||
def test_asr_unknown_backend_raises(monkeypatch):
|
||||
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "not-a-real-asr")
|
||||
with pytest.raises(ValueError):
|
||||
asr_backend.get_active_asr_backend()
|
||||
|
||||
|
||||
# ── LLM ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -238,3 +345,160 @@ def test_hf_retry_is_single_shot():
|
||||
with pytest.raises(RuntimeError):
|
||||
tts_backend._retry_once_with_fresh_hf_client(loader, what="test")
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
# ── #977: MLX-Audio Kokoro language-code resolution ─────────────────────────
|
||||
# Kokoro's own vendored pipeline (mlx_audio.tts.models.kokoro.pipeline)
|
||||
# hard-asserts `lang_code` against a fixed single-letter table. The old code
|
||||
# blindly truncated a full language name — "Dutch"[:2].lower() == "du" — into
|
||||
# that assert, crashing with an unreadable `(lang_code, LANG_CODES)` repr
|
||||
# instead of a clean error. The resolution tests need the real mlx-audio
|
||||
# package (Apple-Silicon-only) since they validate against ITS installed
|
||||
# table, never a hardcoded guess; they skip cleanly where mlx-audio isn't
|
||||
# installed (every non-macOS-ARM CI runner).
|
||||
|
||||
|
||||
def test_mlx_audio_kokoro_resolves_supported_language_names():
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
resolve = tts_backend.resolve_kokoro_lang_code
|
||||
assert resolve("English") == "a"
|
||||
assert resolve("Spanish") == "e"
|
||||
assert resolve("French") == "f"
|
||||
assert resolve("Hindi") == "h"
|
||||
assert resolve("Italian") == "i"
|
||||
assert resolve("Portuguese") == "p"
|
||||
assert resolve("Japanese") == "j"
|
||||
assert resolve("Chinese") == "z"
|
||||
# Some callers may already pass an ISO code — those resolve unchanged
|
||||
# through Kokoro's own ALIASES table, not just our full-name map.
|
||||
assert resolve("es") == "e"
|
||||
assert resolve("en-gb") == "b"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("language", ["Dutch", "German"])
|
||||
def test_mlx_audio_kokoro_rejects_unsupported_language_cleanly(language):
|
||||
# The literal #977 report case ("Dutch") plus one more Kokoro doesn't
|
||||
# support ("German") — neither's first two letters happen to alias to a
|
||||
# valid Kokoro code, so both used to crash.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
with pytest.raises(ValueError) as ei:
|
||||
tts_backend.resolve_kokoro_lang_code(language)
|
||||
msg = str(ei.value)
|
||||
assert language in msg
|
||||
assert "Kokoro" in msg
|
||||
assert "English" in msg # names what Kokoro DOES support
|
||||
|
||||
|
||||
def test_mlx_audio_generate_rejects_unsupported_kokoro_language_before_calling_model():
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._model_id = backend.CURATED_MODELS["kokoro"]
|
||||
backend._ensure_loaded = lambda: None # never actually load the model
|
||||
|
||||
def _boom_generate(**kw):
|
||||
raise AssertionError("model.generate() must not run for a rejected language")
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_boom_generate)
|
||||
|
||||
with pytest.raises(ValueError, match="Dutch"):
|
||||
backend.generate("hello", language="Dutch")
|
||||
|
||||
|
||||
def test_mlx_audio_generate_passes_ref_text_through_for_cloning():
|
||||
# #1012/#1013: MLXAudioBackend.generate() read voice/ref_audio/language/
|
||||
# speed from kwargs but silently dropped ref_text — CSM (sesame.py) only
|
||||
# builds its cloning context when BOTH ref_audio and ref_text are
|
||||
# present, so cloning on CSM always raised an opaque
|
||||
# "IndexError: list index out of range" deep inside mlx-audio instead of
|
||||
# ever attempting the clone. Community-diagnosed with the exact fix.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", ref_audio="/tmp/ref.wav", ref_text="the reference line")
|
||||
|
||||
assert captured.get("ref_text") == "the reference line"
|
||||
assert captured.get("ref_audio") == "/tmp/ref.wav"
|
||||
|
||||
|
||||
def test_mlx_audio_generate_omits_ref_text_without_ref_audio():
|
||||
# ref_text alone (no ref_audio) means nothing to CSM's context builder —
|
||||
# don't pass a stray kwarg an engine that isn't cloning doesn't expect.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", ref_text="orphaned text, no audio")
|
||||
|
||||
assert "ref_text" not in captured
|
||||
|
||||
|
||||
def test_mlx_audio_generate_design_path_unaffected_without_any_ref():
|
||||
# Absorbed from community PR #1015 (MahdiHedhli) — the design/instruct
|
||||
# path (no ref_audio, no ref_text at all) must stay untouched by the
|
||||
# ref_text forwarding fix; neither kwarg may leak into the model call.
|
||||
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._ensure_loaded = lambda: None
|
||||
|
||||
captured = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
captured.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello")
|
||||
|
||||
assert "ref_text" not in captured
|
||||
assert "ref_audio" not in captured
|
||||
|
||||
|
||||
def test_mlx_audio_generate_auto_language_skips_lang_code_entirely():
|
||||
# Matches the "Auto" convention other engines in this file use
|
||||
# (OmniVoiceBackend.generate(), _run_backend_inference) — never resolved,
|
||||
# never forwarded as lang_code.
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._model_id = backend.CURATED_MODELS["kokoro"]
|
||||
backend._ensure_loaded = lambda: None
|
||||
seen_kwargs = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
seen_kwargs.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=[0.0, 0.0, 0.0, 0.0])])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", language="Auto")
|
||||
assert "lang_code" not in seen_kwargs
|
||||
|
||||
|
||||
def test_mlx_audio_generate_non_kokoro_model_ignores_kokoro_validation():
|
||||
# Qwen3-TTS (and CSM/Dia/Chatterbox/MeloTTS/OuteTTS) don't use Kokoro's
|
||||
# lang_code convention — a language Kokoro would reject must NOT be
|
||||
# rejected when a different curated model is active (#977 nuance).
|
||||
backend = tts_backend.MLXAudioBackend()
|
||||
backend._model_id = backend.CURATED_MODELS["qwen3-tts"]
|
||||
backend._ensure_loaded = lambda: None
|
||||
seen_kwargs = {}
|
||||
|
||||
def _fake_generate(**kw):
|
||||
seen_kwargs.update(kw)
|
||||
return iter([types.SimpleNamespace(audio=[0.0, 0.0, 0.0, 0.0])])
|
||||
|
||||
backend._model = types.SimpleNamespace(generate=_fake_generate)
|
||||
backend.generate("hello", language="Dutch") # must not raise
|
||||
assert seen_kwargs.get("lang_code") == "du"
|
||||
|
||||
@@ -143,6 +143,37 @@ def test_classify_socks_proxy_support_missing():
|
||||
assert failure.classify("ProxyError: connection refused by 10.0.0.1:8080") == ""
|
||||
|
||||
|
||||
def test_classify_ssl_handshake_failure():
|
||||
# #976: the exact error a Windows user behind a corporate/antivirus
|
||||
# TLS-inspecting proxy sees on every model install — the TCP connection
|
||||
# succeeds, but the handshake fails because the OS trusts the proxy's
|
||||
# re-signed CA and Python's bundled certifi list doesn't. A different
|
||||
# failure mode from #984's TCP-level "host unreachable" fix.
|
||||
reason = (
|
||||
"Install failed: Got: ConnectError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] "
|
||||
"ssl/tls alert handshake failure (_ssl.c:1016)"
|
||||
)
|
||||
assert failure.classify(reason) == "SSL_HANDSHAKE_FAILURE"
|
||||
evt = failure.build_failure(reason, stage="install", include_diagnostic=False)
|
||||
assert evt["docs_topic"] == "SSL_HANDSHAKE_FAILURE"
|
||||
assert evt["hint"], "the SSL-handshake class must carry an actionable hint"
|
||||
# A CERTIFICATE_VERIFY_FAILED-style message (the other common corporate-MITM
|
||||
# shape) must classify the same way.
|
||||
cert_reason = (
|
||||
"requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', "
|
||||
"port=443): Max retries exceeded with url: / (Caused by SSLError("
|
||||
"SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate "
|
||||
"verify failed: unable to get local issuer certificate')))"
|
||||
)
|
||||
assert failure.classify(cert_reason) == "SSL_HANDSHAKE_FAILURE"
|
||||
# append_hint is the raw-string surface (setup/download.py's install SSE) —
|
||||
# the detail keeps the real error AND gains the hint.
|
||||
out = failure.append_hint(reason)
|
||||
assert out.startswith(reason) and "truststore" in out
|
||||
# A plain, unrelated connection error must NOT be mislabelled as SSL.
|
||||
assert failure.classify("ConnectionError: connection refused") == ""
|
||||
|
||||
|
||||
def test_classify_generic_still_empty():
|
||||
# A genuinely unknown reason must still classify to "" (no false hint).
|
||||
assert failure.classify("some totally unrelated failure") == ""
|
||||
|
||||
@@ -139,6 +139,10 @@ def test_gpu_host_keeps_vram_guidance(monkeypatch):
|
||||
msg = _guidance_for(monkeypatch, "cuda")
|
||||
assert "VRAM-starved" in msg
|
||||
assert "set the engine to CPU" in msg
|
||||
# #939: the sibling ASR timeout guard already recommends Flush/Unload —
|
||||
# this message was missing it, forcing the maintainer to explain it
|
||||
# manually in every report instead of the error saying so upfront.
|
||||
assert "Flush" in msg
|
||||
|
||||
|
||||
def test_probe_failure_defaults_to_gpu_wording(monkeypatch):
|
||||
|
||||
@@ -14,7 +14,11 @@ import torch
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
|
||||
|
||||
from api.routers.generation import _sanitize_audio, _oom_friendly_reraise # noqa: E402
|
||||
from api.routers.generation import ( # noqa: E402
|
||||
_oom_friendly_reraise,
|
||||
_safe_exc_text,
|
||||
_sanitize_audio,
|
||||
)
|
||||
|
||||
|
||||
def test_sanitize_replaces_non_finite_with_silence():
|
||||
@@ -257,3 +261,41 @@ def test_config_failure_does_not_swallow_real_oom():
|
||||
with pytest.raises(RuntimeError) as ei:
|
||||
_oom_friendly_reraise(RuntimeError("CUDA error: out of memory"))
|
||||
assert "ran out of memory" in str(ei.value)
|
||||
|
||||
|
||||
# ── #977: generic exception formatters never leak a raw container repr ─────
|
||||
# mlx-audio's vendored Kokoro pipeline raises
|
||||
# `assert lang_code in LANG_CODES, (lang_code, LANG_CODES)` — an
|
||||
# AssertionError whose .args is a (str, dict) tuple. str(e) on that
|
||||
# interpolates the ENTIRE table straight into the user-facing 500 message
|
||||
# ("Underlying error: ('du', {'a': 'American English', ...})"). Any engine's
|
||||
# generate() can raise something shaped like this, not just Kokoro, so the
|
||||
# guard is class-level: _safe_exc_text() backs both of generation.py's
|
||||
# generic (catch-all) exception formatters.
|
||||
|
||||
|
||||
def test_safe_exc_text_plain_message_uses_house_style():
|
||||
err = RuntimeError("plain readable message")
|
||||
assert _safe_exc_text(err) == "RuntimeError: plain readable message"
|
||||
|
||||
|
||||
def test_safe_exc_text_container_args_do_not_leak_raw_repr():
|
||||
# Mirrors the exact #977 AssertionError shape.
|
||||
err = AssertionError(("du", {"a": "American English", "b": "British English"}))
|
||||
text = _safe_exc_text(err)
|
||||
assert text.startswith("AssertionError")
|
||||
assert "American English" not in text
|
||||
assert "{" not in text and "}" not in text
|
||||
assert "(" not in text and ")" not in text
|
||||
|
||||
|
||||
def test_unrecognized_error_catchall_does_not_leak_container_repr():
|
||||
# End-to-end through _oom_friendly_reraise's catch-all fallback (none of
|
||||
# the specific classifiers above it match an AssertionError like this).
|
||||
err = AssertionError(("du", {"a": "American English", "b": "British English"}))
|
||||
with pytest.raises(RuntimeError) as ei:
|
||||
_oom_friendly_reraise(err)
|
||||
msg = str(ei.value)
|
||||
assert "AssertionError" in msg
|
||||
assert "American English" not in msg
|
||||
assert "{" not in msg and "}" not in msg
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
"""
|
||||
Regression tests: the mastering pre-stage must not hide a reverb.
|
||||
|
||||
A hardcoded Reverb inside apply_mastering() used to bake echo into every
|
||||
non-raw synthesis regardless of the chosen effect preset (field reports of
|
||||
echoey voices; the podcast preset even promises "no reverb", and cinematic/
|
||||
warm got doubled reverb). Reverb is preset-declared only — these tests pin
|
||||
that contract.
|
||||
|
||||
Kept separate from test_effects_chain.py on purpose: that file skips
|
||||
entirely without pedalboard, while the data-shape guards here must always
|
||||
run. sys.path for backend imports is handled by tests/conftest.py.
|
||||
"""
|
||||
|
||||
import builtins
|
||||
import math
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
import torch
|
||||
|
||||
from services.audio_dsp import (
|
||||
EFFECT_PRESETS,
|
||||
MASTERING_CHAIN,
|
||||
apply_mastering,
|
||||
)
|
||||
|
||||
|
||||
def _stage_types(chain):
|
||||
return [fx["type"] for fx in chain]
|
||||
|
||||
|
||||
def _make_test_audio(duration_s=1.0, sample_rate=24000) -> torch.Tensor:
|
||||
"""Create a test audio tensor with a simple sine wave."""
|
||||
t = torch.linspace(0, duration_s, int(duration_s * sample_rate))
|
||||
return torch.sin(2 * math.pi * 440 * t).unsqueeze(0) # 440 Hz sine, mono
|
||||
|
||||
|
||||
class TestMasteringChainHasNoHiddenReverb:
|
||||
def test_mastering_chain_contains_no_reverb(self):
|
||||
"""The recurrence guard: nobody re-adds a reverb outside the preset system."""
|
||||
assert "reverb" not in _stage_types(MASTERING_CHAIN)
|
||||
|
||||
def test_mastering_chain_keeps_highpass_and_compressor(self):
|
||||
"""Removing the reverb must not gut the rest of the pre-stage."""
|
||||
types = _stage_types(MASTERING_CHAIN)
|
||||
assert "highpass" in types
|
||||
assert "compressor" in types
|
||||
|
||||
|
||||
class TestPresetReverbContract:
|
||||
@pytest.mark.parametrize("preset_id", ["broadcast", "podcast"])
|
||||
def test_no_reverb_presets_stay_reverb_free(self, preset_id):
|
||||
"""podcast's description literally promises "no reverb"."""
|
||||
assert "reverb" not in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
|
||||
|
||||
@pytest.mark.parametrize("preset_id", ["cinematic", "warm"])
|
||||
def test_user_chosen_reverb_survives(self, preset_id):
|
||||
"""Presets that deliberately declare reverb must keep it."""
|
||||
assert "reverb" in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
|
||||
|
||||
|
||||
class TestApplyMasteringFunctional:
|
||||
def test_returns_same_shape_and_device(self):
|
||||
pytest.importorskip("pedalboard")
|
||||
audio = _make_test_audio()
|
||||
result = apply_mastering(audio, sample_rate=24000)
|
||||
assert isinstance(result, torch.Tensor)
|
||||
assert result.shape == audio.shape
|
||||
assert result.device == audio.device
|
||||
|
||||
def test_no_echo_tail_bleeds_into_silence(self):
|
||||
"""A burst followed by silence must stay silent after mastering.
|
||||
|
||||
Fails with the old hidden Reverb (its tail rings past the burst);
|
||||
passes with highpass + compressor only.
|
||||
"""
|
||||
pytest.importorskip("pedalboard")
|
||||
sr = 24000
|
||||
burst = _make_test_audio(duration_s=0.25, sample_rate=sr)
|
||||
audio = torch.cat([burst, torch.zeros(1, sr)], dim=1) # + 1 s silence
|
||||
result = apply_mastering(audio, sample_rate=sr)
|
||||
# Skip 50 ms after the burst so the filters settle; a reverb tail is
|
||||
# far louder and longer than that.
|
||||
tail = result[:, burst.shape[1] + int(0.05 * sr):]
|
||||
assert tail.abs().max().item() < 1e-3
|
||||
|
||||
def test_passthrough_when_pedalboard_missing(self, monkeypatch):
|
||||
"""Graceful degradation: no pedalboard, audio returned unmodified."""
|
||||
real_import = builtins.__import__
|
||||
|
||||
def no_pedalboard(name, *args, **kwargs):
|
||||
if name == "pedalboard" or name.startswith("pedalboard."):
|
||||
raise ImportError("pedalboard unavailable (simulated)")
|
||||
return real_import(name, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(builtins, "__import__", no_pedalboard)
|
||||
monkeypatch.delitem(sys.modules, "pedalboard", raising=False)
|
||||
audio = _make_test_audio()
|
||||
result = apply_mastering(audio, sample_rate=24000)
|
||||
assert torch.equal(result, audio)
|
||||
@@ -100,6 +100,45 @@ def test_list_loaded_asr_row_is_honest(monkeypatch):
|
||||
assert asr.get("note") # explains the disabled unload button
|
||||
|
||||
|
||||
def test_list_loaded_attributes_resident_tts_to_its_engine(monkeypatch):
|
||||
# Field report: OmniVoice stays resident in VRAM after switching to
|
||||
# voxcpm2, and the panel offered no hint it wasn't the routed engine.
|
||||
class _Model:
|
||||
_asr_pipe = object()
|
||||
def parameters(self): raise StopIteration
|
||||
monkeypatch.setattr(mm, "model", _Model())
|
||||
monkeypatch.setattr(mm, "_diar_pipeline", None)
|
||||
|
||||
# String-target setattr: other suites pop+reimport services.* modules
|
||||
# mid-run (see module docstring), so the collection-time `tb` alias can go
|
||||
# stale — patch the module object _active_tts_id late-imports at call time.
|
||||
monkeypatch.setattr("services.tts_backend.active_backend_id", lambda: "voxcpm2")
|
||||
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
|
||||
assert rows["tts"]["engine_id"] == "omnivoice"
|
||||
assert rows["tts"]["is_active_engine"] is False
|
||||
# ASR isn't competing with the TTS selection — must not be mislabeled.
|
||||
assert "is_active_engine" not in rows["asr"]
|
||||
|
||||
monkeypatch.setattr("services.tts_backend.active_backend_id", lambda: "omnivoice")
|
||||
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
|
||||
assert rows["tts"]["is_active_engine"] is True
|
||||
|
||||
|
||||
def test_list_loaded_attribution_failure_degrades(monkeypatch):
|
||||
# Attribution is advisory: a raising prefs layer must not break the
|
||||
# panel, just leave the active flag unknown.
|
||||
class _Model:
|
||||
_asr_pipe = None
|
||||
def parameters(self): raise StopIteration
|
||||
monkeypatch.setattr(mm, "model", _Model())
|
||||
monkeypatch.setattr(mm, "_diar_pipeline", None)
|
||||
def _boom():
|
||||
raise RuntimeError("prefs unavailable")
|
||||
monkeypatch.setattr("services.tts_backend.active_backend_id", _boom)
|
||||
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
|
||||
assert rows["tts"]["is_active_engine"] is None
|
||||
|
||||
|
||||
def test_facade_unload_unknown_raises():
|
||||
with pytest.raises(ValueError):
|
||||
_run(ml.unload("bogus"))
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Regression test for issue #974.
|
||||
|
||||
The nemo-parakeet install hint used to tell users to run
|
||||
`pip install nemo_toolkit[asr]` directly into OmniVoice's shared venv.
|
||||
nemo_toolkit[asr]==2.7.3 hard-pins transformers>=4.57,<4.58, which is
|
||||
UNSATISFIABLE alongside the app's own transformers>=5.3 requirement (needed
|
||||
by omnivoice/models/omnivoice.py for HiggsAudioV2TokenizerModel). A user who
|
||||
followed the old hint ended up with a backend that wouldn't even start
|
||||
(ImportError: cannot import name 'HiggsAudioV2TokenizerModel').
|
||||
|
||||
nemo-parakeet has no isolated-venv option yet (unlike dots-tts /
|
||||
moss-tts-v15 / confucius4-tts, which do), so the hint must not imply a safe
|
||||
one-line fix — it must say plainly that installing into the shared venv
|
||||
will break the backend.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from services.asr_backend import _INSTALL_HINTS, list_backends
|
||||
|
||||
|
||||
def test_nemo_parakeet_hint_does_not_recommend_shared_venv_install():
|
||||
hint = _INSTALL_HINTS["nemo-parakeet"]
|
||||
# The literal old, destructive recommendation must never reappear.
|
||||
assert "pip install nemo_toolkit[asr]" not in hint, (
|
||||
f"nemo-parakeet install_hint regressed to the shared-venv-breaking "
|
||||
f"bare pip install: {hint!r}"
|
||||
)
|
||||
|
||||
|
||||
def test_nemo_parakeet_hint_warns_about_transformers_conflict():
|
||||
hint = _INSTALL_HINTS["nemo-parakeet"]
|
||||
assert "transformers" in hint
|
||||
lowered = hint.lower()
|
||||
assert "conflict" in lowered or "break" in lowered
|
||||
|
||||
|
||||
def test_nemo_parakeet_hint_does_not_imply_isolated_venv_exists():
|
||||
"""Unlike dots-tts/moss-tts-v15/confucius4-tts, nemo-parakeet has no
|
||||
isolated-venv env var yet — the hint must not invent one."""
|
||||
hint = _INSTALL_HINTS["nemo-parakeet"]
|
||||
assert "OMNIVOICE_NEMO_PARAKEET_DIR" not in hint
|
||||
|
||||
|
||||
def test_nemo_parakeet_hint_surfaced_in_list_backends():
|
||||
rows = list_backends()
|
||||
row = next(r for r in rows if r["id"] == "nemo-parakeet")
|
||||
assert row["install_hint"] == _INSTALL_HINTS["nemo-parakeet"]
|
||||
@@ -81,7 +81,11 @@ def test_design_save_creates_row_when_model_unavailable(iso, monkeypatch):
|
||||
assert row["kind"] == "design"
|
||||
# Sample is pending — no rendered identity wav was forced at save time.
|
||||
assert not row["ref_audio_path"]
|
||||
assert json.loads(row["vd_states"]) == _VD
|
||||
# #983: vd_states is completed to all 6 known categories before persisting
|
||||
# (missing ones default to 'Auto') — _VD only sets 3, so the stored value
|
||||
# is a superset of it, not an exact match.
|
||||
stored = json.loads(row["vd_states"])
|
||||
assert stored == {**_VD, "Style": "Auto", "EnglishAccent": "Auto", "ChineseDialect": "Auto"}
|
||||
|
||||
|
||||
def test_all_auto_design_is_saveable(iso, monkeypatch):
|
||||
|
||||
@@ -170,12 +170,41 @@ def test_design_create_renders_sample_and_stores_params(app_client, fake_render)
|
||||
assert body["kind"] == "design"
|
||||
profile = client.get(f"/profiles/{body['id']}").json()
|
||||
assert profile["kind"] == "design"
|
||||
assert json.loads(profile["vd_states"]) == _VD
|
||||
# #983: the server now completes vd_states to all 6 known categories
|
||||
# (missing ones default to 'Auto') before persisting — _VD only sets 3,
|
||||
# so the stored value is a superset of it, not an exact match.
|
||||
stored = json.loads(profile["vd_states"])
|
||||
assert stored == {**_VD, "Style": "Auto", "EnglishAccent": "Auto", "ChineseDialect": "Auto"}
|
||||
assert profile["seed"] == 42 # deterministic identity sample
|
||||
wav = os.path.join(cfg.VOICES_DIR, profile["ref_audio_path"])
|
||||
assert os.path.exists(wav) and os.path.getsize(wav) > 0
|
||||
|
||||
|
||||
def test_design_normalizes_partial_vd_states_to_all_categories(app_client, fake_render):
|
||||
"""#983: a design profile must never persist with a partial vd_states shape.
|
||||
|
||||
A client (older frontend build, hand-edited payload, third-party API
|
||||
caller) that only sends a subset of the 6 known category keys used to be
|
||||
saved as-is — selecting that profile later handed the frontend an
|
||||
incomplete vdStates object, crashing DesignMethodPanel's render
|
||||
("Cannot read properties of undefined (reading 'replace')"). The server
|
||||
now fills every missing category with 'Auto' before persisting, so the
|
||||
stored vd_states is always complete regardless of which client wrote it.
|
||||
"""
|
||||
client, _ = app_client
|
||||
r = client.post(
|
||||
"/profiles",
|
||||
data={"name": "Partial", "kind": "design", "vd_states": json.dumps({"Gender": "male"})},
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
profile = client.get(f"/profiles/{r.json()['id']}").json()
|
||||
stored = json.loads(profile["vd_states"])
|
||||
assert set(stored) == {"Gender", "Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect"}
|
||||
assert stored["Gender"] == "male"
|
||||
for cat in ("Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect"):
|
||||
assert stored[cat] == "Auto"
|
||||
|
||||
|
||||
# ── Migration 0005 ───────────────────────────────────────────────────────────
|
||||
|
||||
def _run_alembic(direction: str, db_path: str, target: str = "head"):
|
||||
|
||||
@@ -154,6 +154,24 @@ class _FakeModel:
|
||||
return [torch.full((1, n), val)]
|
||||
|
||||
|
||||
class _FakeBackend:
|
||||
"""Adapts the list-returning _FakeModel above to the TTSBackend.generate()
|
||||
contract (a single tensor, not a list) that resolve_generation_backend()
|
||||
now hands dub_generate.py (issue #312 class)."""
|
||||
|
||||
applies_own_mastering = False
|
||||
|
||||
def __init__(self, model):
|
||||
self._model = model
|
||||
|
||||
@property
|
||||
def sample_rate(self):
|
||||
return self._model.sampling_rate
|
||||
|
||||
def generate(self, *a, **kw):
|
||||
return self._model.generate(*a, **kw)[0]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def patched_generate(monkeypatch, tmp_path):
|
||||
"""Patch api.routers.dub_generate so `_stream` runs hermetically:
|
||||
@@ -162,8 +180,8 @@ def patched_generate(monkeypatch, tmp_path):
|
||||
|
||||
model = _FakeModel()
|
||||
|
||||
async def _fake_get_model():
|
||||
return model
|
||||
async def _fake_resolve_generation_backend(**kwargs):
|
||||
return _FakeBackend(model)
|
||||
|
||||
job = {
|
||||
"duration": 2.0,
|
||||
@@ -173,7 +191,7 @@ def patched_generate(monkeypatch, tmp_path):
|
||||
job_dir = tmp_path / "jobX"
|
||||
job_dir.mkdir()
|
||||
|
||||
monkeypatch.setattr(dg, "get_model", _fake_get_model)
|
||||
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
|
||||
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
|
||||
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
|
||||
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
|
||||
|
||||
@@ -269,6 +269,63 @@ def test_preflight_network_handles_offline():
|
||||
assert _probe_network(host="10.255.255.1", timeout=0.3) is False
|
||||
|
||||
|
||||
def test_preflight_network_unreachable_is_warn_not_blocker():
|
||||
"""A dead network must NOT hard-block the wizard (restricted-network
|
||||
first-run, e.g. China where huggingface.co is blocked): the check is a
|
||||
warning and the aggregate `ok` is unaffected by it."""
|
||||
from api.routers.setup import wizard as setup_mod
|
||||
|
||||
with patch.object(setup_mod, "_probe_network", return_value=False):
|
||||
body = client_factory().get("/setup/preflight").json()
|
||||
|
||||
net = next(c for c in body["checks"] if c["id"] == "network")
|
||||
assert net["status"] == "warn", net
|
||||
assert "continue" in (net["fix"] or "").lower()
|
||||
# ok must still equal "no fail among checks" — network can't be the fail.
|
||||
any_fail = any(c["status"] == "fail" for c in body["checks"])
|
||||
assert body["ok"] is (not any_fail)
|
||||
|
||||
|
||||
def test_preflight_network_probes_configured_mirror():
|
||||
"""With HF_ENDPOINT set, the probe targets the mirror host — not the
|
||||
hardcoded official host that may be blocked on the user's network."""
|
||||
import os
|
||||
from api.routers.setup import wizard as setup_mod
|
||||
|
||||
seen_hosts: list[str] = []
|
||||
|
||||
def fake_probe(host="huggingface.co", port=443, timeout=2.0):
|
||||
seen_hosts.append(host)
|
||||
return True
|
||||
|
||||
with patch.dict(os.environ, {"HF_ENDPOINT": "https://mirror.example.test"}), \
|
||||
patch.object(setup_mod, "_probe_network", side_effect=fake_probe):
|
||||
body = client_factory().get("/setup/preflight").json()
|
||||
|
||||
net = next(c for c in body["checks"] if c["id"] == "network")
|
||||
assert "mirror.example.test" in net["label"]
|
||||
assert net["status"] == "pass"
|
||||
assert "mirror.example.test" in seen_hosts
|
||||
|
||||
|
||||
def test_preflight_network_suggests_reachable_mirror():
|
||||
"""Official endpoint blocked but hf-mirror.com reachable → the fix names
|
||||
the mirror and the check carries mirror_reachable=True for the wizard's
|
||||
quick-pick affordance."""
|
||||
from api.routers.setup import wizard as setup_mod
|
||||
|
||||
def fake_probe(host="huggingface.co", port=443, timeout=2.0):
|
||||
return host == "hf-mirror.com"
|
||||
|
||||
with patch.object(setup_mod, "_probe_network", side_effect=fake_probe):
|
||||
body = client_factory().get("/setup/preflight").json()
|
||||
|
||||
net = next(c for c in body["checks"] if c["id"] == "network")
|
||||
assert net["status"] == "warn"
|
||||
assert net.get("mirror_reachable") is True
|
||||
assert "hf-mirror.com" in (net["fix"] or "")
|
||||
|
||||
|
||||
# ── RAM thresholds ───────────────────────────────────────────────────────
|
||||
|
||||
def test_preflight_ram_fail_threshold():
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
"""A quit mid-preload must not report a clean shutdown while a GPU-pool
|
||||
thread is still running (#1000 class).
|
||||
|
||||
Field report: a backend log showed three rapid restart cycles, each ending
|
||||
with "Shutdown: done." immediately followed by a "Model loading failed:
|
||||
Could not import module 'AutoFeatureExtractor'" error — transformers' own
|
||||
generic lazy-import wrapper, not a real dependency problem. The real cause:
|
||||
`preload_task` (and the optional `capture_preload_task`) were created at
|
||||
startup but never referenced in the shutdown block, so `idle_task`/
|
||||
`worker_task` got cancelled-and-awaited while the preload task was simply
|
||||
abandoned — the process declared "done" while a background GPU-pool thread
|
||||
was still mid-`import`, and got torn down by interpreter finalization under
|
||||
it.
|
||||
|
||||
`_cancel_and_await_tasks` is the extracted, directly-testable shutdown
|
||||
helper — the full `lifespan()` context manager touches too much startup
|
||||
machinery (DB init, gallery init, MCP session manager) to drive directly in
|
||||
a unit test (this suite's own test_mcp_mount.py notes exactly this: running
|
||||
the full lifespan contaminates other tests' event loops).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
|
||||
|
||||
from main import _cancel_and_await_tasks # noqa: E402
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def test_a_task_that_finished_before_cancel_keeps_its_result():
|
||||
"""An early-stage task (mirrors preload still importing, not yet deep in
|
||||
blocking weight-load work) that completes on its own before the shutdown
|
||||
helper even reaches it must not be treated as an error — `.cancel()` on
|
||||
an already-done task is a no-op, and its real result survives. This is
|
||||
the fix: previously preload_task was never referenced in shutdown at
|
||||
all, so this case (the common one — most quits don't land mid-import)
|
||||
was never even checked."""
|
||||
finished = []
|
||||
|
||||
async def _quick():
|
||||
await asyncio.sleep(0.01)
|
||||
finished.append("done")
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_quick())
|
||||
await asyncio.sleep(0.05) # long enough for _quick() to fully finish
|
||||
assert t.done()
|
||||
await _cancel_and_await_tasks(t, timeout=1.0) # must not raise on a done task
|
||||
|
||||
_run(_scenario())
|
||||
assert finished == ["done"]
|
||||
|
||||
|
||||
def test_none_entries_are_skipped_without_error():
|
||||
"""capture_preload_task is None when OMNIVOICE_PRELOAD_CAPTURE_ASR=0 —
|
||||
the helper must not crash on a mix of real tasks and None."""
|
||||
async def _noop():
|
||||
return None
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_noop())
|
||||
await _cancel_and_await_tasks(t, None, timeout=1.0)
|
||||
|
||||
_run(_scenario()) # must not raise
|
||||
|
||||
|
||||
def test_a_task_stuck_past_the_bound_times_out_without_hanging():
|
||||
"""A task that never yields back (mirroring a GPU-pool thread stuck in a
|
||||
blocking native call) must not hang shutdown forever — the bound is the
|
||||
backstop, same as the pre-existing idle_task/worker_task pattern."""
|
||||
async def _wedged():
|
||||
await asyncio.sleep(10.0)
|
||||
|
||||
async def _scenario():
|
||||
t = asyncio.create_task(_wedged())
|
||||
await asyncio.sleep(0.01)
|
||||
await _cancel_and_await_tasks(t, timeout=0.2)
|
||||
|
||||
import time
|
||||
start = time.monotonic()
|
||||
_run(_scenario())
|
||||
elapsed = time.monotonic() - start
|
||||
assert elapsed < 2.0, f"shutdown helper did not bound its wait: took {elapsed:.2f}s"
|
||||
|
||||
|
||||
def test_multiple_tasks_are_all_cancelled_before_any_await():
|
||||
"""Cancel-then-await (not cancel-then-immediately-await-one-at-a-time) —
|
||||
every task gets its cancellation requested up front, so a slow task
|
||||
earlier in the list can't delay a later task's cancel signal."""
|
||||
cancelled_order = []
|
||||
|
||||
async def _tracked(name, delay):
|
||||
try:
|
||||
await asyncio.sleep(delay)
|
||||
except asyncio.CancelledError:
|
||||
cancelled_order.append(name)
|
||||
raise
|
||||
|
||||
async def _scenario():
|
||||
t1 = asyncio.create_task(_tracked("slow", 5.0))
|
||||
t2 = asyncio.create_task(_tracked("fast", 5.0))
|
||||
await asyncio.sleep(0.01)
|
||||
await _cancel_and_await_tasks(t1, t2, timeout=0.5)
|
||||
|
||||
_run(_scenario())
|
||||
assert set(cancelled_order) == {"slow", "fast"}
|
||||
|
||||
|
||||
def test_production_shutdown_wait_is_generous_enough_for_a_cold_import():
|
||||
"""Post-merge code-review finding (Greptile, PR #1002): the original 3s
|
||||
bound left a real residual window — cancelling the asyncio task doesn't
|
||||
stop the underlying OS thread, so a cold transformers import taking
|
||||
longer than the bound could still let shutdown report "done" while that
|
||||
thread was alive, the exact #1000 class again just with lower odds.
|
||||
Python can't forcibly kill a running thread, so no finite bound
|
||||
eliminates this outright — this pins the production call site to a
|
||||
materially more generous wait (20s, not 3s) rather than letting a future
|
||||
edit quietly shrink it back down without deliberate consideration.
|
||||
|
||||
Source-level guard, not a live-timing test: driving an actual >3s cold
|
||||
import through this suite would make it slow and environment-dependent
|
||||
for no real benefit.
|
||||
"""
|
||||
import re
|
||||
|
||||
src = open(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
|
||||
"backend", "main.py")).read()
|
||||
call = re.search(
|
||||
r"await _cancel_and_await_tasks\(\s*idle_task,\s*worker_task,\s*preload_task,"
|
||||
r"\s*capture_preload_task,\s*timeout=([\d.]+),?\s*\)",
|
||||
src,
|
||||
)
|
||||
assert call, "production shutdown call site not found in main.py"
|
||||
assert float(call.group(1)) >= 15.0, (
|
||||
f"shutdown wait bound regressed to {call.group(1)}s — see PR #1002 review history "
|
||||
"before shrinking this"
|
||||
)
|
||||
@@ -52,6 +52,24 @@ class _FakeModel:
|
||||
return [torch.full((1, n), 0.25)]
|
||||
|
||||
|
||||
class _FakeBackend:
|
||||
"""Adapts the list-returning _FakeModel above to the TTSBackend.generate()
|
||||
contract (a single tensor, not a list) that resolve_generation_backend()
|
||||
now hands dub_generate.py (issue #312 class)."""
|
||||
|
||||
applies_own_mastering = False
|
||||
|
||||
def __init__(self, model):
|
||||
self._model = model
|
||||
|
||||
@property
|
||||
def sample_rate(self):
|
||||
return self._model.sampling_rate
|
||||
|
||||
def generate(self, *a, **kw):
|
||||
return self._model.generate(*a, **kw)[0]
|
||||
|
||||
|
||||
async def _fake_stretch(wav, target_samples, sr):
|
||||
"""Stand-in for the ffmpeg atempo pipe: deterministic linear interp."""
|
||||
if target_samples <= 0 or wav.shape[-1] == target_samples:
|
||||
@@ -67,8 +85,8 @@ def patched_generate(monkeypatch, tmp_path):
|
||||
|
||||
model = _FakeModel()
|
||||
|
||||
async def _fake_get_model():
|
||||
return model
|
||||
async def _fake_resolve_generation_backend(**kwargs):
|
||||
return _FakeBackend(model)
|
||||
|
||||
job = {
|
||||
"duration": 4.0,
|
||||
@@ -78,7 +96,7 @@ def patched_generate(monkeypatch, tmp_path):
|
||||
job_dir = tmp_path / "jobX"
|
||||
job_dir.mkdir()
|
||||
|
||||
monkeypatch.setattr(dg, "get_model", _fake_get_model)
|
||||
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
|
||||
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
|
||||
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
|
||||
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
|
||||
|
||||
@@ -21,6 +21,8 @@ from services.speaker_clone import (
|
||||
MIN_SLICE_DURATION_S,
|
||||
_pick_reference_slices,
|
||||
extract_speaker_clones,
|
||||
refine_ref_text,
|
||||
refine_ref_texts,
|
||||
)
|
||||
|
||||
SR = 16000
|
||||
@@ -124,3 +126,83 @@ class TestExtractSpeakerClones:
|
||||
# or every real turn boundary would be flagged.
|
||||
from services.segmentation import SPEAKER_GAP
|
||||
assert 0 < ADJACENT_TURN_GUARD_S < SPEAKER_GAP
|
||||
|
||||
|
||||
class _FakeASR:
|
||||
"""Stands in for the active ASR backend's .transcribe() — no model, no
|
||||
network. `chunks_by_path` maps a ref_audio path to the canned chunk list
|
||||
that path's re-transcription should return."""
|
||||
def __init__(self, chunks_by_path=None, raises_for=()):
|
||||
self.chunks_by_path = chunks_by_path or {}
|
||||
self.raises_for = set(raises_for)
|
||||
self.calls = []
|
||||
|
||||
def transcribe(self, path, *, word_timestamps=True):
|
||||
self.calls.append(path)
|
||||
if path in self.raises_for:
|
||||
raise RuntimeError("simulated ASR failure")
|
||||
return {"chunks": self.chunks_by_path.get(path, []), "language": "es"}
|
||||
|
||||
|
||||
class TestRefineRefText:
|
||||
# Issue #1004: the ASR segment's `text` field and its `[start, end]`
|
||||
# timestamps routinely drift (a trailing word audible in the slice but
|
||||
# missing from the text, or vice versa) — pairing a mismatched (ref_audio,
|
||||
# ref_text) breaks zero-shot TTS prompt priming badly enough that the
|
||||
# clone can speak the reference text verbatim instead of the target text.
|
||||
# Re-transcribing the actual written clip guarantees the pair matches.
|
||||
|
||||
def test_replaces_mismatched_text_with_the_actual_clip_transcript(self):
|
||||
asr = _FakeASR(chunks_by_path={
|
||||
"/tmp/ref.wav": [{"text": "hola"}, {"text": "que tal"}],
|
||||
})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="mismatched source text")
|
||||
assert out == "hola que tal"
|
||||
assert asr.calls == ["/tmp/ref.wav"]
|
||||
|
||||
def test_falls_back_to_original_text_on_asr_failure(self):
|
||||
asr = _FakeASR(raises_for={"/tmp/ref.wav"})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
def test_falls_back_to_original_text_on_empty_transcript(self):
|
||||
# A clip ASR can't get any text out of (e.g. near-silent) shouldn't
|
||||
# wipe out a usable original — empty is worse than stale.
|
||||
asr = _FakeASR(chunks_by_path={"/tmp/ref.wav": []})
|
||||
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
def test_no_asr_backend_is_a_strict_no_op(self):
|
||||
# Preflight ASR load failure, or any other reason the caller has no
|
||||
# backend to hand in — never a crash, never blocks the original path.
|
||||
out = refine_ref_text("/tmp/ref.wav", None, fallback_text="original text")
|
||||
assert out == "original text"
|
||||
|
||||
|
||||
class TestRefineRefTexts:
|
||||
def test_refines_every_entry_in_place_and_returns_the_dict(self):
|
||||
asr = _FakeASR(chunks_by_path={
|
||||
"/tmp/spk1.wav": [{"text": "hola amigo"}],
|
||||
"/tmp/spk2.wav": [{"text": "buenos dias"}],
|
||||
})
|
||||
clones = {
|
||||
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "stale 1"},
|
||||
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
|
||||
}
|
||||
out = refine_ref_texts(clones, asr)
|
||||
assert out is clones # mutated in place, returned for call-and-reassign
|
||||
assert clones["Speaker 1"]["ref_text"] == "hola amigo"
|
||||
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
|
||||
|
||||
def test_a_failing_entry_does_not_affect_the_others(self):
|
||||
asr = _FakeASR(
|
||||
chunks_by_path={"/tmp/spk2.wav": [{"text": "buenos dias"}]},
|
||||
raises_for={"/tmp/spk1.wav"},
|
||||
)
|
||||
clones = {
|
||||
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "kept on failure"},
|
||||
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
|
||||
}
|
||||
refine_ref_texts(clones, asr)
|
||||
assert clones["Speaker 1"]["ref_text"] == "kept on failure"
|
||||
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""The conftest torch-dtype guard resets a leaked default dtype between tests.
|
||||
|
||||
The CI "flaky trio" (test_effects_chain / test_generation_audio_guard /
|
||||
test_persona_bundle) failed intermittently on CI-Linux with signatures that
|
||||
all trace to one leak: some earlier test leaves
|
||||
``torch.set_default_dtype(torch.float16)`` behind. Reproduced locally with a
|
||||
simulated polluter — ``torch.tensor([0.1, …])`` under fp16 yields exactly the
|
||||
0.0999755859375 CI observed, and Pedalboard refuses fp16 audio outright
|
||||
("only supports 32-bit and 64-bit floating point"), silently returning
|
||||
unmodified audio for every preset so their outputs compare identical.
|
||||
|
||||
These two tests are order-dependent BY DESIGN (pytest runs tests within a
|
||||
file in definition order): the first leaks, the second proves the autouse
|
||||
guard in conftest.py reset the leak before the next test began.
|
||||
"""
|
||||
import torch
|
||||
|
||||
|
||||
def test_a_deliberate_dtype_leak():
|
||||
# Simulates the CI polluter. The conftest guard must clean this up (and
|
||||
# emit a UserWarning naming this exact test as the offender).
|
||||
torch.set_default_dtype(torch.float16)
|
||||
assert torch.get_default_dtype() is torch.float16
|
||||
|
||||
|
||||
def test_b_next_test_starts_back_at_float32():
|
||||
# If the guard is ever removed/broken, this fails — and so, eventually,
|
||||
# does the flaky trio on CI, much less legibly.
|
||||
assert torch.get_default_dtype() is torch.float32
|
||||
# The exact fp16 signature the trio's CI failures showed, as documentation:
|
||||
assert torch.tensor([0.1]).item() != 0.0999755859375
|
||||
@@ -3207,7 +3207,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "omnivoice"
|
||||
version = "0.3.11"
|
||||
version = "0.3.14"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "accelerate" },
|
||||
@@ -3247,6 +3247,7 @@ dependencies = [
|
||||
{ name = "torchaudio", version = "2.8.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'linux' and sys_platform != 'win32'" },
|
||||
{ name = "torchaudio", version = "2.8.0+cu128", source = { registry = "https://download.pytorch.org/whl/cu128" }, marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
|
||||
{ name = "transformers" },
|
||||
{ name = "truststore" },
|
||||
{ name = "uvicorn" },
|
||||
{ name = "webdataset" },
|
||||
{ name = "websockets" },
|
||||
@@ -3328,6 +3329,7 @@ requires-dist = [
|
||||
{ name = "torchaudio", marker = "sys_platform != 'linux' and sys_platform != 'win32'", specifier = ">=2.4" },
|
||||
{ name = "torchaudio", marker = "sys_platform == 'linux' or sys_platform == 'win32'", specifier = ">=2.4", index = "https://download.pytorch.org/whl/cu128" },
|
||||
{ name = "transformers", specifier = ">=5.3.0" },
|
||||
{ name = "truststore", specifier = ">=0.9" },
|
||||
{ name = "unidecode", marker = "extra == 'eval'" },
|
||||
{ name = "uvicorn" },
|
||||
{ name = "webdataset" },
|
||||
@@ -6234,6 +6236,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/20/63/8cb444ad5cdb25d999b7d647abac25af0ee37d292afc009940c05b82dda0/triton-3.4.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7936b18a3499ed62059414d7df563e6c163c5e16c3773678a3ee3d417865035d", size = 155659780, upload-time = "2025-07-30T19:58:51.171Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "truststore"
|
||||
version = "0.10.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typer"
|
||||
version = "0.24.1"
|
||||
|
||||
Reference in New Issue
Block a user