Compare commits

...
38 Commits
Author SHA1 Message Date
8e6344ee4e docs(changelog): add the non-English language fixes to v0.3.7 (#533/#505/#502) (#566)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:55:21 +05:30
8ed76b40a6 fix(lang): propagate profile/request language into generation + longform (#533/#505/#502) (#565)
Non-English voices drifted to English/wrong-language because the request's or
profile's language wasn't reaching the model:

- #533: generate_speech() read instruct/ref_text/seed from a resolved profile row
  but never row['language'] (and collapsed Auto→None), so a German archetype
  previewed in German yet generated in English on the user's own call (and via
  Docker/API). Fall back to the profile's stored language when the request didn't
  pin one; an explicit non-Auto request language still wins. (Frontend already
  sets the dropdown on profile-select; this is the authoritative backend fix.)
- #505 (B2): the audiobook/longform synth hardcoded language=None, so the engine
  re-autodetected per chunk and a non-English clone flipped language mid-render.
  Add _resolve_default_language (request → profile → autodetect) and thread the
  resolved language through _build_synth/_prepare_synth/_render_longform_sse, the
  three longform request models, the preview path, and the resume manifest.
  Genuine Auto/unset behavior is unchanged.
- #502 (partial): the duration estimator weights combining marks (U+0300–036F)
  at 0.0, so NFD/decomposed text under-allocated frames → rushed audio. NFC-
  normalize text at the estimator entry — fixes the whole diacritic-script class
  (no-op for precomposed text). (The residual "distorted" core still needs the
  reporter's sample; tracked separately.)

Tests (fail-before/pass-after): profile language reaches the engine (German→de;
explicit/Auto override semantics); longform synth gets the resolved language
(→ja), not None; NFD vs NFC duration parity (Korean Hangul diverges ~3x pre-fix).
Full suite: tests/ 1740 passed, backend/tests/ 114 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:45:58 +05:30
ad78a418bc docs(changelog): v0.3.7 release notes (#563)
The stabilization release: tags the startup-crash fixes already on main + the
wave of 0.3.6-line fixes (voice design [object Object], consent_audio_path schema
self-heal, Linux WebKitGTK buttons, pip install, ASR float16 fallback, audiobook
import, Windows auto-play, download errors, relocated-venv self-heal, About
version), and folds in the MOSS-TTS/dots.tts engines (#498/#531) + the
Linux/Android audio-playback fix (#510) that landed on main.

release.yml extracts this section verbatim as the v0.3.7 GitHub release body.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:49:55 +05:30
b8cc0de44a fix: actionable download errors + self-heal a relocated venv (#554/#536/encodings) (#562)
Two robustness fixes that share the failure taxonomy:

- Video downloads (#554 douyin "Unsupported URL", #536 "Broken pipe"): yt-dlp's
  raw error surfaced with no next step. classify() now names UNSUPPORTED_VIDEO_URL
  (non-downloadable link shape — paste a direct video page or drop a file) and
  VIDEO_DOWNLOAD_NETWORK (transient CDN/network drop — just retry; the partial
  download is already cleaned up), each with an actionable hint. yt-dlp is on a
  current pin, so this is graceful classification, not a dependency bump.

- "No module named 'encodings'" (relocated/copied/restored venv whose interpreter
  can't bootstrap its stdlib — exit 1, not 106): slipped BOTH #314 self-heal
  matchers, so the user saw the error forever. Widen
  backend_exit_indicates_broken_venv to also match the full quoted phrase, routing
  it into the existing rebuild-once self-heal. Kept narrow so an app-level import
  of an 'encodings'-prefixed package can't trigger a rebuild. Plus a BROKEN_VENV
  hint for the case where the rebuild itself can't run.

Tests: classify() maps the 3 new classes with hints (a generic reason still ""),
and the 'encodings'-prefixed-package negative guard holds; the Rust matcher test
gains the encodings positive + negative cases. 5 pytest passed; the matcher is
compiled by CI's Tauri shell check.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:46:30 +05:30
a7ab148483 fix(asr): float16-unsupported GPUs fall back to int8 instead of "no segments" (#561)
#551: both CTranslate2 ASR backends request compute_type="float16" on CUDA with
NO fallback. On GPUs without efficient fp16 (older Maxwell/Pascal, GTX 16xx) or a
CTranslate2/cuDNN binary mismatch, WhisperModel/whisperx.load_model raise a
ValueError at construction — which escaped the existing OOM-only `except
RuntimeError`, so every chunk failed and the user got "Transcription produced no
segments". Add a per-device compute_type fallback chain (cuda: float16 →
int8_float16 → int8; cpu: int8 → float32) to both backends + the ASR sidecar,
alongside (not replacing) the existing OOM→CPU path, with an ASR_COMPUTE_TYPE
override for exotic hardware (documented in README).

Also in the same ASR-robustness pass:
- #549: PyTorchWhisperBackend._ensure_pipe wraps the transformers pipeline load
  and re-raises an actionable error (reinstall transformers / use faster-whisper)
  instead of a bare "Could not import module 'AutoFeatureExtractor'".
- #516: the /dub/transcribe SSE generator is wrapped so it can NEVER close
  without a terminal event — any unanticipated exception now yields a structured
  `error` (with build_failure's hint) + `done`, turning "stream dropped, likely
  ASR failed" into the real cause + Retry.
- failure.py: COMPUTE_TYPE_UNSUPPORTED + TRANSFORMERS_IMPORT classes so the
  no-segments toast is actionable.

Tests (fail-before/pass-after): float16-unsupported → int8 for both WhisperX +
FasterWhisper; a generic non-OOM RuntimeError still raises; classify() maps the
two new classes; the SSE stream always terminates with error→done. 7 + 1 passed,
17 in the failure suite (no regression).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:34:06 +05:30
51ad469ef2 fix(projects): preview finished renders in-app, not via window.open (#532) (#538)
The audiobook/story library card called window.open(audioUrl, '_blank').
Under Tauri's WebView2 on Windows that handed the file to a new webview/OS
media surface, spawning a separate black playback window with centered
controls that couldn't be closed without force-quitting the whole app.

Route the render through the shared single-playback manager (playBlobAudio)
so it previews inside the app — identical behavior on macOS/Windows/Linux,
and starting another preview stops this one. This is the only raw
window.open on a media URL in the frontend, so it fixes the whole class.

Adds a regression test: the card plays in-app and never calls window.open.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:15:44 +05:30
7678d4cca3 fix(audiobook): add AudiobookPlan.chapter_count so import doesn't 500 (#544)
POST /audiobook/import ended with `plan.chapter_count`, but AudiobookPlan
only exposed `char_count` (a property) and emitted `chapter_count` from
`to_dict()` — so the attribute access raised AttributeError, surfacing as
"500 Internal Server Error: 'AudiobookPlan' object has no attribute
'chapter_count'". The parse itself succeeded, so this hit every import
format (.txt/.md/.epub/.pdf), not just PDF.

Add a `chapter_count` property mirroring `char_count`, and have `to_dict()`
derive its key from it so the attribute and serialized key can't drift.
No API/schema/data change.

Tests: unit property test + a direct-handler /audiobook/import regression
(pdf/md/txt) that fails-before with the AttributeError.

Fixes #543

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:15:40 +05:30
6dccf0390d fix(settings): About → Version never blank (web/Pinokio build) (#560)
In the web/Pinokio (non-Tauri) build with the backend idle, Settings → About
rendered an EMPTY Version cell. Both sources were unavailable: appVersion is set
only inside the isTauri()-gated effect, and info.app_version comes from
/system/info which never resolves while the backend is IDLE (retry: Infinity).
So `appVersion || info?.app_version || '—'` produced nothing.

Add resolveAboutVersion() in a small util that falls back to the build-time
__APP_VERSION__ (Vite-injected from package.json — always present regardless of
Tauri/backend), and use it at BOTH the About → Version row AND the diagnostics-
copy block (which had no fallback at all — the whole-class fix). Tauri/live-
backend sources still take precedence, so the packaged build is unchanged.

Test: resolveAboutVersion prefers Tauri → backend → build constant, and is never
blank/dash. vitest 2 passed; typecheck clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:15:36 +05:30
3f23a77eab fix(engines): pin uv pip install to the running interpreter (#529/#527) (#559)
The engine Install chip (deep_translator / openai / argostranslate) 500'd with
"No virtual environment found". run_pip shells out to bare `uv pip install`,
which discovers its target venv from VIRTUAL_ENV / a .venv in CWD — NOT from the
running interpreter. The desktop spawns `<venv>/bin/python -m uvicorn` without
exporting VIRTUAL_ENV and CWDs outside the venv, so uv finds nothing. The
existing `--system` fallback never fires because it's gated on _in_virtualenv()
being False, but the running interpreter genuinely IS in a venv (it just can't be
auto-discovered) — the heuristic answers the wrong question.

Pass `--python sys.executable` for uv install/uninstall: targets the same
interpreter _probe()/is_installed() import from, fixing the whole class
(spawned-venv, system, conda). It takes precedence when both flags are present,
so the Docker `--system` path is untouched. (#527's openai is already bundled by
#484; this hardens the chip for the remaining runtime-installed engines.)

Test: run_pip's spawned argv contains `--python <sys.executable>` after
install/uninstall. 2 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:11:20 +05:30
cb8ff16d86 fix(ui): fill the shell on WebKitGTK so Generate/Settings buttons aren't clipped (#558)
#523 "voice synthesis not pressable" / #524 "Settings button not visible" — and
the Discord "where is the Generate button / the clone button disappeared" reports
on 0.3.6/0.3.7 with the backend UP. The #504 zoom fix only half-worked: it sizes
.app-container as calc(100vw/scale) magnified back by `zoom`, which round-trips to
the viewport on Chromium — but older WebKitGTK (Linux AppImage/deb) treats `zoom`
as a LAYOUT NO-OP, so the box stays shrunk to 77vw, leaving a ~23% black band and
pushing the bottom-pinned Synthesize/Generate CTA + the NavRail Settings footer
off-screen. The old comment's "FILLS the window (no black bands)" claim was false.

No single static rule satisfies both engines (they disagree on whether `zoom`
lays out), so detect it at runtime:
- App.jsx: a one-shot probe measures a real `zoom:2` element; if its rect isn't
  magnified, the engine treats zoom as a no-op → set html[data-zoom-layout=off].
  Robust where @supports(zoom)/UA-sniffing aren't (both lie on WebKitGTK), and
  future-proof (flips back to the zoom path on engines that start honoring it).
- index.css: html[data-zoom-layout=off] .app-container renders at 1.0 filling
  100vw/100vh — no band, no clipped CTAs. Chromium keeps the scaled zoom path.
- appShellScale.test.js: guard BOTH branches (the existing calc+zoom path AND
  the 100vw/100vh fallback) so a future "simplification" can't re-break one engine.

vitest 4 passed; typecheck clean. NOTE: CI e2e is chromium-only, so the WebKitGTK
fallback path must be eyeballed on a real Linux build before the v0.3.7 tag.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:05:26 +05:30
915256cb8c fix(db): self-heal additive schema columns so consent_audio_path 500s stop (#557)
A profile/persona/consent endpoint 500'd with "no such column:
consent_audio_path" (#552/#547) — and the same class for kind/vd_states/is_demo.
The 0003/0005 migrations exist and are wired, but init_db's
CREATE TABLE IF NOT EXISTS never adds columns to a pre-existing table, the legacy
_migrate only knows pre-0.3 columns, and _run_alembic_upgrade swallows every
failure. So on a DB whose alembic_version is stamped at a revision no longer in
versions/ (common after running a preview build) or where alembic isn't
importable, the alembic-era columns silently never land.

Fix the whole class: add _reconcile_additive_columns(conn), which builds the
canonical schema from _BASE_SCHEMA in-memory and ALTER TABLE ADD COLUMN any
column an existing table is missing (additive only — never drops/retypes;
names/types from _BASE_SCHEMA so injection-safe). Call it in init_db (so the
schema converges regardless of alembic) and again in the alembic-failure branch.
Also correct the false "_BASE_SCHEMA guarantees the schema regardless" comment.

Test (fail-before/pass-after): init_db on a legacy voice_profiles whose
alembic_version is a removed revision now lands consent_audio_path/kind/etc.
without raising; reconcile converges to the canonical column set; idempotent +
additive-only. 21 passed (incl. existing 0003/0005 migration tests).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:00:47 +05:30
1ee4b3c2a0 fix(voice): stop poisoning design profiles with "[object Object]" instruct (#556)
Voice Studio "Save design as profile" passed buildDesignInstruct()'s RETURN
OBJECT ({instruct, unsupported, duplicates}) straight to FormData.append, which
string-coerced it to the literal "[object Object]". That got persisted into
voice_profiles.instruct and 400'd on first preview/use with "Unsupported instruct
items found in [object Object]" (#550 #545 #542 #537 #530 #525).

Fix the whole class + heal already-poisoned data (backward-compatible-data rule):
- CloneDesignTab.jsx:609 — pass `.instruct` (the string), not the builder object.
- useProfiles.js — append via new instructToFormValue() helper, which extracts
  `.instruct` if an object ever slips through again (defense-in-depth).
- omnivoice/_resolve_instruct — drop the "[object Object]" sentinel instead of
  raising, so any value that slips through (e.g. generation_history) degrades to
  neutral conditioning rather than a hard 400. A genuine unsupported token still
  raises (keeps the #114/#115 user feedback).
- migration 0006 — UPDATE voice_profiles SET instruct='' WHERE it's the sentinel,
  idempotent + table-guarded, to heal profiles saved on the buggy build.

Tests (fail-before/pass-after): frontend voiceInstruct (instructToFormValue never
yields "[object Object]"); backend _resolve_instruct tolerates the sentinel but
still rejects a real bad token; alembic 0006 heals a poisoned row, leaves a
healthy one untouched. Frontend 9 passed + typecheck clean; backend 5 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 08:50:42 +05:30
Palash Debnath 9a1d549686 Merge pull request #531 from debpalash/debpalash/feature-adding-moss-tts-v1.5-8b-model-as-an-alte
feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498)
2026-06-20 08:11:34 +05:30
mergetestandClaude Opus 4.8 3777d3a62c feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498)
Adds two zero-shot voice-cloning TTS engines requested in #498, both
opt-in and subprocess-isolated with their own dedicated venv — the same
pattern as IndexTTS-2. The dedicated venv is forced, not just chosen:
each upstream pins a transformers version that conflicts with the
parent's >=5.3 (MOSS-TTS-v1.5 ==5.0.0, dots.tts ==4.57.0), so they cannot
share the parent interpreter.

Because they use the clone+venv bootstrap (env var -> clone -> uv venv),
this touches no pyproject.toml / uv.lock / bun.lock — `uv sync
--all-extras` and Docker's `bun install --frozen-lockfile` are unchanged,
so main's CI/Docker matrix stays green.

Engines:
- moss-tts-v15: 8B, 31 langs, ~16 GB weights, 24 kHz. AutoModel/
  AutoProcessor via trust_remote_code. gpu_compat=(cuda,cpu) — MPS is
  undocumented/untested upstream so it is never claimed; on a Mac it runs
  on CPU. Apache-2.0, no license gate.
- dots-tts: 2B, 24 langs, ~9 GB weights, 48 kHz. DotsTtsRuntime;
  continuation cloning (prompt_audio_path+prompt_text). Upstream is
  Linux/macOS-only, so is_available() gates it off cleanly on Windows
  (cross-platform parity rule — it is opt-in, never a broken default).

Wiring: registered in _LAZY_REGISTRY + _INSTALL_HINTS. list_backends()
surfaces both as subprocess/[cuda,cpu]/available-until-installed; the
data-driven Settings engine picker needs no frontend change.

Tests (19, fail-before/pass-after): registry resolution, subprocess
marker, no-MPS gpu_compat, the Windows gate, not-installed honesty, and
the parent-side generate() kwarg arbitration. Existing engine suite still
55 passed / 5 skipped. Sidecar inference follows the upstream-documented
APIs but, like IndexTTS/Supertonic, can't be executed in CI without the
multi-GB model clones.

Docs (same-PR per docs-sync rule): README + README_CN engine tables, new
docs/engines/moss-tts-v15.md + dots-tts.md, disk-usage.md (torch-dedup
note), CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 00:02:27 +05:30
Palash Debnath 80891d3239 Merge pull request #519 from debpalash/fix/ui-scale-clipping-504
fix(ui): shrink layout box by uiScale so zoomed shell fits viewport (#504)
2026-06-17 17:27:48 +05:30
mergetest c0727c261d fix(ui): shrink layout box by uiScale so zoomed shell fits viewport (#504)
CSS `zoom` magnifies visually but does NOT enlarge the layout box. At the
default `uiScale=1.3` the shell was drawn at 130vw x 130vh while the window
stayed 100vw x 100vh, so the bottom/right edges were clipped by `overflow:
hidden` on `#root` and `.app-container`. This hid Settings save, Dub transcribe,
and Clone buttons (only reachable via Tab).

Fix: shrink the layout box with `calc(100vw / var(--ui-scale))` /
`calc(100vh / var(--ui-scale))`, then let `zoom` magnify it back to exactly the
viewport on Chromium. On WebKitGTK `zoom` is a no-op, so the UI renders smaller
but still fills the window - no black-band regression.

Updates the appShellScale regression test to match the new intentional
`calc(... / --ui-scale)` rule and keeps the `transform: scale()` blacklist.
2026-06-17 17:27:21 +05:30
Palash Debnath 2eac93c645 Merge pull request #518 from debpalash/docs/readme-reorder-header
docs(readme): nav links above badges, download badges to Quickstart
2026-06-17 16:55:12 +05:30
mergetest 4ec0ece457 docs(readme): move nav links above badges, download badges to Quickstart
- Nav links (Quickstart, Features, Why OVS, etc.) now sit above the
  badge row in the header for better scannability
- Download badges (macOS/Windows/Linux/Debian) moved from the header
  to the Quickstart section where the user actually installs
2026-06-17 16:54:47 +05:30
Palash Debnath 50fa5c206f Merge pull request #509 from debpalash/discord/fix-508
fix(stories): apply global reading speed to preview + stem export
2026-06-17 16:54:01 +05:30
Palash Debnath ef93835e22 Merge pull request #517 from debpalash/docs/readme-shorten-why-heading
docs(readme): shorten 'Why OmniVoice Studio?' to 'Why OVS?'
2026-06-17 16:04:31 +05:30
mergetest 25a471fe59 docs(readme): shorten 'Why OmniVoice Studio?' to 'Why OVS?' 2026-06-17 16:03:10 +05:30
Palash Debnath e02b7941a8 Merge pull request #513 from debpalash/fix/donation-progress-10-of-200
fix(donate): correct progress from $137.50 to actual $10/$200 raised
2026-06-17 12:23:23 +05:30
mergetest ad443e853a fix(donate): correct progress from $137.50 to actual $10/$200 raised
The in-app goal bar and its bundled snapshot showed $137.50 / $200 (23
sponsors) — fabricated numbers. Updated both the runtime JSON and the
TypeScript fallback to reflect the real amount: $10 / $200, 1 sponsor.

Also updates the README badge color from red to yellow (in-progress).
2026-06-17 12:23:03 +05:30
Palash Debnath 1fa2dda7c8 Merge pull request #511 from paoloantinori/fix/audio-mime-and-audiocontext-unlock
fix(web): audio playback on Linux/Android (MIME + AudioContext unlock)
2026-06-17 12:17:31 +05:30
Palash Debnath abb5f00d81 Merge pull request #512 from debpalash/docs/readme-update-donate-cta-and-features
docs(readme): add donate CTA, update features/engines/roadmap for v0.3.6
2026-06-17 12:17:29 +05:30
mergetest 69b8c7c0bc docs(readme): add donate CTA, update features/engines/roadmap for v0.3.6
- Add Sponsor/Donate section with Ko-fi, PayPal, GitHub Sponsors
  links and progress bar ($10/$200 agent bill fund)
- Add Ko-fi + GitHub Sponsors badges to header
- Expand features table: Audiobook, Stories, Diagnostics,
  Engine Routing, Portable Personas, Unlimited TTS,
  Remote Backend, Dictation+LLM (3x4 → 5x4)
- Update TTS engines: 6 → 11 (add GPT-SoVITS, Sherpa-ONNX,
  IndexTTS 2, OmniVoice GGUF, Supertonic 3)
- Update ASR engines: 7 → 8 (add isolated Faster-Whisper)
- Expand roadmap Shipped with v0.3.6 additions (Longform,
  engine routing, diagnostics, MCP server, remote backend,
  reliability, etc.)
- Update architecture diagram (100+ endpoints, engine routing)
- Add comparison table rows: Audiobook/Stories, TTS count,
  ASR count, MCP Server, Self-check
- Update FAQ engine count to 11
- Fix CTranslate2 link typo
- Remove already-shipped Audiobook Editor from Up Next
2026-06-17 12:16:27 +05:30
Paolo Antinori f4e7a76886 docs(changelog): note MIME + AudioContext fixes for Linux/Android playback 2026-06-17 07:25:37 +02:00
Paolo Antinori 74d63f099c test(audioUnlock): unit-test the gesture-driven AudioContext resume
Locks down the resume path from the parent commit:

- Every AudioContext is tracked at construction (the wrap is active)
- unlockAudio() resumes all suspended tracked contexts in parallel
- Idempotent: repeated calls do not re-resume
- Contexts created after unlock are not re-resumed by a second call
- resume() rejections are swallowed — one bad context doesn't block others
- installAudioUnlock() is idempotent (the _installed gate works)

The unlock path was the fix for the "click does nothing" bug on Linux
Firefox/Chrome and Android Chrome, where AudioContexts created before a
user gesture stay suspended — decodeAudioData hangs → WaveSurfer's ready
never fires → the play button never enables. Without this test, breaking
the gesture wiring silently regresses every non-macOS browser.

Adds the __resetForTesting() export so the unlock can be exercised
repeatedly against the same module instance (the unlock is meant to be
a one-shot per page load).
2026-06-17 06:48:43 +02:00
Paolo AntinoriandClaude Opus 4.7 f448c1c73c fix(web): resume AudioContext on first gesture so play button works on Linux/Android (#fix)
Browser autoplay policy (Linux Firefox/Chrome, Android Chrome, mobile
Safari): AudioContexts created before a user gesture start in "suspended"
state — decodeAudioData hangs and WaveSurfer's `ready` event never fires.
The play button gated on `ready` stays disabled forever, so the click
silently does nothing and no /audio/ request ever fires.

macOS Safari/Chrome are more lenient (typically auto-resume on first
interaction) which masked the bug cross-platform.

Fix has three parts:

1. `frontend/src/utils/audioUnlock.js` (new) — monkey-patches
   `window.AudioContext` (and `webkitAudioContext`) to track every
   instance ever created. Exports `installAudioUnlock()` which wires a
   one-time pointerdown/keydown/touchstart listener that resumes all
   suspended contexts on the first user gesture. The patch MUST install
   before any module constructs an AudioContext, so this file is imported
   first in main.jsx before the dynamic import of main-app.jsx.

2. `frontend/src/main.jsx` — imports and installs the unlock before any
   other module loads.

3. `frontend/src/components/WaveformPlayer.jsx` — three changes:
   - Remove the `Loader` spinner that gated on `ready`. The spinner
     itself was a visual signal that the user was waiting on a state
     the browser refuses to produce without user interaction.
   - Button is now `disabled={!resolvedUrl}` — clickable as soon as the
     audio URL exists, so the user's click IS the gesture that unlocks
     the AudioContext.
   - `togglePlay()` explicitly awaits `unlockAudio()` before calling
     `playPause()` to close any race with the global gesture listener.

The console warning "An AudioContext was prevented from starting
automatically" may still appear once on page load — that's the
informational signal that the pre-gesture context was created suspended;
it's harmless because we explicitly resume on first interaction.

Tested: Linux Firefox 151.0.3 — before fix, clicking play did nothing
(no /audio/ request fired, button never enabled). After fix, single
click on play resumes AudioContext + starts playback, waveform animates.

Pairs with the audio/wav MIME fix in the same PR — both bugs had the
same user-visible symptom (silent play button on Linux) but different
root causes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-17 06:33:21 +02:00
mergetestandClaude Opus 4.8 35e7dbc11c fix(stories): apply global reading speed to preview + stem export (#508)
The #415 global speed only flowed through the full longform export
(storyToSpans). Per-segment preview and stem export resolved speed with a
hardcoded `track.speed || 1.0`, silently dropping the global → generated
audio played at 1.0x even with the global set to e.g. 0.70x.

Add a pure `effectiveSpeed(track, globalSpeed)` helper (mirrors
effectiveProfile / storyToSpans precedence: per-line override → global →
engine default) and use it at both call sites so all three generation
paths agree. Regression-tested in storyCast.test.js.

Fixes #508

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 00:13:57 +05:30
05369e748a fix(settings): fill sparse tabs + route Appearance strings through i18n (#507)
* fix(settings): fill the panel on short tabs instead of a stunted box in a void

Settings tabs with little content (Appearance — UI scale/theme/font; About;
Privacy) rendered the accent-bordered .settings-content as a short box with the
rest of the page as empty black void below it (reported on Appearance).

Make .settings-page a flex column with min-height:100% (a FLOOR — tall tabs grow
past it and scroll exactly as before) and let .settings-content flex:1 1 auto
grow to fill. Safe by design: it only adds space when there's slack, and if the
parent height is ever indeterminate the rule no-ops rather than constraining
content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(settings): route AppearancePanel strings through i18n

The Appearance panel hardcoded English ("UI scale", "Color theme", "Font", and
the help paragraph) — against the localization hard rule. Wire them through
t('settings.*', { defaultValue }) matching the ApiKeysPanel pattern; reuse the
existing settings.appearance/ui_scale/theme keys and add color_theme/font/
appearance_help to en.json (the reference superset — other locales fall back to
English and backfill via the translation pipeline; the i18n parity gate only
requires valid JSON). Also renamed the THEMES.map(t =>) variable to `th` so it
no longer shadows the translation `t`.

typecheck + Appearance vitest (3) + i18n parity probe (4) all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:53:49 +05:30
ade951688e fix(test): resolve package.json version reference in the desktop probe spec (#506)
#503 made tauri.conf.json derive its version from package.json
("version": "../package.json"), but the L3 desktop probe spec
(desktop_smoke.probe.yaml) asserts config.version == pyproject_version and was
reading the literal path string "../package.json" — reddening main.

Resolve the package.json reference in load_tauri_config() the way Tauri does, so
the integrity check sees the effective bundle version. The check now validates
the *real* thing end-to-end: the resolved desktop bundle version matches the
project version (and implicitly that package.json == pyproject).

Full suite: 1691 passed, 20 skipped, 11 xfailed, 3 xpassed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:08:57 +05:30
5a28c04c44 chore(version): make frontend/package.json the single source of truth (#503)
Five hand-maintained version literals (pyproject, Cargo.toml, tauri.conf.json,
package.json, version.py) drifting is what shipped a 0.3.6 build calling itself
0.3.5 (package.json lagged; the frozen backend's literal lagged). Collapse to
one canonical source.

- frontend/package.json is canonical: vite already injects __APP_VERSION__ from
  it (first-run setup footer + bug reports).
- tauri.conf.json now reads its bundle version from it ("version":
  "../package.json", a supported Tauri v2 feature) — the MSI/dmg/updater version
  can no longer drift from the UI. Removes the most error-prone literal.
- Cargo.toml + pyproject.toml + version.py's _FALLBACK_VERSION remain as
  toolchain-required CI-guarded mirrors, bumped in lockstep from the canonical.
- release.yml: the preview-stamp and version-bump jobs now read/write
  package.json (the canonical) and no longer touch the derived tauri.conf.json.
- tests/test_app_version.py: new test_tauri_version_derives_from_package_json
  guards the path; the lockstep test now checks the mirrors against the
  canonical package.json.
- CLAUDE.md versioning rule updated to document the single-source model.

6 version tests pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:50:39 +05:30
Paolo AntinoriandClaude Opus 4.7 8db5feccc7 fix(web): serve .wav/.flac with IANA-canonical MIME so Linux/Android browsers play inline (#fix)
Python's `mimetypes.guess_type()` returns `audio/x-wav` for `.wav` and
`audio/x-flac` for `.flac` — vendor-experimental types (x- prefix) that
were never IANA-registered. macOS Chrome/Safari MIME-sniff leniently via
CoreAudio so playback works there, but Linux Chrome/Firefox (FFmpeg) and
Android Chrome (ExoPlayer) strictly honor the declared type and treat
the x- variants as download-only.

Result: the play button in the browser web app silently did nothing on
Linux/Android (download prompt instead of inline playback), while the
Tauri desktop shell worked because its WebView is lenient. The
diagnostic signal — Chromium short-circuits to download BEFORE the
<audio> element sees the response, so no MEDIA_ERR_SRC_NOT_SUPPORTED
fires; just a download prompt that's easy to miss.

Fix: register `audio/wav` and `audio/flac` (the IANA-canonical types)
via `mimetypes.add_type()` before the StaticFiles mounts in main.py.
No browser-side workaround exists (no chrome://flags, no about:config
pref) — the server is the only place this can be fixed.

Existing comment in dub_export.py:766 already acknowledges this exact
quirk for video files; this applies the same treatment to audio.

Test: regression test in test_api.py asserts `/audio/<file>.wav` returns
`Content-Type: audio/wav`. Without the fix this returns `audio/x-wav`.

Ref: https://www.iana.org/assignments/media-types/media-types.xhtml#audio

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 18:42:18 +02:00
0337e15a2e fix(version): frozen backend reports real version, not the 0.3.5 fallback (#501)
The desktop app's About panel, /health, /system/info, diagnostics, bug reports,
and exported persona/marketplace bundle metadata all read core.version.APP_VERSION.
In a synced env that resolves from package metadata (correct), so CI and the
lockstep test were green — but the PyInstaller-frozen backend has no omnivoice
.dist-info, hits PackageNotFoundError, and fell back to a hardcoded
APP_VERSION = "0.3.5". The version-bump job never touched that literal, so every
0.3.x desktop build has been reporting 0.3.5 regardless of its real version.

Fix (belt and suspenders, so it can't recur):
- backend.spec: copy_metadata('omnivoice') so importlib.metadata resolves in the
  frozen build — the primary path now works there too.
- backend/core/version.py: resolution chain is metadata → pyproject (walked up,
  correct for raw source checkouts) → a named _FALLBACK_VERSION literal as last
  resort (no longer the only fallback).
- tests/test_app_version.py: _FALLBACK_VERSION joins the lockstep (now FIVE
  sources); + a test that the fallback resolves to pyproject, + a test that
  backend.spec copies the metadata (so the frozen path can't silently regress).
- release.yml version-bump: also bumps _FALLBACK_VERSION so the lockstep guard
  never reddens main after a release.

Already-shipped binaries can't be fixed, but every build from here (tonight's
preview, the next stable) reports its real version. 5 version tests pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:05:11 +05:30
e58106d552 fix(updater): preview channel builds nightly from main + prerelease/parity guards (#500)
The Preview update channel was effectively dead: its only build trigger was a
manual workflow_dispatch, so "preview = main" was never enforced — the live
preview manifest was stuck at 0.3.5-41 (June 7) while main moved to 0.3.7.
It also shipped two latent hazards: the `preview` GitHub release had drifted to
isPrerelease=false (a non-prerelease `preview` is eligible to become GitHub's
"Latest" — the exact URL the *stable* updater reads, so it could hijack the
Stable channel), and its updater manifest dropped darwin-x86_64 (Intel-Mac
preview users silently got no updates — a cross-platform-parity breach).

Changes (release.yml):
- Add a nightly `schedule` (07:00 UTC) that rebuilds the rolling `preview`
  prerelease from main. A new `preview-gate` job no-ops the 4-platform matrix
  on nights when main didn't move, so idle days cost only a ~30s gate job.
- Centralize the preview-vs-stable decision in `preview-gate.outputs.is_preview`
  (schedule OR workflow_dispatch+publish_preview), consumed by the stamp step,
  tauri-action, and preview-notes — replacing the repeated inline conditions.
- Harden the prerelease flag: preview-notes' `gh release edit` now re-asserts
  `--prerelease` every run, and a new post-publish step fails the run if the
  preview release isn't a prerelease or its manifest is missing any platform
  stable ships (catches the Intel-Mac regression in CI).

Docs (docs-sync): update docs/update-channels.md — previews are no longer
"manual / no scheduled spend"; they build nightly from main (+ on demand).

The live `preview` release was re-flagged prerelease out-of-band to close the
hazard immediately; this makes it recurrence-proof.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:04:27 +05:30
364611f7d7 docs(changelog): backfill high-quality v0.3.6 release notes + make it a hard rule (#499)
v0.3.6 was tagged with the "Auto-generated release for v0.3.6…" fallback body
because CHANGELOG.md had no matching section (release.yml's "Extract CHANGELOG
section for tag" step found nothing). Backfill a real, user-facing
`## [0.3.6] — 2026-06-16` section (Longform suite, engine routing, dubbing +
install reliability, FSL→AGPL relicense) and fold the shipped `.ovsvoice`
persona entry out of [Unreleased]. The live GH release body was updated
in-place to match (notes + the per-platform checksum blocks preserved).

Add a "Release notes / changelog" hard rule to CLAUDE.md: every tagged release
(and preview build) gets a high-quality, house-style CHANGELOG section before
the tag — never the auto-generated fallback — since release.yml ships that
section verbatim as the release body.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:04:22 +05:30
github-actions[bot] 4471bc0770 chore(version): main -> 0.3.7 after v0.3.6 release 2026-06-16 12:30:21 +00:00
78 changed files with 4521 additions and 246 deletions
+101 -24
View File
@@ -4,12 +4,16 @@
# - push of a tag matching `v*` (e.g. `v0.2.0`) → full STABLE release,
# publishes artifacts + signed updater manifest (`latest.json`) to the
# tag's GH Release. This is the default Stable updater channel.
# - workflow_dispatch (publish_preview=true) → builds the selected branch and
# publishes a rolling `preview` PRERELEASE with its own signed
# `latest.json` at releases/download/preview/. This feeds the opt-in
# Preview updater channel (Settings → About → Update channel). The stable
# `latest` release is untouched. Run this manually whenever you want to cut
# a preview from `main`.
# - schedule (nightly, 07:00 UTC) → rolling `preview` PRERELEASE built from
# `main` with its own signed `latest.json` at releases/download/preview/.
# Feeds the opt-in Preview updater channel (Settings → About → Update
# channel). The `preview-gate` job skips the matrix on nights when `main`
# didn't move, so an idle day costs only a ~30s gate job — keeping Preview
# ≤24h behind `main` at a predictable ~1-matrix/day cost. The stable
# `latest` release is untouched.
# - workflow_dispatch (publish_preview=true) → the same preview build on
# demand from the selected branch (e.g. to preview a feature branch, or to
# refresh immediately without waiting for the nightly).
# - workflow_dispatch (publish_preview=false) → on-demand build (prior
# behavior; draft release named after the branch).
#
@@ -28,6 +32,10 @@ name: Desktop Release
on:
push:
tags: ['v*']
schedule:
# 07:00 UTC daily — rolling `preview` prerelease from `main`. The
# preview-gate job no-ops the matrix when main hasn't moved in a day.
- cron: '0 7 * * *'
workflow_dispatch:
inputs:
draft:
@@ -121,8 +129,41 @@ jobs:
working-directory: frontend
run: node --experimental-strip-types --no-warnings --test ../tests/frontend/*.test.mjs
# Decide preview-vs-stable, and for nightly runs whether `main` actually
# moved in the last day. Outputs gate the expensive matrix (`build`) and the
# `preview-notes` job, so a no-commit night costs only this ~30s job.
preview-gate:
name: Preview gate
runs-on: ubuntu-22.04
outputs:
is_preview: ${{ steps.decide.outputs.is_preview }}
proceed: ${{ steps.decide.outputs.proceed }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 50
- id: decide
shell: bash
run: |
set -euo pipefail
event="${{ github.event_name }}"
if [ "$event" = "schedule" ] || { [ "$event" = "workflow_dispatch" ] && [ "${{ inputs.publish_preview }}" = "true" ]; }; then
echo "is_preview=true" >> "$GITHUB_OUTPUT"
else
echo "is_preview=false" >> "$GITHUB_OUTPUT"
fi
# Nightly: skip the matrix when main hasn't moved in the last day.
if [ "$event" = "schedule" ] && [ -z "$(git log --since='25 hours ago' --oneline)" ]; then
echo "No new commits on main in the last day — skipping nightly preview."
echo "proceed=false" >> "$GITHUB_OUTPUT"
else
echo "proceed=true" >> "$GITHUB_OUTPUT"
fi
build:
needs: test
needs: [test, preview-gate]
# Nightly runs with no new commits on main skip the 4-platform matrix.
if: needs.preview-gate.outputs.proceed == 'true'
strategy:
fail-fast: false
matrix:
@@ -427,11 +468,14 @@ jobs:
# the Windows MSI ProductVersion (which strips the prerelease → 0.3.6)
# is also correctly above the last stable.
- name: Stamp preview version
if: github.event_name == 'workflow_dispatch' && inputs.publish_preview
if: needs.preview-gate.outputs.is_preview == 'true'
shell: bash
run: |
set -euo pipefail
CONF=frontend/src-tauri/tauri.conf.json
# package.json is the single source of truth; tauri.conf.json reads its
# version from it ("version": "../package.json"), so stamping
# package.json restamps the whole bundle.
CONF=frontend/package.json
BASE=$(jq -r .version "$CONF")
# MSI/WiX requires the semver pre-release identifier to be numeric-only
# (and <= 65535). "preview.N" hard-fails the Windows bundler, so the
@@ -471,11 +515,11 @@ jobs:
# rolling `preview` prerelease for the updater's Preview channel.
# Every other invocation — crucially the `v*` tag-push stable release
# — evaluates these expressions to exactly their prior values.
tagName: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'preview' || github.ref_name }}
releaseName: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'OmniVoice Studio (Preview)' || format('OmniVoice Studio {0}', github.ref_name) }}
tagName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
releaseName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'OmniVoice Studio (Preview)' || format('OmniVoice Studio {0}', github.ref_name) }}
releaseBody: ${{ steps.changelog.outputs.body }}
releaseDraft: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'false' || (inputs.draft || 'true') }}
prerelease: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) || false }}
releaseDraft: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'false' || (inputs.draft || 'true') }}
prerelease: ${{ needs.preview-gate.outputs.is_preview == 'true' }}
updaterJsonPreferNsis: false
includeUpdaterJson: true
@@ -661,8 +705,8 @@ jobs:
# preview-only — stable `v*` releases keep their CHANGELOG section + the
# appended checksums.
preview-notes:
needs: build
if: github.event_name == 'workflow_dispatch' && inputs.publish_preview
needs: [build, preview-gate]
if: needs.preview-gate.outputs.is_preview == 'true'
runs-on: ubuntu-22.04
permissions:
contents: write
@@ -693,9 +737,39 @@ jobs:
echo ""
echo "$CONTRIB"
} > /tmp/preview-notes.md
gh release edit preview --repo "$REPO" --notes-file /tmp/preview-notes.md
# --prerelease re-asserts the flag every run: a non-prerelease
# `preview` release is eligible to become GitHub's "Latest", which is
# the exact URL the Stable updater channel reads — so it must never
# flip off.
gh release edit preview --repo "$REPO" --prerelease --notes-file /tmp/preview-notes.md
echo "Applied auto-generated release notes + contributors to the preview release."
- name: Verify preview updater manifest (prerelease + platform parity)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
# The preview release must stay a prerelease (or it can hijack the
# Stable channel's releases/latest endpoint), and its updater manifest
# must cover every platform stable does (else those users — e.g. Intel
# Mac — silently get no preview updates).
is_pre=$(gh release view preview --repo "$REPO" --json isPrerelease -q .isPrerelease)
test "$is_pre" = "true" || { echo "::error::preview release is not a prerelease"; exit 1; }
curl -fsSL "https://github.com/$REPO/releases/download/preview/latest.json" -o /tmp/preview-latest.json
curl -fsSL "https://github.com/$REPO/releases/latest/download/latest.json" -o /tmp/stable-latest.json
python3 - <<'PY'
import json, re
prev = json.load(open("/tmp/preview-latest.json"))
stab = json.load(open("/tmp/stable-latest.json"))
v = prev.get("version", "")
assert re.fullmatch(r"\d+\.\d+\.\d+-\d+", v), f"preview version not X.Y.Z-N: {v!r}"
pk, sk = set(prev.get("platforms", {})), set(stab.get("platforms", {}))
missing = sk - pk
assert not missing, f"preview manifest missing platforms vs stable: {sorted(missing)}"
print(f"preview manifest OK: {v} platforms={sorted(pk)}")
PY
# ── Post-release version bump (versioning hard rule, owner-set 2026-06-11) ──
# main is always last-release + 1 patch. The moment a stable v* tag is
# released, bump the three version sources on main to the next patch so every
@@ -718,23 +792,26 @@ jobs:
RELEASED="${GITHUB_REF_NAME#v}"
IFS=. read -r MAJ MIN PAT <<< "$RELEASED"
NEXT="$MAJ.$MIN.$((PAT + 1))"
CURRENT=$(jq -r .version frontend/src-tauri/tauri.conf.json)
# frontend/package.json is the SINGLE SOURCE OF TRUTH: vite injects
# __APP_VERSION__ from it, and tauri.conf.json reads its bundle version
# from it ("version": "../package.json"). Read CURRENT from it.
CURRENT=$(jq -r .version frontend/package.json)
if [ "$(printf '%s\n' "$NEXT" "$CURRENT" | sort -V | tail -1)" = "$CURRENT" ] && [ "$NEXT" != "$CURRENT" ]; then
echo "main is already at $CURRENT (>= $NEXT) — nothing to bump"; exit 0
fi
tmp=$(mktemp)
jq --arg v "$NEXT" '.version = $v' frontend/src-tauri/tauri.conf.json > "$tmp"
mv "$tmp" frontend/src-tauri/tauri.conf.json
# frontend/package.json drives __APP_VERSION__ (vite.config.js) — the
# first-run footer + every auto bug report. Keep it in lockstep too,
# set absolutely (jq) so any prior drift self-heals. (#248-sweep finding)
# Bump the canonical (package.json), set absolutely so any prior drift
# self-heals. tauri.conf.json needs no edit — it derives from this.
tmp=$(mktemp)
jq --arg v "$NEXT" '.version = $v' frontend/package.json > "$tmp"
mv "$tmp" frontend/package.json
# The remaining files are CI-guarded mirrors (cargo/uv require a
# literal; the version.py literal is the frozen-backend last resort) —
# bump them in lockstep with the canonical.
sed -i "0,/^version = \"$CURRENT\"/s//version = \"$NEXT\"/" frontend/src-tauri/Cargo.toml
sed -i "0,/^version = \"$CURRENT\"/s//version = \"$NEXT\"/" pyproject.toml
sed -i "0,/_FALLBACK_VERSION = \"$CURRENT\"/s//_FALLBACK_VERSION = \"$NEXT\"/" backend/core/version.py
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add frontend/src-tauri/tauri.conf.json frontend/package.json frontend/src-tauri/Cargo.toml pyproject.toml
git add frontend/package.json frontend/src-tauri/Cargo.toml pyproject.toml backend/core/version.py
git commit -m "chore(version): main -> $NEXT after $GITHUB_REF_NAME release"
git push origin main
+309
View File
@@ -8,7 +8,139 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
## [Unreleased]
_Nothing yet — `main` is at v0.3.7 + 1 patch. New work lands here._
## [0.3.7] — 2026-06-20
A stabilization release. It tags the startup-crash fixes already on `main` (so
users hitting "Can't reach the local backend" on v0.3.5/v0.3.6 only need to
update), and clears the wave of issues reported on the 0.3.6 line across voice
design, dubbing, transcription, install, and the Linux UI.
### Added
- **Two opt-in heavyweight TTS engines: MOSS-TTS-v1.5 (8B) and dots.tts (2B).**
Both are zero-shot voice-cloning engines added per [#498](https://github.com/debpalash/OmniVoice-Studio/issues/498),
running in their own isolated subprocess venv (each pins a `transformers`
version that conflicts with the parent's `>=5.3` — MOSS `==5.0`, dots.tts
`==4.57`) via the same dedicated-venv pattern as IndexTTS-2. Point
`OMNIVOICE_MOSS_TTS_V15_DIR` / `OMNIVOICE_DOTS_TTS_DIR` at a local clone to
enable. CUDA/CPU only — neither claims Apple-Silicon MPS; dots.tts upstream
is Linux/macOS only (gated off on Windows). No change to the default install
or its lockfile. See [docs/engines/moss-tts-v15.md](docs/engines/moss-tts-v15.md)
and [docs/engines/dots-tts.md](docs/engines/dots-tts.md). (#498)
### Fixed
- **Audio playback on Linux Firefox/Chrome and Android Chrome.** Two separate
root causes both masquerade as "the play button doesn't work" on non-macOS
browsers — and both are invisible when developing on macOS, which is why they
shipped. (1) The backend served `.wav` / `.flac` with Python's default
`audio/x-wav` / `audio/x-flac` (vendor-experimental, never IANA-registered);
macOS CoreAudio MIME-sniffs leniently and plays anyway, but Linux FFmpeg and
Android ExoPlayer strictly honor the declared type and prompt to download.
Fixed by registering the canonical `audio/wav` / `audio/flac` types before
any `StaticFiles` mount. (2) WaveSurfer's `AudioContext` is constructed at
component-mount time — i.e. before any user gesture — so on Linux FF/Chrome
and Android Chrome it stays `suspended`, `decodeAudioData` hangs, the
`ready` event never fires, and the play button never enables. macOS
Safari/Chrome auto-resume on first interaction. Fixed by patching
`window.AudioContext` to track every instance and resuming them on the first
`pointerdown` / `keydown` / `touchstart`, plus resuming inline on the play
click itself. The MIME fix has a backend regression test; the unlock path
has a Vitest unit test covering idempotency, post-unlock contexts, and
error isolation. (#510)
- **Voice Studio "Save design as profile" poisoned the profile with
"[object Object]" and then 400'd every generation** ("Unsupported instruct
items found in [object Object]"). The save passed the instruct *builder
object* to the form instead of its string. Fixed at the source + defended with
a coercion helper; the engine now tolerates the sentinel, and a migration
heals already-saved profiles. (#550, #545, #542, #537, #530, #525)
- **Profile / persona / consent endpoints 500'd with `no such column:
consent_audio_path`** (and the same class for `kind`/`vd_states`/…) after an
in-place upgrade. The alembic migration existed but couldn't always apply
(stamped at a removed revision, or alembic not importable) and the failure was
swallowed. The runtime schema now self-heals — it ADDs any missing additive
column from the canonical schema on startup. (#552, #547)
- **Generate / Settings / Clone buttons were missing / unpressable on Linux.**
The UI-scale fix round-trips correctly on Chromium, but older WebKitGTK treats
`zoom` as a layout no-op, leaving a ~23% black band that pushed the bottom CTAs
off-screen. The shell now probes the engine and fills the window when `zoom`
doesn't lay out. (#523, #524)
- **The engine "Install" button 500'd with "No virtual environment found."**
`uv pip install` now targets the running interpreter (`--python
sys.executable`) instead of relying on a venv it couldn't auto-discover.
(#529, #527)
- **Transcription failed with "no segments" on GPUs without efficient float16.**
Both CTranslate2 ASR backends now fall back float16 → int8 instead of crashing
at model load; a transcribe stream can no longer close without a terminal
error event; and an incomplete `transformers` install reports an actionable
message instead of "Could not import module 'AutoFeatureExtractor'".
(#551, #549, #516)
- **Audiobook import 500'd** with `'AudiobookPlan' object has no attribute
'chapter_count'` for every format (.txt/.md/.epub/.pdf). (#543)
- **Windows: generated audio auto-played in a separate, un-closeable black
window.** Renders now play in-app through the shared playback manager. (#532)
- **Cryptic video-download errors** now carry actionable hints: an unsupported
link shape ("paste a direct video page, not a share/feed link") vs a transient
network drop ("just retry — the partial download was cleaned up"). (#554, #536)
- **About → Version rendered blank in the web/Pinokio build** (no Tauri, backend
idle); it now falls back to the build-time version.
- **A relocated, copied, or restored backend venv ("No module named
'encodings'") now self-heals** (rebuilds once) instead of failing on every
launch.
- **Non-English voices drifted to English / the wrong language.** A voice
profile's stored language wasn't propagated into generation (a German
archetype previewed in German but generated in English), the audiobook/longform
synth hardcoded `language=None` (a non-English clone could flip language
mid-render), and the duration estimator under-allocated frames for decomposed
(NFD) diacritic text. The profile/request language is now threaded through both
the single-shot and longform paths, and text is NFC-normalized. (#533, #505, #502)
- The **"Can't reach the local backend" startup-crash wave** (pkg_resources
#248, `scalar_fastapi` #307, exit-106 broken venv) was fixed in v0.3.6 — this
release carries those fixes, so updating from v0.3.5/older resolves them.
## [0.3.6] — 2026-06-16
A large release (168 commits since v0.3.5). The headline is the **Longform
suite** — produce full audiobooks and multi-voice stories from text, EPUB, or
PDF — alongside a real **engine-routing** layer that tells you up front when an
engine will fall back to CPU instead of finding out mid-synth. Dubbing,
first-run, and install reliability all get a pass too.
### Added
- **Longform: Stories + Audiobook editors.** Two new tabs turn long text into
finished audio. **Audiobook** takes a script (or imports plain text / EPUB /
PDF), auto-splits it into chapters, and renders a chaptered `.m4b` with
metadata, cover art, and per-chapter preview/resume. **Stories** is a
multi-voice editor — assign a different voice per line, preview, and export
the whole thing through the same server-side renderer. Both share one render
core (loudness, metadata, cover art) and one live SSE progress stream, and
you can convert a project between Story and Audiobook in place.
(#402, #403, #404, #408, #409, #411, #412, #413, #426, #435, #436, #447)
- **Longform: PDF & EPUB ingest.** "Import" on the Audiobook tab accepts EPUB
and PDF (not just plain text) and auto-chapters the result, so an existing
ebook becomes an audiobook without manual copy-paste. (#412, #459)
- **Longform: two-pass loudnorm mastering.** Audiobook/Story exports now run a
measure-then-normalize loudnorm pass for accurate ACX/podcast loudness
targets. A slow or broken measure pass degrades gracefully to single-pass
rather than aborting the render. (#449, #455)
- **Longform: crash-resume.** An interrupted render is resumable without
re-submitting the original input — the compiled plan is persisted to the job
dir and finished chapters are reused, so a crash mid-book doesn't cost you the
whole render. (#470)
- **Longform: pronunciation control + SSML-lite prosody.** A per-render
pronunciation lexicon (word respelling) plus an in-app pronunciation editor
and markup reference, and inline prosody markers — `[slow]` / `[fast]` /
`[emphasis]` / `[spell]` — for fine-grained delivery. (#419, #421, #422)
- **Stories: global reading-speed control.** A toolbar slider (0.52.0×) sets
one speed for every line that doesn't have its own per-line override; the
per-line slider still wins. Persisted as a UI preference. (#415, #416)
- **Unified LongformProject store.** Audiobook metadata, scripts, and prefs
persist in a single project store (with a `v4→v5` migration), and finished
books/stories now show up alongside other work in **Projects**. (#417, #443,
#444)
- **Portable personas (`.ovsvoice`).** Export any voice as a self-contained,
fully-local persona bundle — identity, optional reference clip, consent
attestation, SPDX license, and a watermarked preview — and import it back into
@@ -17,6 +149,183 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
be forged by hand-editing a bundle (real recording + consent text + attestation
required). Legacy `.omnivoice` files still import. See
[docs/persona-format.md](docs/persona-format.md). (#29)
- **Engine routing — no more silent CPU fallback.** A host device probe and
routing resolver now decide where each engine actually runs, and the verdict
is surfaced before you hit Synthesize: the **Settings → Engines** picker shows
a per-engine compatibility matrix, and **preflight** / **diagnose** report the
active engine's GPU verdict (accelerated / caveat / CPU-fallback /
unavailable). At synth time every TTS entry point (`/generate`,
`/v1/audio/speech`) enforces the same routing — an engine that can't use this
host's GPU returns an explicit error or an `X-OmniVoice-Routing` header instead
of silently dropping to CPU or dying mid-synth. (#21)
- **Diagnostics suite.** New self-check tooling for when something's wrong: a
`/system/diagnose` report (and matching backend `--diagnose`), a persistent
**error journal** surfaced in Settings, and a scrubbed **diagnostic bundle**
(home dirs stripped to `~/`, no tokens/keys) you can attach to a bug report.
Paired with structured GitHub **Issue Forms** (bug / install / feature) for
cleaner reports. (#433, #456)
- **Dubbing: multi-speaker per-speaker voice assignment.** When diarization
detects multiple speakers, each segment is now bound to its speaker's cloned
voice automatically instead of landing on "Default" and needing manual fixes;
per-segment reference clips are still preferred for quality where present. Also
adds an optional speaker-count hint for diarization. (#275, #486, #490)
- **Dubbing: Smart Fit timing + second-pass QC.** A Smart Fit timing strategy
(planner, fingerprints, per-segment video retime + drift absorption + fitted
subtitles) plus a second-pass ASR QC that flags lines whose dub drifts from the
target timing — wired into the dub editor UI. Includes a timeline segment
editor (drag, snap-to-onset, keyboard a11y), speech-onset alignment, regional
dialect targeting, and per-segment clone references. (#280, #347, #350, #369,
#370, #458)
- **Dubbing: dedicated Dub home.** A projects/history landing for dubbing with
project rename. (#435)
- **Voice Console workspace.** Clone and Design are consolidated into one Voice
workspace with right-side panels, a shared waveform player, an identity recipe
line / Active-voice card, and a free-text "describe your voice" field that maps
natural language to design parameters. (#317, #374, #376, #378, #395, #396,
#397)
- **Unified first-run setup.** Nothing installs until you confirm a plan: pick an
install mode (installed / portable), a storage location, and (on restricted
networks) custom PyPI/HF/python-build-standalone mirrors — with a
minimum-free-space gate before anything downloads. Followed by a guided
studio-console wizard with platform-aware hints, resume reassurance, and
download ETAs. (#286, #295, #297, #298)
- **Dictation: local-LLM refinement.** Opt-in local-LLM cleanup of final
transcripts (collapsing Whisper hallucination loops), available on both live
dictation and the REST `/transcribe` path; plus opt-in NLMS acoustic echo
cancellation for dictating over playback. Configure a remote LLM endpoint
(Ollama / vLLM / LM Studio) in Settings. (#356, #357, #363, #399, #400, #457)
- **Unlimited-length TTS + streaming.** Sentence-boundary chunking with
crossfade removes the per-generation length cap, and a new sentence-by-sentence
`/ws/tts` streams audio as it's produced. An inline `[pause Nms]` marker
inserts measured silence in generated speech. (#276, #357, #358)
- **MCP server v1.** OmniVoice mounts an MCP server on `/mcp` (with a stdio shim
and per-agent voice binding) so it can act as a local TTS/STT provider for
agentic pipelines. (#368)
- **Remote-backend access.** Point the desktop UI at a remote backend URL with a
bearer key (Tailscale-documented), and an opt-in Hugging Face token field in
the setup flow. (#303, #364)
- **"Fund Claude Max" support experience.** The donate page gets a real goal bar
with a "Join N supporters" social-proof line and suggested amounts, plus Pip
the mascot and a non-blocking "postcard" toast that appears only *after* a
success (a finished dub, a saved clone, a longform export) — never on errors,
setup, or first run — with escalating cooldowns and a one-click "don't ask
again". (#494)
### Fixed
- **Transcription/dubbing failed when ffmpeg wasn't on `PATH`** (notably on
Windows). WhisperX now decodes audio through OmniVoice's own validated ffmpeg
binary instead of a bare `PATH` lookup, so ASR works without a system ffmpeg
install. (#479)
- **Translation defaulted the source language to English.** Dubbing/translation
now guesses the source language from the text instead of assuming `en`,
fixing wrong-direction translations. (#478)
- **Cinematic / LLM dubbing features failed out of the box** because `openai`
wasn't bundled. The client is now a runtime dependency, so those paths work on
a fresh install. (#484)
- **`pkg_resources missing` install dead-end (#248).** The auto-repair ran
`uv pip install setuptools`, which `uv` treated as a no-op when setuptools
*metadata* was present but its files had been removed (commonly by Windows
Defender quarantine or a partial extract). Both repair sites now use
`--reinstall` to force re-extraction, and the error/hint text suggests the
working command plus an antivirus-exclusion note. (#248)
- **A stuck backend trapped users on a buttonless splash (#474).** The bootstrap
splash now has a per-stage stall watchdog: if a non-terminal stage sits past
its budget (20 min for dep install, 120 s otherwise), it flips to the failed
state with actionable hints, the live log, and Retry / Clean-&-Retry — instead
of polling forever with no way out. (#474)
- **Changing the model-download location in Settings had no effect (#480).** The
desktop launcher injected a stale models dir that overrode the per-user value,
so new downloads kept going to the old folder and "Effective location" stayed
wrong. The per-user env file now wins, so the in-app Settings path is
authoritative. (#480)
- **Backend crashed on app upgrade with a stale venv (#307).** Dependencies are
now synced on upgrade, and a structurally broken venv self-heals instead of
exiting `106`. `scalar_fastapi` is now optional so its absence can't break
startup. (#307, #314)
- **`/generate` ignored the selected TTS engine (#312)** and GGUF speech-control
parameters weren't forwarded — both now honored. (#306, #312)
- **TTS generation failed on some GPUs.** `torch.compile` failures now fall back
to eager execution so generation never hard-fails on unsupported GPUs, and
cudagraph-compiled inference is pinned to one dedicated thread to avoid
crashes. (#278, #315)
- **Re-dub ignored transcript edits (#281).** Fingerprints are canonicalized, the
preview cache is busted, and the mux is atomic, so editing the transcript and
re-dubbing actually reflects your changes. Translated subtitles now burn in
correctly and subtitle save no longer throws a JSON error. (#281, #309)
- **macOS: app wouldn't open without using Terminal.** Builds are now ad-hoc
signed (with signing/notarization verification), so the app launches normally.
(#290)
- **macOS dictation auto-paste stole focus**; it now writes the clipboard
natively without grabbing focus, and microphone-permission handling adds OS
usage descriptions, a WebView grant handler, and an actionable denied-state UI.
(#287, #323)
- **Clone-reference transcription was broken** (it used a removed transformers
pipeline); it now routes through the ASR registry. A crash-isolated
faster-whisper subprocess backend keeps an ASR crash from taking down the app.
(#308, #393)
- **Realtime status probe hit a gated route.** It now probes the auth-exempt
`/health` instead of the gated `/model/status`, and the UI polls the backend
over HTTP before opening the WebSocket to avoid startup `ECONNREFUSED`. (#439,
#450)
- **Non-executable or unreachable engine binaries showed cryptic errors** — these
now produce actionable messages. (#437, #438, #454, #466)
- **Design-profile save was coupled to a TTS render (#476)**, so saving a profile
needlessly triggered synthesis; the two are now decoupled. (#476)
- **UI scale / black bands.** The app shell now scales via `transform: scale` and
always fills the viewport, fixing the WebKitGTK black-band issue on Linux and
cramped/black layouts at narrow widths — a permanent fix across platforms.
(#445, #452)
- **Clone popover/CTA clipping and a non-resizable textarea** are fixed, the
WaveformPlayer no longer pauses itself on play or ignores clicks, and several
layout/history-display issues (phantom sidebar gap, title clamping, flicker)
are cleaned up. (#379, #384, #398, #481)
- **Windows: `desktop-prod` now runs from cmd/PowerShell** via a cross-platform
launcher, `tqdm` is disabled on non-TTY to avoid an `OSError`, and ffmpeg
validation guards against `WinError 193`. (#282, #305, #377)
- **MLX import hardened** against PyInstaller dylib failures, with a proper
platform gate so it's only loaded where it works. (#390)
### Changed
- **Restricted-network support.** A Hugging Face mirror (`HF_ENDPOINT`) setting,
custom PyPI / HF / python-build-standalone mirrors in first-run setup, and
region presets help installs complete behind restrictive networks. (#286, #391)
- **Engine memory management.** Subprocess-engine sidecars now unload on demand
and idle-reap to free VRAM. (#401, #406)
- **Faster, more accurate model downloads** via a Xet fast path with accurate
progress reporting, plus a model-management cleanup pass. (#424, #428)
- **Voice profiles unified** under one model with a `kind` discriminator and
stored design params, and consent-locked profiles (`verified_own_voice` +
spoken-consent flow). (#354, #376)
- **Updater** preview channel now offers the newest build across channels, and
preview versions carry an MSI-legal numeric pre-release stamp. (#293, #326)
- **Performance.** Voice-clone prompt embeddings are cached, and dub retime
batches seek to their window instead of decoding from frame 0. (#387, #427)
### License
- **Relicensed from FSL-1.1-ALv2 to AGPL-3.0 (open-core).** The project is now
under the GNU Affero General Public License v3, with a paid commercial license
retained for proprietary/closed-source use without AGPL obligations. The
bundled `omnivoice/` TTS model package stays Apache-2.0 upstream
(AGPL-compatible). Manifests declare `AGPL-3.0-only`; the in-app Commercial
License copy and README are updated, and the old "converts to Apache 2.0 after
two years" FAQ is removed. In-app commercial-license strings are translated
across all 20 locales. (#292)
### CI
- **macOS Intel (x86_64) build target reinstated** on `macos-15-intel`, so Intel
Mac users get installers again. (#342)
- **Docker Hub publishing.** Images now also publish to Docker Hub
(`palashdeb/omnivoice-studio`), with the Docker Hub overview maintained in-repo
and auto-synced from `main` (sync is non-fatal so it can't redden a build).
(#375, #410, #414)
- **Docs-drift guard.** A daily job compares the canonical feature inventory
against README / docs / registries to catch stale docs. (#353)
- **Security scans never cancel on `main`,** so merge trains no longer leave red
✗ on intermediate commits. (#340)
## [0.3.5] — 2026-06-03
+3 -1
View File
@@ -190,7 +190,7 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
<!-- GSD:conventions-start source:CONVENTIONS.md -->
## Conventions
**Versioning (hard rule, owner-set 2026-06-11):** main is always **latest release + 1 patch**. The moment `vX.Y.Z` is released, main's version files (`frontend/src-tauri/tauri.conf.json`, `frontend/src-tauri/Cargo.toml`, `pyproject.toml`, **and `frontend/package.json`** keep all **four** in lockstep; `package.json` drives the runtime `__APP_VERSION__` via vite, shown in the first-run footer + every auto bug report, so a drift ships a build that misreports its own version — guarded by `tests/test_app_version.py::test_all_version_files_in_lockstep`) bump to `X.Y.(Z+1)`. Consequences:
**Versioning (hard rule, owner-set 2026-06-11; single-source 2026-06-16):** main is always **latest release + 1 patch**. **`frontend/package.json` is the SINGLE SOURCE OF TRUTH for the app version** — vite injects `__APP_VERSION__` from it (first-run footer + every auto bug report), and `frontend/src-tauri/tauri.conf.json` reads its bundle version from it (`"version": "../package.json"`, so the MSI/dmg/updater version can't drift from the UI). Three toolchain-required **mirrors** are kept equal to it and bumped in lockstep `frontend/src-tauri/Cargo.toml` + `pyproject.toml` (cargo/uv need a literal) and `backend/core/version.py`'s `_FALLBACK_VERSION` (the frozen-backend last resort; at runtime the backend reads its version from package metadata via `importlib.metadata`, which `backend.spec`'s `copy_metadata('omnivoice')` makes work in the frozen build too). Never hand-edit any mirror or re-hardcode a literal in `tauri.conf.json`. Guarded by `tests/test_app_version.py` (`test_all_version_files_in_lockstep` + `test_tauri_version_derives_from_package_json`). The moment `vX.Y.Z` is released, bump `package.json` (+ the mirrors) to `X.Y.(Z+1)`. Consequences:
- Every PR and preview build identifies as the **next** version. Preview builds stamp `X.Y.(Z+1)-N` (run number), which semver-sorts **above** the last stable `X.Y.Z` — the updater ordering is natural, no comparator tricks needed.
- Releasing = tag `vX.Y.(Z+1)` from main (version files already match), then immediately bump main to `X.Y.(Z+2)`. The post-release bump is automated by the `version-bump` job in release.yml; if it fails, do it manually in the same day.
- Docker: `ghcr.io/debpalash/omnivoice-studio:latest` = **main** (rolling preview); `:X.Y.Z` + `:X.Y` + `:stable` = tagged releases. `:latest` is the preview channel by design — stable users pin `:stable` or a version tag.
@@ -198,6 +198,8 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
**Docs-sync (hard rule, owner-set 2026-06-11):** any change that alters something these docs describe — README.md, CONTRIBUTING.md, SECURITY.md, SUPPORT.md, LICENSE, or `docs/**` (install flows, Docker tag semantics, platform support, versioning/release behavior, review process, supported versions) — must update those docs **in the same PR** as the change. If a doc impact is discovered after merge, the docs fix is the immediate next commit, not backlog. Stale docs are treated as bugs.
**Release notes / changelog (hard rule, owner-set 2026-06-16):** every tagged release gets a **high-quality, user-facing `## [X.Y.Z] — DATE` section in `CHANGELOG.md`** before (or in the same hour as) the tag — never the "Auto-generated release for vX.Y.Z…" fallback. `release.yml` extracts that section verbatim as the GitHub Release body (the `Extract CHANGELOG section for tag` step), so a missing/empty section ships a bare release. Quality bar = the existing house style: a one-paragraph headline, then `### Added` / `### Fixed` / `### Changed` / `### License` / `### CI` subsections; each entry is a **bold one-line lead** (what the user gets), 13 lines of plain-English why, and the `(#NNN)` issue/PR ref — grouped by theme, written for users, **not** raw commit dumps. This applies to **preview builds too**: preview release notes summarize what's new on `main` since the last stable, in the same style. Workflow: as features merge, keep `## [Unreleased]` current; at release time rename it to the version + date. If a release was already cut with the fallback body, the next action is to backfill `CHANGELOG.md` **and** `gh release edit <tag>` the live body — not backlog.
**Localization (hard rule):** No hardcoded non-English (CJK) **user-facing text** anywhere in the codebase except the translation layer (`frontend/src/i18n/`). All UI strings go through i18n (`t('...')` keys in `locales/*.json`); native language names live in `i18n/index.ts` (`LANGUAGES`). Functional CJK is allowed and tracked via the allowlist in `tests/test_no_hardcoded_cjk.py` — text-processing regexes, model/engine vocabulary & identifiers (e.g. CosyVoice speaker IDs), localized error matching, demo/eval data, and test fixtures. CI fails on any hardcoded CJK outside the allowlist; to add legitimate functional CJK, extend `_ALLOWED_FILES` there with a justification.
**Fix quality (hard rule, owner-set 2026-06-16):** Fix issues *properly* and future-maintenance-proof — don't stop at the symptom. Root-cause fully, fix the whole **class** of the bug (not just the one reported instance), add a fail-before/pass-after regression test, and harden against recurrence (e.g. if a lockfile drift only fails in Docker, also make CI catch it). Go the extra mile where it durably pays off. Be token-efficient about it — extra **effort**, not extra **verbosity**: no padding, no redundant re-checks, the smallest correct change that is also recurrence-proof. Don't be shy to spend the effort a proper fix needs; do be shy about wasting tokens.
+137 -50
View File
@@ -4,34 +4,26 @@
<h3>The open-source ElevenLabs alternative.</h3>
<p>Real-time dictation, zero-shot voice cloning, and cinematic video dubbing — all on your desktop.<br/>Open-source, no API keys, fully local. <b>646 languages.</b></p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Stars" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="Release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
</p>
<p>
<a href="#quickstart">Quickstart</a> ·
<a href="#features">Features</a> ·
<a href="#why-omnivoice-studio">Why OmniVoice Studio?</a> ·
<a href="#why-ovs">Why OVS</a> ·
<a href="#tts-engines">TTS Engines</a> ·
<a href="#asr-engines">ASR Engines</a> ·
<a href="#sponsor--donate">Donate</a> ·
<a href="#contributing">Contributing</a> ·
<a href="https://discord.gg/bzQavDfVV9">Discord</a> ·
<a href="README_CN.md"><strong>简体中文</strong></a>
</p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<!-- Pre-built macOS bundle is Apple Silicon. Intel Macs: build from source (docs/install/macos.md); a pre-built Intel target is tracked in #279. -->
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
</p>
<p>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Stars" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="Release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_Us-FF5E5B?style=flat-square&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
<a href="https://github.com/sponsors/debpalash"><img src="https://img.shields.io/badge/GitHub-Sponsor-ff69b4?style=flat-square&logo=github&logoColor=white" alt="GitHub Sponsors" /></a>
</p>
</div>
@@ -58,20 +50,28 @@
<table>
<tr>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎙️ Voice Cloning</h3>
<p>3-second clip → mirror any voice.<br/><b>646 languages</b>, zero-shot.</p>
</td>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎨 Voice Design</h3>
<p>Gender, age, accent, pitch, speed,<br/>emotion, dialect — <b>dial it in</b>.</p>
</td>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎬 Video Dubbing</h3>
<p>YouTube URL or file → transcribe →<br/>translate → re-voice → <b>MP4</b>.</p>
</td>
<td align="center" width="25%">
<h3>📖 Audiobook Editor</h3>
<p>Import text, EPUB, or PDF. Auto-chapter,<br/>loudnorm, metadata. Export <b>.m4b</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🎭 Stories</h3>
<p>Multi-voice editor. Assign voices<br/>per-line, preview, <b>export full cast</b>.</p>
</td>
<td align="center" valign="top">
<h3>⌨️ Dictation Widget</h3>
<p><code>⌘+⇧+Space</code> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
@@ -98,6 +98,10 @@
<h3>🛡️ AI Watermark</h3>
<p>AudioSeal (Meta). <b>Invisible</b>,<br/>survives compression.</p>
</td>
<td align="center" valign="top">
<h3>🔬 Diagnostics</h3>
<p>Self-check, error journal,<br/>scrubbed <b>diagnostic bundle</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
@@ -110,7 +114,29 @@
</td>
<td align="center" valign="top">
<h3>🧩 Extensible</h3>
<p>Subclass <code>TTSBackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
<p>Subclass <code>TTSbackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧭 Engine Routing</h3>
<p>Preflight GPU check per engine.<br/><b>No silent CPU fallback</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🎒 Portable Personas</h3>
<p>Export voices as <code>.ovsvoice</code><br/>bundles — identity + <b>watermark</b>.</p>
</td>
<td align="center" valign="top">
<h3>♾️ Unlimited TTS</h3>
<p>Sentence-chunked generation.<br/><b>No length cap</b>. Streaming via WS.</p>
</td>
<td align="center" valign="top">
<h3>🌐 Remote Backend</h3>
<p>Point UI at a remote server.<br/>Tailscale-friendly. <b>Bearer auth</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧠 Dictation + LLM</h3>
<p>Local LLM cleanup of transcripts.<br/>Optional echo <b>cancellation</b>.</p>
</td>
</tr>
</table>
@@ -119,6 +145,15 @@
## Quickstart
<div align="center">
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
<br/>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
</div>
Per-OS install guides — pick yours and follow it end-to-end:
- **macOS** — [docs/install/macos.md](docs/install/macos.md)
@@ -191,7 +226,7 @@ options, see [docs/downloading-models.md](docs/downloading-models.md).
---
## Why OmniVoice Studio?
## Why OVS?
ElevenLabs charges **$5$330/mo** and processes your audio on their servers. OmniVoice Studio runs **on your hardware, with no usage limits.**
@@ -200,12 +235,17 @@ ElevenLabs charges **$5$330/mo** and processes your audio on their servers. O
| **Pricing** | $5$330/mo, per-character billing | Free & open-source (AGPL-3.0) · [Commercial license](#license) for proprietary use |
| **Voice Cloning** | ✅ 3s clip | ✅ 3s clip, zero-shot |
| **Voice Design** | ✅ Gender, age | ✅ Gender, age, accent, pitch, style, dialect |
| **Audiobook / Stories** | ❌ | ✅ Full audiobook editor + multi-voice stories (EPUB/PDF import, .m4b export) |
| **Languages** | 32 | **646** |
| **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
| **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
| **API Keys** | Required | Not needed |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm · CPU |
| **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
| **TTS Engines** | 1 | **11** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3) |
| **ASR Engines** | 1 | **8** (WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR, isolated Faster-Whisper) |
| **MCP Server** | ❌ | ✅ Use from Claude, Cursor, any MCP client |
| **Self-check** | ❌ | ✅ Diagnostics suite, error journal, scrubbed debug bundles |
| **Customizable** | ❌ Closed | ✅ Fork it, extend it, ship it |
OmniVoice Studio gives you professional-grade AI tools without the subscription or the cloud.
@@ -241,12 +281,21 @@ OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is al
|--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
| **OmniVoice** (default) | 600+ | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Built-in |
| **CosyVoice 3** | 9 + 18 dialects | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | | ✅ Native | ❌ | Varies |
| **GPT-SoVITS** | 5 | ✅ | — | ✅ CUDA/CPU | | ✅ CUDA/CPU | MIT |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | | | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MOSS-TTS-Nano** | 20 | ✅ | | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | | | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | ❌ | ✅ Native | ❌ | Varies |
| **Sherpa-ONNX** | 20+ | — | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **IndexTTS 2** ⚡ | Multi | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
| **OmniVoice GGUF** ⚡ | 600+ | ✅ | ✅ | ✅ CPU | ✅ CPU | ✅ CPU | Built-in |
| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ (8B) | 31 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ (2B) | 24 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only.
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
>
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path.
### ASR Engines
@@ -256,31 +305,35 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
|--------|-------------------------|:---------:|----------|
| **WhisperX** (default) | `whisperx` | ~100 | Dubbing & subtitles — word-level timing via wav2vec2 forced alignment |
| **Faster-Whisper** | `faster-whisper` | ~100 | Fast transcription on Linux / macOS / Windows (CTranslate2) |
| **Faster-Whisper (isolated)** | `faster-whisper-isolated` | ~100 | Same as Faster-Whisper but crash-isolated in a subprocess — an ASR crash won't take down the app |
| **MLX Whisper** | `mlx-whisper` | ~100 | Native Apple Silicon speed (Apple MLX / Metal) |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers (no cuDNN 8 needed) |
| **Parakeet TDT** | `nemo-parakeet` | English + 25 EU | SOTA English accuracy, auto language detection (NVIDIA NeMo, GPU only) |
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. Every engine runs on-device — no API keys, no cloud.
> **GPU without efficient float16?** On older NVIDIA GPUs (Maxwell/Pascal, GTX 16xx) or after a CTranslate2/cuDNN mismatch, the CTranslate2 ASR engines (WhisperX, Faster-Whisper) can't run `float16` and OmniVoice automatically retries on `int8` — no config needed. If transcription still fails, pin the compute type with the `ASR_COMPUTE_TYPE` env var (escape hatch): `ASR_COMPUTE_TYPE=int8` (or `float32` for CPU). Set it to `int8` and restart the backend.
---
## Architecture
```
┌─────────────────────────────────────────────────┐
│ Frontend (React) │
│ DubTab · VoicePreview · BatchQueue · Gallery │
├─────────────────────────────────────────────────┤
│ Backend (FastAPI) │
97 API endpoints · SSE streaming · SQLite
├──────────┬──────────┬──────────┬────────────────┤
│ WhisperX │ Demucs │OmniVoice │ Pyannote │
ASR │ Source TTS │ Diarization
│ Sep.
└──────────┴──────────┴──────────┴────────────────┘
CUDA / MPS / ROCm / CPU (auto-detected)
┌─────────────────────────────────────────────────────────────
Frontend (React)
│ DubTab · VoiceConsole · Stories · Audiobook · Gallery
│ Dictation · BatchQueue · Diagnostics · MCP Client │
├─────────────────────────────────────────────────────────────┤
Backend (FastAPI)
│ 100+ API endpoints · SSE+WSS streaming · SQLite │
├──────────┬──────────┬──────────┬──────────┬────────────────┤
WhisperX │ Demucs │OmniVoice │ Pyannote │ Engine Routing
(+7 ASR │ Source(+10Diariz- │ ↳ GPU preflight
│ engines) │ Sep. │ TTS) │ ation │ ↳ No silent CPU │
└──────────┴──────────┴──────────┴──────────┴────────────────┘
CUDA / MPS / ROCm / CPU (auto-detected + routed)
```
---
@@ -291,27 +344,57 @@ OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictatio
| Category | Features |
|----------|----------|
| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS |
| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags |
| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export |
| **Longform** | Audiobook editor (text/EPUB/PDF → chaptered .m4b), Stories multi-voice editor, two-pass loudnorm mastering, crash-resume for interrupted renders, pronunciation control + SSML-lite prosody |
| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS, per-speaker voice assignment, Smart Fit timing + second-pass QC, dedicated Dub home |
| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags, portable persona bundles (`.ovsvoice`), voice console workspace |
| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export, unlimited-length TTS via sentence-chunked generation |
| **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
| **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading |
| **ASR** | 8 engines (WhisperX, Faster-Whisper, isolated Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet TDT, Moonshine, FunASR/SenseVoice), crash-isolated subprocess backend |
| **TTS** | 11 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3), engine routing with GPU preflight |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading, engine routing (no silent CPU fallback), diagnostics suite & error journal, restricted-network mirror support |
| **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system, UI scale fix for Linux/WebKitGTK |
| **Real-time Events** | WebSocket event bus — instant sidebar refresh on data mutations, exponential backoff reconnect |
| **State Management** | Zustand store migration — `uiSlice`, `pillSlice`, `dubSlice`, `generateSlice`, `prefsSlice`, `glossarySlice` |
| **Desktop** | Cross-platform Tauri installers (macOS DMG, Windows MSI, Linux deb/AppImage), auto-update infrastructure |
| **Windows Hardening** | Cross-platform log paths, Triton workaround, HF symlink bypass, 300s health check timeout |
| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste |
| **Desktop** | Cross-platform Tauri installers (macOS DMG/Intel, Windows MSI, Linux deb/AppImage), auto-update infrastructure, single-instance enforcement, close-to-tray, macOS Gatekeeper fix |
| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste, customizable hotkey, local-LLM transcript refinement |
| **Batch Pipeline** | Full batch TTS: extract → transcribe → translate → generate → mix → export, with live progress tracking |
| **MCP Server** | OmniVoice as a local TTS/STT provider for Claude, Cursor, and any MCP client |
| **Remote Backend** | Point the desktop UI at a remote backend URL with bearer auth (Tailscale-documented) |
| **Reliability** | Stall watchdog on bootstrap splash, per-engine GPU compatibility matrix, actionable errors for non-executable engine binaries, setuptools auto-repair |
### 🔜 Up Next
- 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 📖 **Audiobook Editor** — chapter-aware long-form narration
- 🌐 **Hosted Demo** — try OmniVoice without installing anything
- 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
---
## Sponsor / Donate
OmniVoice Studio is built by one developer using Claude Code and AI agents — and the agent bills are real. Over the last three months I've spent thousands of dollars on Claude subscriptions to keep the features shipping, the bugs fixed, and your issues answered. If OmniVoice has created value for you, helping cover those bills means I can keep developing full-time.
<div align="center">
**This month's agent bill fund**
<img src="https://img.shields.io/badge/raised_%2410_of_%24200-5%25-EAB308?style=for-the-badge" alt="$10 / $200 raised" />
<br/><br/>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_❤️-FF5E5B?style=for-the-badge&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
&nbsp;&nbsp;
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=for-the-badge&logo=paypal&logoColor=white" alt="PayPal" /></a>
&nbsp;&nbsp;
<a href="https://github.com/sponsors/debpalash"><img src="https://img.shields.io/badge/GitHub-Sponsor-ff69b4?style=for-the-badge&logo=github&logoColor=white" alt="GitHub Sponsors" /></a>
<br/>
<sub>Every dollar goes directly to agent bills — keeping OmniVoice development continuous.</sub>
</div>
---
@@ -380,7 +463,7 @@ Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are availab
<details>
<summary><b>Can I add my own TTS engine?</b></summary>
<br/>
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary at the bottom. Six engines are built in: OmniVoice, CosyVoice, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, and KittenTTS. See the <a href="#tts-engines">TTS Engines</a> section for details.
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary. Eleven engines are built in: OmniVoice, CosyVoice 3, GPT-SoVITS, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, KittenTTS, Sherpa-ONNX, plus lazy-registered IndexTTS 2, OmniVoice GGUF, and Supertonic 3. See the <a href="#tts-engines">TTS Engines</a> section for details.
</details>
---
@@ -410,6 +493,9 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
| [**CTranslate2**](https://github.com/OpenNMT/CTranslate2) | Optimized Transformer inference on CPU and GPU |
| [**AudioSeal (Meta)**](https://github.com/facebookresearch/audioseal) | Invisible neural audio watermarking for AI provenance |
| [**Tauri**](https://tauri.app) | Native desktop app framework |
| [**Supertone / Supertonic 3**](https://huggingface.co/Supertone/supertonic-3) | ONNX TTS engine — 31 languages, CPU-efficient |
| [**Sherpa-ONNX**](https://github.com/k2-fsa/sherpa-onnx) | WASM-ready universal TTS/ASR runtime |
| [**GPT-SoVITS**](https://github.com/RVC-Boss/GPT-SoVITS) | Zero-shot TTS engine — 5 languages, RTF 0.014 |
---
@@ -419,7 +505,8 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
If you read this far, you're our kind of person.<br/>
**[⭐ Star this repo](https://github.com/debpalash/OmniVoice-Studio)** so others can find it too.<br/>
**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.
**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.<br/>
**[❤️ Support development](https://ko-fi.com/debpalash)** — fund the AI agent bills that keep OmniVoice shipping.
<br/>
+4
View File
@@ -372,9 +372,13 @@ OmniVoice 配备多引擎 TTS 后端。默认引擎(OmniVoice)始终可用
| **MLX-Audio**Kokoro, Qwen3-TTS, CSM, Dia 等) | 多语言 | 因引擎而异 | 因引擎而异 | ❌ | ✅ 原生 | ❌ | 因引擎而异 |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-v1.5**(8B,可选装) | 31 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts**(2B,可选装) | 24 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **KittenTTS** | 英语 | ❌ | ❌ | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
> **CUDA** = GPU 加速 · **MPS** = Apple Silicon Metal · **CPU** = 随处可运行,大模型较慢 · KittenTTS 和 MOSS-TTS-Nano 可在 CPU 上实时运行 · MLX-Audio 仅限 Apple Silicon。
>
> **MOSS-TTS-v1.5**8B,约 16 GB 权重)和 **dots.tts**2B,约 9 GB 权重)是重量级可选引擎,从本地克隆在独立 venv 中运行——参见 [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) 和 [dots.tts](docs/engines/dots-tts.md)。两者均不支持 Apple Silicon **MPS**(上游仅支持 CUDA/CPU;在 Mac 上以 CPU 运行)。dots.tts 上游仅支持 Linux/macOS——无 Windows 路径。
---
+7 -1
View File
@@ -13,12 +13,18 @@
# Run: uv run pyinstaller backend.spec --noconfirm --clean
import platform
import sys
from PyInstaller.utils.hooks import collect_data_files, collect_all, collect_submodules
from PyInstaller.utils.hooks import collect_data_files, collect_all, collect_submodules, copy_metadata
IS_MAC_ARM = sys.platform == "darwin" and platform.machine() == "arm64"
datas = []
binaries = []
# Bundle the omnivoice package's .dist-info so importlib.metadata.version()
# resolves inside the frozen build. Without it the backend can't read its own
# version and falls back to the literal in backend/core/version.py — which is
# how a 0.3.6 desktop build shipped reporting "0.3.5" in About + bug reports.
datas += copy_metadata('omnivoice')
hiddenimports = [
# Web stack
'uvicorn', 'uvicorn.logging', 'uvicorn.loops', 'uvicorn.loops.auto',
+60 -13
View File
@@ -164,6 +164,7 @@ async def audiobook_cover(cover: UploadFile = File(...)) -> dict:
class AudiobookRequest(BaseModel):
text: str
default_voice: str | None = None # voice profile id; None = engine default
language: str | None = None # None/"Auto" → profile language, else autodetect (#505)
bitrate: str = "128k"
format: str = "m4b" # "m4b" | "mp3"
loudness: str | None = None # None/"off" | "acx" | "podcast" (opt-in)
@@ -215,7 +216,35 @@ def _resolve_voice(profile_id: str | None) -> dict:
return out
def _build_synth(default_voice: str | None) -> dict:
def _resolve_default_language(language: str | None, default_voice: str | None) -> str | None:
"""Pick the language to thread into the longform synth callable.
Priority (mirrors the single-shot /generate path, #533): an explicit
non-Auto request ``language`` wins; otherwise the selected profile's stored
language drives it; otherwise ``None`` (genuine Auto the engine
autodetects, exactly as before). Hardcoding ``None`` here (#505 B2) let the
engine re-autodetect per chunk, so a non-English clone flipped to the wrong
language on short/ambiguous chapters.
"""
if language and language != "Auto":
return language
if default_voice:
from core.db import db_conn
with db_conn() as conn:
row = conn.execute(
"SELECT language FROM voice_profiles WHERE id=?", (default_voice,)
).fetchone()
if row:
try:
prof_lang = row["language"]
except (KeyError, IndexError):
prof_lang = None
if prof_lang and prof_lang != "Auto":
return prof_lang
return None
def _build_synth(default_voice: str | None, language: str | None = None) -> dict:
"""Describe how to synthesize for the active TTS engine.
Returns a dict with ``mode``, ``resolve`` (voice-id resolved refs, cached
@@ -223,6 +252,11 @@ def _build_synth(default_voice: str | None) -> dict:
``get_model``; other engines carry a ready ``synth`` + ``sample_rate``.
:func:`_prepare_synth` turns this into a uniform ``(synth, sr, resolve,
engine_id)`` once the (async) model is in hand.
``language`` (already resolved by :func:`_resolve_default_language`) is
threaded into every chunk's ``generate`` so a non-English clone stays in its
language instead of re-autodetecting per chunk (#505 B2). ``None`` keeps the
engine's autodetect behavior unchanged.
"""
from services.tts_backend import OmniVoiceBackend, active_backend_id, get_backend_class
@@ -239,14 +273,14 @@ def _build_synth(default_voice: str | None) -> dict:
if cls is OmniVoiceBackend:
from services.model_manager import get_model
return {"mode": "omnivoice", "resolve": resolve,
"engine_id": engine_id, "get_model": get_model}
"engine_id": engine_id, "get_model": get_model, "language": language}
backend = cls()
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
return backend.generate(
text, language=None, ref_audio=v["ref_audio"],
text, language=language, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0,
)
@@ -254,20 +288,22 @@ def _build_synth(default_voice: str | None) -> dict:
"synth": synth, "sample_rate": backend.sample_rate}
async def _prepare_synth(default_voice: str | None):
async def _prepare_synth(default_voice: str | None, language: str | None = None):
"""Resolve :func:`_build_synth` into ``(synth, sample_rate, resolve,
engine_id)`` awaiting the OmniVoice model load when needed. Shared by the
full job and the per-chapter preview."""
info = _build_synth(default_voice)
full job and the per-chapter preview. ``language`` is threaded into every
chunk so a non-English clone holds its language (#505 B2)."""
info = _build_synth(default_voice, language=language)
resolve, engine_id = info["resolve"], info["engine_id"]
if info["mode"] == "omnivoice":
lang = info["language"]
model = await info["get_model"]()
sr = getattr(model, "sampling_rate", 24000)
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
return model.generate(
text=text, language=None, ref_audio=v["ref_audio"],
text=text, language=lang, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0,
)[0]
@@ -323,6 +359,7 @@ class AudiobookPreviewRequest(BaseModel):
text: str
chapter_index: int = 0
default_voice: str | None = None
language: str | None = None # None/"Auto" → profile language, else autodetect
lexicon: dict | None = None
@@ -346,7 +383,10 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
chapter = plan.chapters[req.chapter_index]
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache") # shared with _render_longform_sse
os.makedirs(cache_dir, exist_ok=True)
synth, sr, resolve, engine_id = await _prepare_synth(req.default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
req.default_voice,
language=_resolve_default_language(req.language, req.default_voice),
)
loop = asyncio.get_running_loop()
wav_path, dur, was_cached = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
@@ -364,6 +404,7 @@ async def _render_longform_sse(
plan,
*,
default_voice: str | None,
language: str | None = None,
fmt: str = "m4b",
bitrate: str = "128k",
loudness: str | None = None,
@@ -412,7 +453,8 @@ async def _render_longform_sse(
for c in plan.chapters
],
params={
"default_voice": default_voice, "fmt": fmt, "bitrate": bitrate,
"default_voice": default_voice, "language": language,
"fmt": fmt, "bitrate": bitrate,
"loudness": loudness, "cover_path": cover_path,
"metadata": metadata, "lexicon": lexicon,
},
@@ -453,7 +495,9 @@ async def _render_longform_sse(
loop = asyncio.get_running_loop()
try:
synth, sr, resolve, engine_id = await _prepare_synth(default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
default_voice, language=_resolve_default_language(language, default_voice)
)
total = len(plan.chapters)
chapter_files: list[str] = []
@@ -557,7 +601,8 @@ async def audiobook_synthesize(req: AudiobookRequest):
plan = parse_audiobook_script(req.text, default_voice=req.default_voice)
return StreamingResponse(
_render_longform_sse(
plan, default_voice=req.default_voice, fmt=req.format, bitrate=req.bitrate,
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
lexicon=req.lexicon, job_type="audiobook",
),
@@ -582,6 +627,7 @@ class LongformChapter(BaseModel):
class LongformRenderRequest(BaseModel):
chapters: list[LongformChapter] = []
default_voice: str | None = None
language: str | None = None # None/"Auto" → profile language, else autodetect (#505)
bitrate: str = "128k"
format: str = "m4b"
loudness: str | None = None
@@ -612,7 +658,8 @@ async def longform_render(req: LongformRenderRequest):
plan = AudiobookPlan(chapters=chapters)
return StreamingResponse(
_render_longform_sse(
plan, default_voice=req.default_voice, fmt=req.format, bitrate=req.bitrate,
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
lexicon=req.lexicon, job_type="story",
),
@@ -702,7 +749,7 @@ async def resume_longform(job_id: str):
# never names a work dir / output file (defence-in-depth path-injection).
return StreamingResponse(
_render_longform_sse(
plan, default_voice=p.get("default_voice"),
plan, default_voice=p.get("default_voice"), language=p.get("language"),
fmt=p.get("fmt", "m4b"), bitrate=p.get("bitrate", "128k"),
loudness=p.get("loudness"), cover_path=p.get("cover_path"),
metadata=p.get("metadata"), lexicon=p.get("lexicon"),
+20 -1
View File
@@ -436,7 +436,7 @@ async def dub_transcribe_stream(
)
scene_cuts = job.get("scene_cuts") or []
async def gen():
async def _gen_body():
if preflight_error:
yield _sse_event("error", {"detail": preflight_error})
return
@@ -856,6 +856,25 @@ async def dub_transcribe_stream(
})
yield _sse_event("done", {})
async def gen():
# Terminal-event guard (#516): the SSE stream must NEVER close without a
# terminal event. Any unanticipated exception in the body (e.g. an ASR
# load that escapes the per-chunk handler) previously dropped the
# connection, which the frontend can only report as "stream dropped,
# likely ASR failed" — hiding the real cause. Emit a structured `error`
# (with the actionable hint from build_failure) then `done`, so the user
# sees the real failure + a Retry instead of a silent disconnect.
try:
async for ev in _gen_body():
yield ev
except Exception as e: # noqa: BLE001 — last-resort stream finalizer
logger.exception("transcribe stream crashed (job=%s)", job_id)
from core.failure import build_failure
f = build_failure(e, stage="transcribe", include_diagnostic=False)
detail = f["reason"] + (f"{f['hint']}" if f.get("hint") else "")
yield _sse_event("error", {"detail": detail, "retryable": True})
yield _sse_event("done", {})
return StreamingResponse(
gen(),
media_type="text/event-stream",
+14
View File
@@ -430,6 +430,20 @@ async def generate_speech(
used_seed = row["seed"]
if language == "Auto":
language = None
# #533: a profile's stored language must drive generation when the
# request didn't pin one. Without this the German (etc.) archetype
# generates with language=None and the model drifts to English —
# even though the archetype PREVIEW renders correctly (archetypes.py
# passes the language). An EXPLICIT non-Auto request language still
# wins; we only fill the gap. `row` is a sqlite3.Row, so guard the
# column lookup for pre-language DBs mid-upgrade.
if language is None:
try:
prof_lang = row["language"]
except (KeyError, IndexError):
prof_lang = None
if prof_lang and prof_lang != "Auto":
language = prof_lang
elif ref_audio is not None:
try:
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as f:
+69 -6
View File
@@ -206,6 +206,52 @@ def _migrate(conn, current: int) -> int:
return current
def _reconcile_additive_columns(conn) -> None:
"""Make the live schema converge to ``_BASE_SCHEMA`` by ADDing any column the
canonical schema declares but an existing table is missing the belt for
when alembic can't run on an upgraded DB.
``CREATE TABLE IF NOT EXISTS`` (init_db) never adds columns to a table that
already exists, the legacy ``_migrate`` only knows pre-0.3 columns, and
``_run_alembic_upgrade`` swallows failures. So a DB whose ``alembic_version``
is stamped at a removed revision (e.g. after running a preview build), or
where alembic isn't importable in the bundled interpreter, would otherwise
lose every alembic-era additive column forever the ``no such column:
consent_audio_path`` 500 (#552/#547), and the same class for
``kind``/``vd_states``/``is_demo``/.... Additive only: never drops or retypes
a column, so it is safe and backward-compatible with existing user data. The
canonical names/types/defaults come solely from ``_BASE_SCHEMA`` (developer
controlled), so the ALTER is injection-safe.
"""
canon = sqlite3.connect(":memory:")
try:
canon.executescript(_BASE_SCHEMA)
_tables_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'"
live_tables = {r[0] for r in conn.execute(_tables_sql)}
for table in (r[0] for r in canon.execute(_tables_sql)):
if table not in live_tables:
continue # whole table missing → init_db's CREATE already made it
have = {r[1] for r in conn.execute(f"PRAGMA table_info({table})")}
# (cid, name, type, notnull, dflt_value, pk)
for _cid, name, ctype, notnull, dflt, _pk in canon.execute(f"PRAGMA table_info({table})"):
if name in have or not _IDENT_RE.match(name):
continue
ddl = f'ALTER TABLE "{table}" ADD COLUMN "{name}" {ctype or "TEXT"}'
if dflt is not None:
ddl += f" DEFAULT {dflt}"
elif notnull:
ddl += " DEFAULT ''" # SQLite requires a default to ADD a NOT NULL column
try:
conn.execute(ddl)
logger.info("schema reconcile: added missing column %s.%s", table, name)
except sqlite3.OperationalError as exc:
if "duplicate column" not in str(exc).lower():
logger.warning("schema reconcile ALTER %s.%s failed: %s", table, name, exc)
conn.commit()
finally:
canon.close()
def init_db():
conn = get_db()
try:
@@ -214,6 +260,11 @@ def init_db():
new_version = _migrate(conn, version)
if new_version != version:
conn.execute(f"PRAGMA user_version = {new_version}")
# Converge any alembic-era additive columns that CREATE TABLE IF NOT
# EXISTS + the legacy _migrate don't add to a pre-existing table
# (consent_audio_path, kind, ...). Runs regardless of whether alembic
# below succeeds, so an unrunnable alembic can't leave a 500-ing schema.
_reconcile_additive_columns(conn)
conn.commit()
finally:
conn.close()
@@ -227,10 +278,12 @@ def init_db():
def _run_alembic_upgrade() -> None:
"""Best-effort `alembic upgrade head` on startup. Non-fatal: if alembic
isn't reachable (e.g. user running a stripped-down install or migrations
were already applied out-of-band), log a warning and move on. The
_BASE_SCHEMA CREATE TABLE IF NOT EXISTS above guarantees the runtime
schema is correct regardless."""
isn't reachable (e.g. a stripped-down install) or its version is stamped at
a revision no longer in versions/ (e.g. after running a preview build), log
a warning and move on. The schema is still kept correct by
_reconcile_additive_columns (run in init_db above and again here on failure)
CREATE TABLE IF NOT EXISTS alone does NOT add columns to a pre-existing
table, so the reconcile is what actually guarantees additive columns land."""
try:
import os
from alembic import command
@@ -248,6 +301,16 @@ def _run_alembic_upgrade() -> None:
cfg.set_main_option("sqlalchemy.url", f"sqlite:///{DB_PATH}")
command.upgrade(cfg, "head")
except Exception as exc:
# Don't block startup on a migration tooling problem. The runtime
# schema is already correct via _BASE_SCHEMA.
# Don't block startup on a migration tooling problem. Converge the schema
# directly so a swallowed failure (alembic not importable, or
# alembic_version stamped at a removed revision) still lands the additive
# columns instead of 500-ing on `no such column` (#552/#547).
logger.warning("alembic upgrade head skipped: %s", exc)
try:
conn = get_db()
try:
_reconcile_additive_columns(conn)
finally:
conn.close()
except Exception as exc2: # noqa: BLE001
logger.warning("schema reconcile after alembic failure also failed: %s", exc2)
+30
View File
@@ -37,6 +37,11 @@ _HINTS: dict[str, str] = {
"APPIMAGE_WEBKIT_WHITESCREEN": "Launch with WEBKIT_DISABLE_DMABUF_RENDERER=1 set.",
"HF_AUTH_FAILED": "Set a valid HF_TOKEN in Settings → Hugging Face and retry.",
"PYANNOTE_LICENSE_REQUIRED": "Accept the pyannote model licenses on Hugging Face, then retry.",
"COMPUTE_TYPE_UNSUPPORTED": "Your GPU doesn't support float16 — OmniVoice retried on int8. If transcription still fails, set OMNIVOICE/ASR_COMPUTE_TYPE=int8 or use CPU.",
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
}
@@ -55,10 +60,35 @@ def classify(reason: str) -> str:
return "APPIMAGE_WEBKIT_WHITESCREEN"
if "pyannote" in low or ("gated" in low and "model" in low) or "accept the" in low:
return "PYANNOTE_LICENSE_REQUIRED"
# ASR robustness (#551 / #549): name the class so the no-segments toast is
# actionable. Place before the generic returns so a compute-type/transformers
# failure gets its hint rather than falling through to "".
if "compute type" in low or "efficient float16" in low:
return "COMPUTE_TYPE_UNSUPPORTED"
if "could not import module" in low or "autofeatureextractor" in low:
return "TRANSFORMERS_IMPORT"
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
):
return "HF_AUTH_FAILED"
# Video download (#554/#536): a non-downloadable URL shape vs a transient
# network drop — both previously surfaced as a bare yt-dlp string with no
# next step. UNSUPPORTED first (more specific) so "Unable to download video:
# Broken pipe" still classifies as a network blip.
if "unsupported url" in low or "no video formats" in low or "is not a valid url" in low:
return "UNSUPPORTED_VIDEO_URL"
if (
"broken pipe" in low
or "connection reset" in low
or "unable to download video" in low
or "remote end closed" in low
or "timed out" in low
):
return "VIDEO_DOWNLOAD_NETWORK"
# A relocated/corrupted venv whose interpreter can't bootstrap its stdlib —
# the Rust self-heal rebuilds it; this names the class for the toast.
if "no module named 'encodings'" in low:
return "BROKEN_VENV"
return ""
+33 -4
View File
@@ -2,14 +2,43 @@
Read from the installed package metadata (driven by ``pyproject.toml``) so the
FastAPI/API version and exported-bundle metadata never drift to a stale literal
again (the prior "0.4.0" / "0.2.7" bug). Falls back to a literal only when
running from a raw source checkout that was never ``uv sync``'d.
the prior "0.4.0" / "0.2.7" bug, and the v0.3.6 desktop build that reported
"0.3.5" because the *frozen* backend couldn't read its own metadata.
Resolution order:
1. installed package metadata correct in any ``uv sync``'d env and, thanks
to ``copy_metadata('omnivoice')`` in ``backend.spec``, in the frozen build;
2. ``pyproject.toml`` walked up from this file correct for a raw source
checkout that was never installed;
3. ``_FALLBACK_VERSION`` a last resort, kept in lockstep with the four
version files by ``tests/test_app_version.py`` so it can never silently
drift again.
"""
from __future__ import annotations
import re
from importlib.metadata import PackageNotFoundError, version
from pathlib import Path
# Last-resort literal. Guarded by
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.3.7"
def _fallback_version() -> str:
"""Version for contexts where package metadata is unavailable."""
for parent in Path(__file__).resolve().parents:
pyproject = parent / "pyproject.toml"
if pyproject.is_file():
match = re.search(r'(?m)^version\s*=\s*"([^"]+)"', pyproject.read_text())
if match:
return match.group(1)
return _FALLBACK_VERSION
try:
APP_VERSION = version("omnivoice")
except PackageNotFoundError: # non-installed source checkout
APP_VERSION = "0.3.5"
except PackageNotFoundError: # frozen build w/o metadata, or non-installed checkout
APP_VERSION = _fallback_version()
+30 -2
View File
@@ -50,6 +50,22 @@ def _recv(stream):
return json.loads(bytes(body).decode("utf-8"))
# NOTE: keep this compute_type fallback in lockstep with
# services/asr_backend.py:_compute_type_candidates / _is_compute_type_error.
# This sidecar runs in a child proc with a clean import path, so we duplicate a
# tiny copy rather than cross-importing the heavy services package (#551).
def _ct_candidates(device):
override = os.environ.get("ASR_COMPUTE_TYPE")
if override:
return [override]
return ["float16", "int8_float16", "int8"] if device == "cuda" else ["int8", "float32"]
def _is_ct_error(msg):
low = msg.lower()
return "compute type" in low or "efficient float16" in low
def _get_model():
global _model
if _model is None:
@@ -60,8 +76,20 @@ def _get_model():
device = "cuda" if torch.cuda.is_available() else "cpu"
except Exception:
device = "cpu"
compute = "float16" if device == "cuda" else "int8"
_model = WhisperModel(name, device=device, compute_type=compute)
# Degrade fp16 → int8 rather than crash on GPUs without efficient fp16
# (older Maxwell/Pascal, GTX 16xx, CTranslate2/cuDNN mismatch) (#551).
last_err = None
for compute in _ct_candidates(device):
try:
_model = WhisperModel(name, device=device, compute_type=compute)
break
except (ValueError, RuntimeError) as e:
last_err = e
if _is_ct_error(str(e)):
continue
raise
else:
raise last_err
return _model
+187
View File
@@ -0,0 +1,187 @@
"""dots.tts sidecar package (issue #498).
dots.tts is rednote-hilab's 2B fully-continuous autoregressive TTS — widely
cited as among the strongest open zero-shot voice-cloning models. 24
languages, 48 kHz output, Apache-2.0 (code + checkpoints).
It runs in its own subprocess **and its own venv**, isolated from the
OmniVoice parent, for the same ``transformers`` reason as IndexTTS and
MOSS-TTS-v1.5: dots.tts pins ``transformers==4.57.0`` (verified against
``constraints/recommended.txt``), while OmniVoice pins
``transformers>=5.3.0``. The two cannot share one interpreter.
Cross-platform honesty (the strict default-parity rule): dots.tts's
upstream package declares **Linux + macOS** classifiers only **no
Windows** and its device code is **CUDA-or-CPU with no MPS branch**
(verified in ``runtime.py``). So:
* It is **opt-in** (engine-picker selection + a user-provided clone),
never a default so it never becomes a broken default on any platform.
* ``is_available()`` returns ``False`` with a clear reason on **Windows**
rather than offering an engine that can't run there. Windows users are
pointed at WSL2 / a Linux or macOS host.
* ``gpu_compat = ("cuda", "cpu")`` no MPS claim. On Apple Silicon the
upstream package runs on CPU (slow but correct); the faster MLX path is
a community port we deliberately don't auto-wire here.
Three public entry points: ``DotsTTSBackend`` (this module), ``main.py``
(sidecar, runs under dots.tts's ``transformers==4.57`` venv — never imported
by the parent), and ``bootstrap.py`` (venv probe + lazy bootstrap).
"""
from __future__ import annotations
import logging
import sys
from typing import TYPE_CHECKING
from services.subprocess_backend import SubprocessBackend
if TYPE_CHECKING:
import torch # noqa: F401
logger = logging.getLogger("omnivoice.dots_tts")
class DotsTTSBackend(SubprocessBackend):
"""dots.tts (rednote-hilab) — 2B, 24 langs, zero-shot clone, CUDA/CPU.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv (``transformers==4.57.0``). First synthesize cold-loads
the ~9 GB checkpoint (bf16 on CUDA); subsequent calls reuse the process.
Installation (OmniVoice prefers a user's existing ``${DIR}/.venv``)::
git clone https://github.com/rednote-hilab/dots.tts.git
cd dots.tts
uv venv && uv pip install -e . -c constraints/recommended.txt
Set ``OMNIVOICE_DOTS_TTS_DIR`` to the clone root. OmniVoice creates
``backend/engines/dots_tts/.venv`` lazily on first launch if no venv
exists yet; the user's existing ``${DIR}/.venv`` is preferred if present.
Best cloning quality uses the ``dots.tts-soar`` checkpoint (the default)
and BOTH a reference clip and its exact transcript (continuation
cloning). License: Apache-2.0.
"""
id = "dots-tts"
display_name = (
"dots.tts (2B, 24 langs, zero-shot clone, CUDA/CPU, 48 kHz, Apache-2.0)"
)
supports_voice_design = False # requires ref audio for timbre cloning
# dots.tts emits 48 kHz (verified via checkpoint vocoder.sample_rate).
_DEFAULT_SAMPLE_RATE = 48000
# CUDA + CPU only; no MPS branch in upstream runtime.py.
gpu_compat = ("cuda", "cpu")
# ── availability ───────────────────────────────────────────────────────
@classmethod
def is_available(cls) -> tuple[bool, str]:
# Cross-platform parity: dots.tts upstream is Linux/macOS-only (no
# Windows classifier, no Windows install path). Refuse cleanly on
# Windows instead of advertising an engine that can't run.
if sys.platform == "win32":
return False, (
"dots.tts is not supported on Windows — upstream targets "
"Linux and macOS only. Run OmniVoice under WSL2, or use a "
"Linux/macOS host. See docs/engines/dots-tts.md."
)
# Do NOT import dots_tts here: it pins transformers==4.57, which can't
# coexist with the parent's transformers>=5.3 in one interpreter —
# the reason for the subprocess isolation. Verify the venv on disk
# only; a real health-check is gated on the user's "Test engine"
# action in Settings.
from engines.dots_tts.bootstrap import (
DOTS_TTS_SIDECAR_SCRIPT,
is_dots_tts_installed,
)
if not is_dots_tts_installed():
return False, (
"dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR to your "
"dots.tts clone (the directory containing pyproject.toml) and "
"restart OmniVoice. CUDA or CPU only (no MPS). See "
"docs/engines/dots-tts.md for the full install walk-through."
)
if not DOTS_TTS_SIDECAR_SCRIPT.exists():
return False, (
"dots.tts sidecar script missing at "
f"{DOTS_TTS_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok (CUDA when present, else CPU)"
@classmethod
def venv_python(cls):
from engines.dots_tts.bootstrap import resolve_dots_tts_venv
return resolve_dots_tts_venv()
@classmethod
def sidecar_script(cls):
from engines.dots_tts.bootstrap import DOTS_TTS_SIDECAR_SCRIPT
return DOTS_TTS_SIDECAR_SCRIPT
# ── TTSBackend protocol ────────────────────────────────────────────────
@property
def sample_rate(self) -> int:
return self._DEFAULT_SAMPLE_RATE
@property
def supported_languages(self) -> list[str]:
# 24 languages with auto-detect; expose "multi" on the protocol
# surface and translate the caller's language at synthesize time.
return ["multi"]
# ── generate (parent-side arbitration) ─────────────────────────────────
def generate(self, text: str, **kw) -> "torch.Tensor":
"""Synthesize one utterance through the dots.tts sidecar.
kwargs honored:
* ``ref_audio`` reference clip path ``prompt_audio_path``
(zero-shot cloning). Optional.
* ``ref_text`` the reference transcript ``prompt_text``.
Best cloning fidelity ("continuation"). Upstream
REQUIRES ``prompt_audio_path`` when ``prompt_text``
is set, so we drop a stray ref_text with no
ref_audio rather than let the sidecar raise.
* ``language`` ISO code / name / None (auto-detect).
* ``num_step`` flow-matching steps ``num_steps`` (default 10;
use 4 for the ``dots.tts-mf`` checkpoint).
* ``guidance_scale`` CFG (default 1.2; >2 amplifies energy).
Returns a tensor of shape (1, n_samples) at :attr:`sample_rate`.
"""
forwarded: dict = {}
ref_audio = kw.get("ref_audio")
if ref_audio:
forwarded["ref_audio"] = ref_audio
ref_text = kw.get("ref_text")
if ref_text:
# continuation cloning — only valid alongside ref_audio.
forwarded["ref_text"] = ref_text
elif kw.get("ref_text"):
logger.info(
"dots-tts: ref_text supplied without ref_audio; ignoring "
"(upstream requires prompt_audio_path when prompt_text is set)."
)
language = kw.get("language")
if language:
forwarded["language"] = str(language)
# OmniVoice's generic num_step default is 16; dots.tts's own default
# is 10. Honor an explicit value, else use the dots-appropriate 10.
num_step = kw.get("num_step")
forwarded["num_steps"] = int(num_step) if num_step is not None else 10
# dots.tts's own CFG default is 1.2 (the generic 2.0 over-energises).
guidance = kw.get("guidance_scale")
forwarded["guidance_scale"] = float(guidance) if guidance is not None else 1.2
return super().generate(text, **forwarded)
__all__ = ["DotsTTSBackend"]
+226
View File
@@ -0,0 +1,226 @@
"""dots.tts venv probe + lazy bootstrap (issue #498).
Resolves which Python interpreter runs the dots.tts sidecar. Mirrors
``engines.indextts.bootstrap`` / ``engines.moss_tts_v15.bootstrap`` because
dots.tts has the same shape of problem: a hard ``transformers==4.57.0`` pin
that conflicts with the parent's ``transformers>=5.3`` — so it runs in its
own venv.
Probe order (priority existing power-user installs win, zero migration):
1. ``${OMNIVOICE_DOTS_TTS_DIR}/.venv/`` the user's clone-level venv.
2. ``backend/engines/dots_tts/.venv/`` this package's own venv.
3. Bootstrap: ``uv venv`` then ``uv pip install -e <clone> -c
<clone>/constraints/recommended.txt`` (the upstream-pinned stack:
torch==2.8.0, transformers==4.57.0, ).
Caching: memoised after first success. Tests reset via :func:`invalidate`.
Security: same posture as IndexTTS bootstrap never touches HF_TOKEN; the
sidecar's stderr is redacted by the parent's ``HFTokenRedactor``; the
editable install comes from a user-controlled clone they already trust.
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger("omnivoice.dots_tts.bootstrap")
#: Absolute path to the sidecar entrypoint.
DOTS_TTS_SIDECAR_SCRIPT: Path = Path(__file__).parent / "main.py"
#: This package's owned venv (Probe 2).
_ENGINES_VENV_DIR: Path = Path(__file__).parent / ".venv"
#: Env var pointing at the user's dots.tts clone root.
_CLONE_DIR_ENV: str = "OMNIVOICE_DOTS_TTS_DIR"
#: Per-process resolution cache. Cleared by :func:`invalidate` for tests.
_resolved_python: Optional[Path] = None
_IMPORT_PROBE_TIMEOUT_S = 15
_UV_VENV_TIMEOUT_S = 120
_UV_PIP_INSTALL_TIMEOUT_S = 1800
# ── public API ────────────────────────────────────────────────────────────
def invalidate() -> None:
"""Clear the resolved-python cache. Tests call this between scenarios."""
global _resolved_python
_resolved_python = None
def is_dots_tts_installed() -> bool:
"""Cheap file-existence check for a usable dots.tts venv. Does NOT spawn
the venv Python that's saved for :func:`resolve_dots_tts_venv`."""
for cand in _probe_paths():
if cand.is_file():
return True
return False
def resolve_dots_tts_venv() -> Path:
"""Resolve the sidecar's Python interpreter (probe order in the module
docstring). Memoised. Raises :exc:`RuntimeError` if none can be located
and the bootstrap path is unavailable."""
global _resolved_python
if _resolved_python is not None:
return _resolved_python
clone_dir = os.environ.get(_CLONE_DIR_ENV)
# Probe 1 — user's clone-level venv.
if clone_dir:
cand = _venv_python_path(Path(clone_dir) / ".venv")
if cand.is_file() and _venv_can_import_dots(cand):
logger.info(
"dots.tts venv resolved from %s: %s", _CLONE_DIR_ENV, cand,
)
_resolved_python = cand
return cand
# Probe 2 — this package's own venv.
cand = _venv_python_path(_ENGINES_VENV_DIR)
if cand.is_file() and _venv_can_import_dots(cand):
logger.info("dots.tts venv resolved from engines path: %s", cand)
_resolved_python = cand
return cand
# Probe 3 — bootstrap.
if not clone_dir:
raise RuntimeError(
"dots.tts is not installed. Set the "
f"{_CLONE_DIR_ENV} environment variable to your dots.tts clone "
"(the directory that contains pyproject.toml and constraints/), "
"then restart OmniVoice. See docs/engines/dots-tts.md for the "
"full install walk-through."
)
cand = _bootstrap_engines_venv(Path(clone_dir))
_resolved_python = cand
return cand
# ── internals ─────────────────────────────────────────────────────────────
def _venv_python_path(venv_dir: Path) -> Path:
if sys.platform == "win32":
return venv_dir / "Scripts" / "python.exe"
return venv_dir / "bin" / "python"
def _probe_paths() -> list[Path]:
out: list[Path] = []
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
out.append(_venv_python_path(Path(clone_dir) / ".venv"))
out.append(_venv_python_path(_ENGINES_VENV_DIR))
return out
def _venv_can_import_dots(python_path: Path) -> bool:
"""Spawn the candidate python and verify ``import dots_tts.runtime`` works.
Bounded by ``_IMPORT_PROBE_TIMEOUT_S``. False on any failure."""
try:
proc = subprocess.run(
[str(python_path), "-c", "import dots_tts.runtime"],
capture_output=True,
timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
logger.debug("dots.tts import probe failed for %s: %s", python_path, exc)
return False
if proc.returncode != 0:
logger.debug(
"dots.tts import probe non-zero for %s: %s",
python_path,
proc.stderr.decode("utf-8", errors="replace")[:200],
)
return False
return True
def _locate_uv() -> Optional[str]:
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
if bundled and Path(bundled).is_file():
return bundled
return shutil.which("uv")
def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"""Create engines/dots_tts/.venv and editable-install the user's clone
with the upstream constraints file."""
uv = _locate_uv()
if not uv:
raise RuntimeError(
"uv is required to bootstrap the dots.tts venv but was not found "
"on PATH (and OMNIVOICE_BUNDLED_UV was not set). Install uv from "
"https://docs.astral.sh/uv/ and re-launch OmniVoice, or set "
"OMNIVOICE_BUNDLED_UV to the absolute path of a uv binary."
)
logger.info(
"Bootstrapping dots.tts venv at %s from %s (this can take several "
"minutes on first launch)", _ENGINES_VENV_DIR, clone_dir,
)
try:
subprocess.run(
[uv, "venv", str(_ENGINES_VENV_DIR)],
check=True, timeout=_UV_VENV_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
f"uv venv failed for dots.tts bootstrap at {_ENGINES_VENV_DIR}: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
python_path = _venv_python_path(_ENGINES_VENV_DIR)
install_cmd = [
uv, "pip", "install",
"--python", str(python_path),
"-e", str(clone_dir),
]
# Apply the upstream pin set when it ships with the clone.
constraints = clone_dir / "constraints" / "recommended.txt"
if constraints.is_file():
install_cmd += ["-c", str(constraints)]
try:
subprocess.run(
install_cmd, check=True,
timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install -e failed during dots.tts bootstrap "
f"({clone_dir}): "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}. "
"See docs/engines/dots-tts.md."
) from exc
if not _venv_can_import_dots(python_path):
raise RuntimeError(
"dots.tts bootstrap completed but `import dots_tts.runtime` still "
f"fails from {python_path}. Verify that {clone_dir} is a valid "
"dots.tts clone. See docs/engines/dots-tts.md."
)
logger.info("dots.tts venv bootstrap successful: %s", python_path)
return python_path
__all__ = [
"DOTS_TTS_SIDECAR_SCRIPT",
"invalidate",
"is_dots_tts_installed",
"resolve_dots_tts_venv",
]
+255
View File
@@ -0,0 +1,255 @@
"""dots.tts sidecar entry point (issue #498).
Runs inside ``engines/dots_tts/.venv`` (or the user's existing
``${OMNIVOICE_DOTS_TTS_DIR}/.venv``) with ``transformers==4.57.0``, isolated
from the OmniVoice parent (``transformers>=5.3``). Same isolation rationale
as the IndexTTS / MOSS-TTS-v1.5 sidecars.
Stdlib-only at import time; ``dots_tts`` + torch are imported lazily on the
first synthesize op so the ``ready`` frame fits inside the parent's 30 s
spawn handshake even on a cold filesystem.
Wire protocol length-prefixed JSON over stdin/stdout, byte-identical to
``backend/services/subprocess_backend.py``::
[ 4-byte big-endian uint32 length ][ N bytes UTF-8 JSON ]
Op flow:
1. Sidecar -> parent: {"op": "ready", "engine": "dots-tts",
"sample_rate": 48000}
2. parent -> sidecar: {"op": "ping"} -> {"op": "pong", "vram_mb": N}
3. parent -> sidecar: {"op": "synthesize", "text": "...",
"ref_audio": "/path/ref.wav",
"ref_text": "transcript", "language": "EN",
"num_steps": 10, "guidance_scale": 1.2}
-> {"op": "progress", ...} (cold load) then
-> {"op": "audio", "audio_pcm_b64": "...", "sample_rate": 48000,
"n_samples": N}
4. parent -> sidecar: {"op": "shutdown"} -> exit 0
Restrictions: NO imports from OmniVoice parent code (different venv). NO
logging of ``os.environ`` contents. Single-frame DoS cap matches the
parent's ``MAX_FRAME_BYTES``.
"""
from __future__ import annotations
import base64
import json
import os
import struct
import sys
import traceback
# Mirrors backend/services/subprocess_backend.py::MAX_FRAME_BYTES.
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: dots.tts emits 48 kHz (checkpoint vocoder.sample_rate). Advertised in the
#: ready frame; the real value is re-read from each generate() result.
DOTS_SAMPLE_RATE = 48000
#: Default checkpoint. ``-soar`` is the best-cloning variant; ``-mf`` is the
#: fastest (use num_steps=4). Overridable for air-gapped / mirror installs.
_DEFAULT_REPO = "rednote-hilab/dots.tts-soar"
# ── wire protocol ─────────────────────────────────────────────────────────
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
def _recv(stream):
header = stream.read(4)
if len(header) < 4:
return None # EOF
(n,) = struct.unpack("!I", header)
if n > MAX_FRAME_BYTES:
raise IOError(f"frame too large: {n}")
body = bytearray()
while len(body) < n:
chunk = stream.read(n - len(body))
if not chunk:
raise IOError("short read")
body.extend(chunk)
return json.loads(bytes(body).decode("utf-8"))
def _measure_vram_mb() -> float:
"""This sidecar's own GPU memory in MB (MM2-08). 0 on CPU. Never raises."""
try:
import torch
if torch.cuda.is_available():
return round(torch.cuda.memory_allocated() / (1024 ** 2), 1)
except Exception:
pass
return 0.0
# ── model loading (lazy, on first synthesize) ─────────────────────────────
# Module-level singleton — (runtime,). Device is auto-selected inside the
# dots.tts runtime (cuda-or-cpu, no MPS); we don't pass a device.
_runtime = None
def _load_runtime(stdout):
"""Cold-construct the dots.tts runtime.
``DotsTtsRuntime.from_pretrained`` auto-selects cuda-or-cpu internally
(no MPS path). precision is bf16 on CUDA; on CPU we fall back to fp32
(bf16 CPU kernels are spotty). Both overridable via env.
"""
global _runtime
if _runtime is not None:
return _runtime
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
import torch
from dots_tts.runtime import DotsTtsRuntime # type: ignore[import-not-found]
repo = os.environ.get("OMNIVOICE_DOTS_TTS_MODEL", _DEFAULT_REPO)
default_precision = "bfloat16" if torch.cuda.is_available() else "float32"
precision = os.environ.get("OMNIVOICE_DOTS_TTS_PRECISION", default_precision)
optimize = os.environ.get("OMNIVOICE_DOTS_TTS_OPTIMIZE", "0") == "1"
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 50})
_runtime = DotsTtsRuntime.from_pretrained(
repo,
precision=precision,
optimize=optimize,
)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _runtime
def _tensor_to_pcm_b64(audio, sample_rate: int) -> tuple[str, int, int]:
"""Convert a torch waveform tensor (1, N) in [-1, 1] to base64 int16 PCM."""
import numpy as np
arr = audio.detach().to("cpu").float().numpy()
arr = np.asarray(arr, dtype=np.float32).squeeze()
if arr.ndim > 1:
arr = arr.mean(axis=0) # defensive downmix to mono
arr = np.clip(arr, -1.0, 1.0)
pcm = (arr * 32767.0).astype(np.int16).tobytes()
return base64.b64encode(pcm).decode("ascii"), int(sample_rate), int(arr.shape[0])
def _normalize_language(raw):
"""Map OmniVoice's language value to what dots.tts accepts, or None.
dots.tts accepts None/"auto_detect", ISO codes upper-cased ("EN"/"ZH"),
or names ("english"). A 2-letter ISO code is upper-cased; anything else
is passed through; empty / "auto" None (auto-detect)."""
if not raw or not isinstance(raw, str):
return None
s = raw.strip()
if not s or s.lower() == "auto":
return None
if len(s) == 2 and s.isalpha():
return s.upper()
return s
def _handle_synthesize(msg: dict, stdout) -> None:
"""Dispatch one synthesize request. Emits the audio frame or raises."""
text = msg.get("text")
if not text or not isinstance(text, str):
raise ValueError("synthesize: missing or non-string 'text'")
runtime = _load_runtime(stdout)
gen_kwargs: dict = {
"text": text,
"num_steps": int(msg.get("num_steps", 10)),
"guidance_scale": float(msg.get("guidance_scale", 1.2)),
}
ref_audio = msg.get("ref_audio")
if ref_audio:
gen_kwargs["prompt_audio_path"] = ref_audio
ref_text = msg.get("ref_text")
if ref_text:
# continuation cloning — upstream requires prompt_audio_path when
# prompt_text is set (the parent already enforces this).
gen_kwargs["prompt_text"] = ref_text
language = _normalize_language(msg.get("language"))
if language:
gen_kwargs["language"] = language
result = runtime.generate(**gen_kwargs)
audio = result["audio"]
sample_rate = int(result.get("sample_rate", DOTS_SAMPLE_RATE))
pcm_b64, sr, n_samples = _tensor_to_pcm_b64(audio, sample_rate)
_send(stdout, {
"op": "audio",
"audio_pcm_b64": pcm_b64,
"sample_rate": sr,
"n_samples": n_samples,
})
# ── main loop ─────────────────────────────────────────────────────────────
def main() -> int:
stdin = sys.stdin.buffer
stdout = sys.stdout.buffer
# Ready handshake fires BEFORE any heavy import.
_send(stdout, {
"op": "ready",
"engine": "dots-tts",
"sample_rate": DOTS_SAMPLE_RATE,
})
while True:
try:
msg = _recv(stdin)
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": "recv",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
return 1
if msg is None:
return 0
op = msg.get("op") if isinstance(msg, dict) else None
try:
if op == "ping":
_send(stdout, {"op": "pong", "vram_mb": _measure_vram_mb()})
elif op == "synthesize":
_handle_synthesize(msg, stdout)
elif op == "shutdown":
return 0
else:
_send(stdout, {
"op": "error",
"stage": "dispatch",
"message": f"unknown op: {op!r}",
})
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": op or "unknown",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
if __name__ == "__main__":
sys.exit(main())
+194
View File
@@ -0,0 +1,194 @@
"""MOSS-TTS-v1.5 sidecar package (issue #498).
MOSS-TTS-v1.5 is OpenMOSS's 8B flagship TTS — a Qwen3-8B language backbone
plus a 1.6B audio codec, 31 languages, zero-shot voice cloning, token-level
duration control and inline ``[pause Ns]`` markers. Apache-2.0.
It runs in its own subprocess **and its own venv**, isolated from the
OmniVoice parent process, for the *same* reason IndexTTS does: a hard
``transformers`` version conflict. MOSS-TTS-v1.5's ``torch-runtime`` extra
pins ``transformers==5.0.0`` (verified against the upstream
``pyproject.toml``), while OmniVoice pins ``transformers>=5.3.0``. The two
cannot share one interpreter so MOSS lives behind ``SubprocessBackend``
with a dedicated venv, exactly like ``engines.indextts``.
Three public entry points live in this package:
* ``MossTTSV15Backend`` (this module) the SubprocessBackend subclass
that ``services.tts_backend._LAZY_REGISTRY`` resolves on first access.
Defined HERE (not in ``services.tts_backend``) to break the import
cycle: ``services.subprocess_backend`` imports ``TTSBackend`` from
``services.tts_backend``, so the backend class must live downstream of
that module finishing its import. Same indirection as IndexTTS /
Supertonic-3.
* ``main.py`` the sidecar entrypoint (runs under MOSS's venv with
``transformers==5.0.0``; never imported by the parent).
* ``bootstrap.py`` the venv-probe + lazy-bootstrap helper.
Do NOT import ``main.py`` from the parent process it runs under a
different venv (``transformers==5.0.0``) and importing it in-process would
re-introduce the exact conflict this isolation exists to avoid.
Hardware honesty (cross-platform rule): MOSS-TTS-v1.5's upstream documents
only CUDA and CPU. There is **no documented or tested MPS path** the
custom ``trust_remote_code`` modelling code and the separate audio
tokenizer are unverified on Apple Silicon. We therefore advertise
``gpu_compat = ("cuda", "cpu")`` and the sidecar selects ``cuda`` when
present else ``cpu`` it never silently routes to MPS where it might
crash. On Apple Silicon the engine honestly resolves to CPU (slow but
correct), and the engine is opt-in regardless, so it never becomes a
broken default on any platform.
"""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from services.subprocess_backend import SubprocessBackend
if TYPE_CHECKING:
import torch # noqa: F401
logger = logging.getLogger("omnivoice.moss_tts_v15")
#: 1 second of audio ≈ 12.5 codec tokens (MOSS-TTS-v1.5 model card). Used to
#: translate OmniVoice's ``duration`` (seconds) into the model's ``tokens``
#: duration-control argument.
TOKENS_PER_SECOND: float = 12.5
class MossTTSV15Backend(SubprocessBackend):
"""MOSS-TTS-v1.5 (OpenMOSS) — 8B, 31 langs, zero-shot clone, CUDA/CPU.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv (``transformers==5.0.0``). The first synthesize cold-loads
~16 GB of bf16 weights (CUDA) / fp32 (CPU); subsequent calls reuse the
process and the in-memory model.
Installation (transparent to power users who already cloned MOSS-TTS
OmniVoice prefers their existing ``${DIR}/.venv``)::
git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
# CUDA host:
uv venv && uv pip install -e ".[torch-runtime]"
# non-CUDA host (CPU): install plain torch/transformers instead of +cu128
Set ``OMNIVOICE_MOSS_TTS_V15_DIR`` to the clone root. OmniVoice creates
``backend/engines/moss_tts_v15/.venv`` lazily on first launch if no venv
exists yet (CUDA hosts only the upstream ``torch-runtime`` extra is
``+cu128``); the user's existing ``${DIR}/.venv`` is preferred if
present, so no re-install is needed.
License: Apache-2.0 (code + weights) no acceptance gate needed.
"""
id = "moss-tts-v15"
display_name = (
"MOSS-TTS-v1.5 (8B, 31 langs, zero-shot clone, CUDA/CPU, Apache-2.0)"
)
supports_voice_design = False # requires ref audio for timbre cloning
_DEFAULT_SAMPLE_RATE = 24000
# Honest hardware surface: upstream documents CUDA + CPU only. MPS is
# undocumented / untested, so we do NOT claim it (cross-platform rule).
gpu_compat = ("cuda", "cpu")
# ── availability ───────────────────────────────────────────────────────
@classmethod
def is_available(cls) -> tuple[bool, str]:
# IMPORTANT: do NOT attempt to import MOSS / its transformers==5.0.0
# here. The parent pins transformers>=5.3 — co-importing the two in
# one interpreter is exactly the conflict this subprocess isolation
# exists to avoid. We only verify the venv exists on disk; a real
# health-check (spawn + ping) is gated on the user's "Test engine"
# action in Settings, same as IndexTTS.
from engines.moss_tts_v15.bootstrap import (
MOSS_TTS_V15_SIDECAR_SCRIPT,
is_moss_tts_v15_installed,
)
if not is_moss_tts_v15_installed():
return False, (
"MOSS-TTS-v1.5 venv not found. Set OMNIVOICE_MOSS_TTS_V15_DIR "
"to your MOSS-TTS clone (the directory containing pyproject.toml) "
"and restart OmniVoice. CUDA or CPU only (no MPS). See "
"docs/engines/moss-tts-v15.md for the full install walk-through."
)
if not MOSS_TTS_V15_SIDECAR_SCRIPT.exists():
return False, (
"MOSS-TTS-v1.5 sidecar script missing at "
f"{MOSS_TTS_V15_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok (CUDA when present, else CPU)"
@classmethod
def venv_python(cls):
from engines.moss_tts_v15.bootstrap import resolve_moss_tts_v15_venv
return resolve_moss_tts_v15_venv()
@classmethod
def sidecar_script(cls):
from engines.moss_tts_v15.bootstrap import MOSS_TTS_V15_SIDECAR_SCRIPT
return MOSS_TTS_V15_SIDECAR_SCRIPT
# ── TTSBackend protocol ────────────────────────────────────────────────
@property
def sample_rate(self) -> int:
return self._DEFAULT_SAMPLE_RATE
@property
def supported_languages(self) -> list[str]:
# 31 languages with multilingual handling; expose "multi" on the
# protocol surface (same as OmniVoice / CosyVoice / Supertonic-3) and
# translate the caller's language at synthesize time.
return ["multi"]
# ── generate (parent-side arbitration) ─────────────────────────────────
def generate(self, text: str, **kw) -> "torch.Tensor":
"""Synthesize one utterance through the MOSS-TTS-v1.5 sidecar.
kwargs honored:
* ``ref_audio`` path to a reference clip. When present, MOSS
runs zero-shot voice cloning (``reference=``).
Optional: without it the model uses its own
default voice.
* ``ref_text`` accepted but unused in clone mode (MOSS's
zero-shot path needs only the audio); kept in
the signature so the common call-site doesn't
need engine-specific knowledge.
* ``language`` ISO code or name; mapped to a MOSS language name
in the sidecar, omitted (auto-detect) if unknown.
* ``duration`` target seconds ``tokens`` (1 s 12.5 tokens).
* ``max_new_tokens`` generation cap (default 4096).
Returns a tensor of shape (1, n_samples) at :attr:`sample_rate`.
"""
forwarded: dict = {}
ref_audio = kw.get("ref_audio")
if ref_audio:
forwarded["ref_audio"] = ref_audio
ref_text = kw.get("ref_text")
if ref_text:
forwarded["ref_text"] = ref_text
language = kw.get("language")
if language:
forwarded["language"] = str(language)
duration = kw.get("duration")
if duration is not None:
target_tokens = int(float(duration) * TOKENS_PER_SECOND)
if target_tokens > 0:
forwarded["tokens"] = target_tokens
max_new_tokens = kw.get("max_new_tokens")
if max_new_tokens is not None:
forwarded["max_new_tokens"] = int(max_new_tokens)
return super().generate(text, **forwarded)
__all__ = ["MossTTSV15Backend", "TOKENS_PER_SECOND"]
+272
View File
@@ -0,0 +1,272 @@
"""MOSS-TTS-v1.5 venv probe + lazy bootstrap (issue #498).
The parent process needs to know *which Python interpreter* to spawn the
MOSS-TTS-v1.5 sidecar under. This module owns that resolution. It mirrors
``engines.indextts.bootstrap`` because MOSS has the same shape of problem:
a hard ``transformers`` pin (``==5.0.0``) that conflicts with the parent's
``transformers>=5.3`` so MOSS runs in its own venv.
Probe order (priority existing power-user installs win, zero migration):
1. ``${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv/`` the user's clone-level
venv. Highest priority: a user who already cloned MOSS-TTS and ran
``uv pip install -e ".[torch-runtime]"`` (per upstream docs) gets
reused verbatim, no re-download of the ~16 GB model.
2. ``backend/engines/moss_tts_v15/.venv/`` this package's own venv,
created by step 3 if needed.
3. Bootstrap: ``uv venv`` then ``uv pip install -e
"${DIR}[torch-runtime]"``. Requires ``OMNIVOICE_MOSS_TTS_V15_DIR``.
The upstream ``torch-runtime`` extra is CUDA (``+cu128``), so the
auto-bootstrap targets CUDA hosts; non-CUDA (CPU/Mac) users set up
their own venv per docs/engines/moss-tts-v15.md (Probe 1).
Caching: resolution is memoised after the first successful call. Tests
reset via :func:`invalidate`.
Security: bootstrap never touches HF_TOKEN; the sidecar's stderr is drained
by SubprocessBackend through the parent root logger where Phase 1's
``HFTokenRedactor`` strips token bytes. ``uv pip install -e`` installs from
a user-controlled clone the user already trusts (same posture as IndexTTS).
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger("omnivoice.moss_tts_v15.bootstrap")
#: Absolute path to the sidecar entrypoint. ``MossTTSV15Backend.sidecar_script``
#: returns this; SubprocessBackend spawns it with the resolved venv python.
MOSS_TTS_V15_SIDECAR_SCRIPT: Path = Path(__file__).parent / "main.py"
#: This package's owned venv (Probe 2). The MOSS-TTS clone, when bootstrapped,
#: is installed into this venv via ``uv pip install -e``.
_ENGINES_VENV_DIR: Path = Path(__file__).parent / ".venv"
#: Env var pointing at the user's MOSS-TTS clone root.
_CLONE_DIR_ENV: str = "OMNIVOICE_MOSS_TTS_V15_DIR"
#: Per-process resolution cache. Cleared by :func:`invalidate` for tests.
_resolved_python: Optional[Path] = None
# Timeouts — bounded so a wedged venv never hangs the parent. The bootstrap
# install can take many minutes on a cold cache (MOSS pulls a CUDA torch
# build + transformers + an audio codec stack).
_IMPORT_PROBE_TIMEOUT_S = 15
_UV_VENV_TIMEOUT_S = 120
_UV_PIP_INSTALL_TIMEOUT_S = 1800
# ── public API ────────────────────────────────────────────────────────────
def invalidate() -> None:
"""Clear the resolved-python cache. Tests call this between scenarios."""
global _resolved_python
_resolved_python = None
def is_moss_tts_v15_installed() -> bool:
"""Cheap file-existence check for a usable MOSS-TTS-v1.5 venv.
Returns True if either Probe 1 or Probe 2 has a Python executable on
disk. Does NOT spawn the venv Python that's saved for
:func:`resolve_moss_tts_v15_venv`, which is only invoked on the first
generate() / health_check(). This fires on every Settings render via
``MossTTSV15Backend.is_available()``, so it stays cheap.
"""
for cand in _probe_paths():
if cand.is_file():
return True
return False
def resolve_moss_tts_v15_venv() -> Path:
"""Resolve the path to the Python interpreter that runs the sidecar.
Probe order described in the module docstring. Memoised. Raises
:exc:`RuntimeError` if no working venv can be located AND the bootstrap
path is unavailable.
"""
global _resolved_python
if _resolved_python is not None:
return _resolved_python
clone_dir = os.environ.get(_CLONE_DIR_ENV)
# Probe 1 — user's clone-level venv (highest priority for back-compat).
if clone_dir:
cand = _venv_python_path(Path(clone_dir) / ".venv")
if cand.is_file() and _venv_can_import_moss(cand):
logger.info(
"MOSS-TTS-v1.5 venv resolved from %s: %s", _CLONE_DIR_ENV, cand,
)
_resolved_python = cand
return cand
# Probe 2 — this package's own venv.
cand = _venv_python_path(_ENGINES_VENV_DIR)
if cand.is_file() and _venv_can_import_moss(cand):
logger.info("MOSS-TTS-v1.5 venv resolved from engines path: %s", cand)
_resolved_python = cand
return cand
# Probe 3 — bootstrap.
if not clone_dir:
raise RuntimeError(
"MOSS-TTS-v1.5 is not installed. Set the "
f"{_CLONE_DIR_ENV} environment variable to your MOSS-TTS clone "
"(the directory that contains pyproject.toml), then restart "
"OmniVoice. See docs/engines/moss-tts-v15.md for the full "
"install walk-through."
)
cand = _bootstrap_engines_venv(Path(clone_dir))
_resolved_python = cand
return cand
# ── internals ─────────────────────────────────────────────────────────────
def _venv_python_path(venv_dir: Path) -> Path:
"""Return the python executable path inside a venv directory.
Handles the Unix (``bin/python``) vs Windows (``Scripts/python.exe``)
layout. No filesystem access caller checks .is_file().
"""
if sys.platform == "win32":
return venv_dir / "Scripts" / "python.exe"
return venv_dir / "bin" / "python"
def _probe_paths() -> list[Path]:
"""Ordered list of candidate venv-python paths (no .is_file() check)."""
out: list[Path] = []
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
out.append(_venv_python_path(Path(clone_dir) / ".venv"))
out.append(_venv_python_path(_ENGINES_VENV_DIR))
return out
def _venv_can_import_moss(python_path: Path) -> bool:
"""Spawn the candidate python and verify the MOSS stack imports.
MOSS-TTS-v1.5 loads via ``transformers`` + ``trust_remote_code`` (no
fixed top-level package to import), so the readiness signal is that the
venv has a working ``transformers`` + ``torch`` which only the
``[torch-runtime]`` install provides. Bounded by
``_IMPORT_PROBE_TIMEOUT_S`` so a wedged venv never hangs the parent.
Returns False on any failure (non-zero exit, timeout, OSError).
"""
try:
proc = subprocess.run(
[str(python_path), "-c", "import transformers, torch"],
capture_output=True,
timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
logger.debug("moss-tts-v15 import probe failed for %s: %s", python_path, exc)
return False
if proc.returncode != 0:
logger.debug(
"moss-tts-v15 import probe non-zero for %s: %s",
python_path,
proc.stderr.decode("utf-8", errors="replace")[:200],
)
return False
return True
def _locate_uv() -> Optional[str]:
"""Find the uv binary — bundled first (Tauri-set env var), else PATH."""
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
if bundled and Path(bundled).is_file():
return bundled
sys_uv = shutil.which("uv")
if sys_uv:
return sys_uv
return None
def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"""Create engines/moss_tts_v15/.venv and install the user's clone into it.
Runs ``uv venv <engines_venv>`` then ``uv pip install --python
<engines_venv>/bin/python -e "<clone>[torch-runtime]"``. Verifies the
result by re-probing the import a successful uv invocation that still
can't import the stack indicates a deeper environment problem (e.g. the
``+cu128`` torch-runtime extra can't resolve on a non-CUDA host) and we
raise with whatever stderr we captured plus a docs pointer.
"""
uv = _locate_uv()
if not uv:
raise RuntimeError(
"uv is required to bootstrap the MOSS-TTS-v1.5 venv but was not "
"found on PATH (and OMNIVOICE_BUNDLED_UV was not set). Install uv "
"from https://docs.astral.sh/uv/ and re-launch OmniVoice, or set "
"OMNIVOICE_BUNDLED_UV to the absolute path of a uv binary."
)
logger.info(
"Bootstrapping MOSS-TTS-v1.5 venv at %s from %s (this can take "
"several minutes on first launch)", _ENGINES_VENV_DIR, clone_dir,
)
try:
subprocess.run(
[uv, "venv", str(_ENGINES_VENV_DIR)],
check=True,
timeout=_UV_VENV_TIMEOUT_S,
capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
f"uv venv failed for MOSS-TTS-v1.5 bootstrap at {_ENGINES_VENV_DIR}: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
python_path = _venv_python_path(_ENGINES_VENV_DIR)
try:
subprocess.run(
[
uv, "pip", "install",
"--python", str(python_path),
"-e", f"{clone_dir}[torch-runtime]",
],
check=True,
timeout=_UV_PIP_INSTALL_TIMEOUT_S,
capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install -e failed during MOSS-TTS-v1.5 bootstrap "
f"({clone_dir}). On a non-CUDA host the upstream '[torch-runtime]' "
"extra (cu128) cannot resolve — set up the venv manually per "
"docs/engines/moss-tts-v15.md. Error: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
if not _venv_can_import_moss(python_path):
raise RuntimeError(
"MOSS-TTS-v1.5 bootstrap completed but the transformers/torch "
f"import still fails from {python_path}. Verify that {clone_dir} "
"is a valid MOSS-TTS clone. See docs/engines/moss-tts-v15.md."
)
logger.info("MOSS-TTS-v1.5 venv bootstrap successful: %s", python_path)
return python_path
__all__ = [
"MOSS_TTS_V15_SIDECAR_SCRIPT",
"invalidate",
"is_moss_tts_v15_installed",
"resolve_moss_tts_v15_venv",
]
+303
View File
@@ -0,0 +1,303 @@
"""MOSS-TTS-v1.5 sidecar entry point (issue #498).
Runs inside ``engines/moss_tts_v15/.venv`` (or the user's existing
``${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv``) with ``transformers==5.0.0``,
isolated from the OmniVoice parent process which pins ``transformers>=5.3``.
Same isolation rationale as the IndexTTS sidecar.
Stdlib-only at import time. The model + transformers + torch are imported
lazily on the first synthesize op so the sidecar emits its ``ready`` frame
inside the parent's 30 s spawn handshake even on a cold filesystem (an 8B
model takes well over 30 s to cold-load).
Wire protocol length-prefixed JSON over stdin/stdout, byte-identical to
``backend/services/subprocess_backend.py``::
[ 4-byte big-endian uint32 length ][ N bytes UTF-8 JSON ]
Op flow:
1. Sidecar -> parent: {"op": "ready", "engine": "moss-tts-v15",
"sample_rate": 24000}
2. parent -> sidecar: {"op": "ping"} -> {"op": "pong", "vram_mb": N}
3. parent -> sidecar: {"op": "synthesize", "text": "...",
"ref_audio": "/path/spk.wav", "language": "fr",
"tokens": 325, "max_new_tokens": 4096}
-> {"op": "progress", ...} (cold load only) then
-> {"op": "audio", "audio_pcm_b64": "...", "sample_rate": 24000,
"n_samples": N}
4. parent -> sidecar: {"op": "shutdown"} -> exit 0
Restrictions: NO imports from OmniVoice parent code (different venv). NO
logging of ``os.environ`` contents. Single-frame DoS cap matches the
parent's ``MAX_FRAME_BYTES``.
"""
from __future__ import annotations
import base64
import json
import os
import struct
import sys
import traceback
# Mirrors backend/services/subprocess_backend.py::MAX_FRAME_BYTES.
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: Native sample rate MOSS-TTS-v1.5 emits. Advertised in the ready frame so
#: the parent doesn't have to import MOSS just to learn the rate. Confirmed
#: via ``processor.model_config.sampling_rate`` (the real value is read from
#: the loaded model at synthesize time; this is the handshake default).
MOSS_SAMPLE_RATE = 24000
#: HF repo id for the weights, overridable for air-gapped / mirror installs.
_DEFAULT_REPO = "OpenMOSS-Team/MOSS-TTS-v1.5"
#: ISO-639-1 → MOSS language name. MOSS's ``build_user_message`` takes a
#: language *name* ("French"), not a code. Unknown codes are omitted so the
#: model auto-detects. Covers the high-traffic subset of MOSS's 31 langs.
_ISO_TO_NAME = {
"en": "English", "zh": "Chinese", "ja": "Japanese", "ko": "Korean",
"fr": "French", "de": "German", "es": "Spanish", "it": "Italian",
"pt": "Portuguese", "ru": "Russian", "ar": "Arabic", "hi": "Hindi",
"nl": "Dutch", "pl": "Polish", "tr": "Turkish", "vi": "Vietnamese",
"th": "Thai", "id": "Indonesian", "cs": "Czech", "el": "Greek",
"he": "Hebrew", "fa": "Persian", "uk": "Ukrainian", "sv": "Swedish",
}
# ── wire protocol ─────────────────────────────────────────────────────────
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
def _recv(stream):
header = stream.read(4)
if len(header) < 4:
return None # EOF
(n,) = struct.unpack("!I", header)
if n > MAX_FRAME_BYTES:
raise IOError(f"frame too large: {n}")
body = bytearray()
while len(body) < n:
chunk = stream.read(n - len(body))
if not chunk:
raise IOError("short read")
body.extend(chunk)
return json.loads(bytes(body).decode("utf-8"))
def _measure_vram_mb() -> float:
"""This sidecar's own GPU memory in MB (MM2-08). The parent can't see a
child's VRAM, so we self-report it in the pong. 0 on CPU. Never raises."""
try:
import torch
if torch.cuda.is_available():
return round(torch.cuda.memory_allocated() / (1024 ** 2), 1)
except Exception:
pass
return 0.0
# ── model loading (lazy, on first synthesize) ─────────────────────────────
# Module-level singleton — populated on the first synthesize op and reused.
# Holds (processor, model, device, sample_rate).
_state = None
def _load_model(stdout):
"""Cold-construct the MOSS-TTS-v1.5 processor + model.
Device selection is CUDA-or-CPU only MOSS's upstream documents no MPS
path and the custom ``trust_remote_code`` modelling code is untested on
Apple Silicon, so we never route to MPS where it might crash. dtype is
bf16 on CUDA, fp32 on CPU (bf16 CPU ops are spotty). Emits progress
frames so the parent can surface the multi-GB cold-load latency.
"""
global _state
if _state is not None:
return _state
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
import torch
from transformers import AutoModel, AutoProcessor
repo = os.environ.get("OMNIVOICE_MOSS_TTS_V15_MODEL", _DEFAULT_REPO)
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
# "sdpa" works on CUDA + CPU and needs no extra dep. flash_attention_2
# (Ampere+ CUDA, optional flash-attn) is opt-in via env.
attn = os.environ.get("OMNIVOICE_MOSS_TTS_V15_ATTN", "sdpa")
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
# The audio tokenizer is a separate sub-module that must be moved to the
# device independently (easy to miss — see upstream README).
processor.audio_tokenizer = processor.audio_tokenizer.to(device)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 50})
model = AutoModel.from_pretrained(
repo,
trust_remote_code=True,
attn_implementation=attn,
torch_dtype=dtype,
).to(device)
model.eval()
sample_rate = int(getattr(processor.model_config, "sampling_rate", MOSS_SAMPLE_RATE))
_state = (processor, model, device, sample_rate)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _state
def _tensor_to_pcm_b64(audio, sample_rate: int) -> tuple[str, int, int]:
"""Convert a torch waveform tensor to base64 int16 PCM.
MOSS returns a float tensor in [-1, 1] (1-D or (1, N)); we squeeze to
mono, clip, scale to int16, and base64 so the wire frame stays JSON-safe.
"""
import numpy as np
arr = audio.detach().to("cpu").float().numpy()
arr = np.asarray(arr, dtype=np.float32).squeeze()
if arr.ndim > 1:
arr = arr.mean(axis=0) # defensive downmix to mono
arr = np.clip(arr, -1.0, 1.0)
pcm = (arr * 32767.0).astype(np.int16).tobytes()
return base64.b64encode(pcm).decode("ascii"), int(sample_rate), int(arr.shape[0])
def _resolve_language(raw):
"""Map OmniVoice's language value to a MOSS language name, or None.
Accepts an ISO-639-1 code or a full name. Unknown / empty / "auto"
values return None so MOSS auto-detects."""
if not raw or not isinstance(raw, str):
return None
s = raw.strip()
if not s or s.lower() == "auto":
return None
if s.lower() in _ISO_TO_NAME:
return _ISO_TO_NAME[s.lower()]
# Already a language name (or an unknown code) — pass it through; MOSS
# ignores a language it doesn't recognise.
return s
def _handle_synthesize(msg: dict, stdout) -> None:
"""Dispatch one synthesize request. Emits the audio frame or raises."""
import torch
text = msg.get("text")
if not text or not isinstance(text, str):
raise ValueError("synthesize: missing or non-string 'text'")
processor, model, device, sample_rate = _load_model(stdout)
user_kwargs: dict = {"text": text}
ref_audio = msg.get("ref_audio")
if ref_audio:
# Zero-shot voice cloning: the reference audio alone is enough in
# MOSS's clone mode (ref_text is not consumed here). The processor's
# audio tokenizer encodes the reference into the prompt.
user_kwargs["reference"] = [ref_audio]
language = _resolve_language(msg.get("language"))
if language:
user_kwargs["language"] = language
tokens = msg.get("tokens")
if tokens is not None:
user_kwargs["tokens"] = int(tokens)
max_new_tokens = int(msg.get("max_new_tokens", 4096))
conversations = [[processor.build_user_message(**user_kwargs)]]
with torch.no_grad():
batch = processor(conversations, mode="generation")
outputs = model.generate(
input_ids=batch["input_ids"].to(device),
attention_mask=batch["attention_mask"].to(device),
max_new_tokens=max_new_tokens,
)
decoded = processor.decode(outputs)
audio = decoded[0].audio_codes_list[0]
pcm_b64, sr, n_samples = _tensor_to_pcm_b64(audio, sample_rate)
_send(stdout, {
"op": "audio",
"audio_pcm_b64": pcm_b64,
"sample_rate": sr,
"n_samples": n_samples,
})
# ── main loop ─────────────────────────────────────────────────────────────
def main() -> int:
stdin = sys.stdin.buffer
stdout = sys.stdout.buffer
# Ready handshake fires BEFORE any heavy import — nothing above this line
# touches transformers/torch, so we make the 30 s spawn window even cold.
_send(stdout, {
"op": "ready",
"engine": "moss-tts-v15",
"sample_rate": MOSS_SAMPLE_RATE,
})
while True:
try:
msg = _recv(stdin)
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": "recv",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
return 1
if msg is None:
return 0
op = msg.get("op") if isinstance(msg, dict) else None
try:
if op == "ping":
_send(stdout, {"op": "pong", "vram_mb": _measure_vram_mb()})
elif op == "synthesize":
_handle_synthesize(msg, stdout)
elif op == "shutdown":
return 0
else:
_send(stdout, {
"op": "error",
"stage": "dispatch",
"message": f"unknown op: {op!r}",
})
except Exception as exc:
# Per-op failure is recoverable — emit the error frame and stay
# alive so the parent can retry without paying the respawn +
# multi-GB model-load cost again.
_send(stdout, {
"op": "error",
"stage": op or "unknown",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
if __name__ == "__main__":
sys.exit(main())
+14
View File
@@ -740,6 +740,20 @@ app.add_middleware(NetworkAccessMiddleware)
# keyed non-loopback client must reach them.
app.add_middleware(BearerKeyMiddleware)
# Register canonical audio MIME types before any StaticFiles mount.
# Python's `mimetypes.guess_type()` returns `audio/x-wav` for `.wav` and
# `audio/x-flac` for `.flac` on most platforms — these are vendor-experimental
# (x- prefix, never IANA-registered). macOS Chrome/Safari MIME-sniff leniently
# via CoreAudio so playback works there, but Linux Chrome/Firefox (FFmpeg) and
# Android Chrome (ExoPlayer) strictly honor the declared type and treat the
# x- variants as download-only — manifesting as the play button silently
# doing nothing in the browser app while working in the Tauri desktop shell.
# `audio/wav` / `audio/flac` are the IANA-canonical types.
# Ref: https://www.iana.org/assignments/media-types/media-types.xhtml#audio
import mimetypes as _mimetypes
_mimetypes.add_type("audio/wav", ".wav")
_mimetypes.add_type("audio/flac", ".flac")
app.mount("/audio", StaticFiles(directory=OUTPUTS_DIR), name="audio")
app.mount("/voice_audio", StaticFiles(directory=VOICES_DIR), name="voice_audio")
@@ -0,0 +1,36 @@
"""Heal voice_profiles.instruct poisoned with the "[object Object]" sentinel.
Revision ID: 0006_strip_object_object_instruct
Revises: 0005_unified_profiles
Create Date: 2026-06-20 00:00:00.000000
A pre-fix Voice Studio build ("Save design as profile") passed the
``buildDesignInstruct()`` *object* straight to FormData, which string-coerced it
to the literal ``"[object Object]"`` and persisted that into
``voice_profiles.instruct`` (#550 #545 #542 #537 #530 #525). On first
preview/use that value fails the engine instruct validator with a 400. The
frontend + backend fixes stop any NEW poisoned rows; this migration heals the
ones already saved on the buggy build (the local-first backward-compat rule
existing project data must keep working without manual migration).
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import inspect
revision: str = "0006_strip_object_object_instruct"
down_revision: Union[str, None] = "0005_unified_profiles"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
bind = op.get_bind()
if "voice_profiles" in inspect(bind).get_table_names():
# Idempotent: only touches rows whose instruct is literally the sentinel.
op.execute("UPDATE voice_profiles SET instruct='' WHERE instruct='[object Object]'")
def downgrade() -> None:
# Irreversible heal — the original garbage sentinel is not worth restoring.
pass
+126 -10
View File
@@ -31,6 +31,23 @@ from abc import ABC, abstractmethod
logger = logging.getLogger("omnivoice.asr")
def _compute_type_candidates(device: str) -> list[str]:
"""Per-device compute_type fallback chain. int8 is supported by every
CTranslate2 CUDA+CPU build; float16/int8_float16 only on GPUs with efficient
fp16 so degrade rather than crash (#551). Honors an ASR_COMPUTE_TYPE env
override (power users on exotic hardware can pin int8/float32)."""
import os
override = os.environ.get("ASR_COMPUTE_TYPE")
if override:
return [override]
return ["float16", "int8_float16", "int8"] if device == "cuda" else ["int8", "float32"]
def _is_compute_type_error(msg: str) -> bool:
low = msg.lower()
return "compute type" in low or "efficient float16" in low
def _decode_audio_16k_mono(audio_path: str):
"""Decode `audio_path` to a 16 kHz mono float32 waveform using OmniVoice's
*validated* ffmpeg, instead of whisperx.load_audio's bare ``"ffmpeg"`` PATH
@@ -189,7 +206,40 @@ class WhisperXBackend(ASRBackend):
# vad_method="silero" is the default; keep it so short gaps
# get cleaned up before transcription.
)
except RuntimeError as e:
except (ValueError, RuntimeError) as e:
# #551: GPUs without efficient fp16 (older Maxwell/Pascal, GTX 16xx)
# or a CTranslate2/cuDNN binary mismatch raise a *ValueError*
# ("Requested float16 compute type, but the target device or backend
# do not support efficient float16 computation") at load — not an
# OOM, not a RuntimeError. Retry on the SAME device with the next
# compute_type candidate (cuda: int8_float16 → int8) before touching
# the OOM→CPU path, so we degrade rather than crash every chunk.
if _is_compute_type_error(str(e)):
candidates = _compute_type_candidates(self._device)
try:
nxt = candidates[candidates.index(self._compute_type) + 1:]
except ValueError:
nxt = [c for c in candidates if c != self._compute_type]
for ct in nxt:
logger.warning(
"whisperx %s unsupported on %s — retrying with %s. Detail: %s",
self._compute_type, self._device, ct, e,
)
self._compute_type = ct
try:
self._asr = whisperx.load_model(
self._model_name,
device=self._device,
compute_type=self._compute_type,
)
return
except (ValueError, RuntimeError) as e2:
if _is_compute_type_error(str(e2)):
e = e2
continue
raise
# Exhausted compute-type candidates on this device — re-raise.
raise
# CUDA OOM: a resident TTS model + the GPU worker pool can starve
# VRAM on small (e.g. 8 GB laptop) GPUs, so loading large-v3 on
# CUDA dies here — which previously surfaced as a bare 500 from
@@ -462,6 +512,10 @@ class FasterWhisperBackend(ASRBackend):
"ASR_MODEL_FASTER", "Systran/faster-whisper-large-v3"
)
self._model = None # lazy — first transcribe() loads weights
# Set by _ensure_model() to the device/compute_type that actually loaded
# (after the #551 compute_type / #255 OOM→CPU fallback chain).
self._device: str | None = None
self._compute_type: str | None = None
@classmethod
def is_available(cls) -> tuple[bool, str]:
@@ -490,9 +544,58 @@ class FasterWhisperBackend(ASRBackend):
"faster-whisper loading %s on %s (%s)",
self._model_name, device, compute_type,
)
self._model = WhisperModel(
self._model_name, device=device, compute_type=compute_type
)
# Try the per-device compute_type chain (cuda: float16 → int8_float16 →
# int8; cpu: int8 → float32). A GPU without efficient fp16 (older
# Maxwell/Pascal, GTX 16xx, or a CTranslate2/cuDNN mismatch) raises a
# *ValueError* at construction (#551) — degrade to the next candidate
# instead of failing every chunk. A genuine CUDA OOM falls back to CPU
# (slower, same model/accuracy), preserving the existing #255 behaviour.
candidates = _compute_type_candidates(device)
if compute_type in candidates:
candidates = candidates[candidates.index(compute_type):]
last_err: Exception | None = None
while True:
for ct in candidates:
try:
self._model = WhisperModel(
self._model_name, device=device, compute_type=ct
)
self._device, self._compute_type = device, ct
return
except (ValueError, RuntimeError) as e:
last_err = e
if _is_compute_type_error(str(e)):
logger.warning(
"faster-whisper %s unsupported on %s — trying next "
"compute_type. Detail: %s", ct, device, e,
)
continue
if device == "cuda" and "out of memory" in str(e).lower():
# Stop scanning GPU candidates; fall back to CPU below.
break
raise
# Exhausted candidates for this device. If we were on CUDA and the
# last failure was an OOM, retry on CPU with its candidates (#255).
if device == "cuda" and last_err is not None and (
"out of memory" in str(last_err).lower()
):
logger.warning(
"faster-whisper CUDA OOM loading %s — retrying on CPU "
"(slower). Free VRAM (Flush the TTS model) for GPU-speed "
"ASR. Detail: %s", self._model_name, last_err,
)
try:
import torch
torch.cuda.empty_cache()
except Exception: # noqa: BLE001 — cache clear is best-effort
pass
device = "cpu"
candidates = _compute_type_candidates(device)
compute_type = candidates[0]
continue
# All candidates exhausted (and no OOM→CPU retry available) — surface
# the last error.
raise last_err
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
self._ensure_model()
@@ -681,12 +784,25 @@ class PyTorchWhisperBackend(ASRBackend):
"PyTorchWhisperBackend: loading standalone ASR pipeline %s on %s",
model_name, device,
)
self._pipe = hf_pipeline(
"automatic-speech-recognition",
model=model_name,
dtype=asr_dtype,
device_map=device,
)
try:
self._pipe = hf_pipeline(
"automatic-speech-recognition",
model=model_name,
dtype=asr_dtype,
device_map=device,
)
except Exception as e:
# #549: an incomplete transformers install fails to build the ASR
# pipeline (e.g. "Could not import module 'AutoFeatureExtractor'").
# The raw error is opaque; re-raise with an actionable next step so
# the toast tells the user how to recover instead of "no segments".
raise RuntimeError(
"transformers ASR pipeline failed to import (AutoFeatureExtractor) "
"— your transformers install is incomplete; reinstall with "
"`uv pip install --reinstall transformers`, or use faster-whisper "
"(OmniVoice's default ASR) which avoids the transformers pipeline. "
f"Underlying: {e}"
) from e
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
import soundfile as sf
+5 -1
View File
@@ -65,10 +65,14 @@ class AudiobookPlan:
def char_count(self) -> int:
return sum(c.char_count for c in self.chapters)
@property
def chapter_count(self) -> int:
return len(self.chapters)
def to_dict(self) -> dict:
return {
"chapters": [c.to_dict() for c in self.chapters],
"chapter_count": len(self.chapters),
"chapter_count": self.chapter_count,
"char_count": self.char_count,
}
+10
View File
@@ -177,6 +177,16 @@ async def run_pip(args: list[str], timeout: float = 600.0) -> tuple[int, str]:
"""
base = _installer_cmd()
using_uv = base[:1] == ["uv"]
# Pin `uv pip` to the interpreter the backend ACTUALLY runs under. The desktop
# spawns `<venv>/bin/python -m uvicorn` WITHOUT exporting VIRTUAL_ENV, so bare
# `uv pip install` finds no venv and 500s with "No virtual environment found"
# (#529/#527) — and the `--system` branch below never fires, because the
# running interpreter genuinely IS in a venv (uv just can't auto-discover it).
# `--python sys.executable` targets the same interpreter _probe()/is_installed()
# import from, and takes precedence when both flags are present, so the Docker
# `--system` path is unaffected.
if using_uv and args and args[0] in ("install", "uninstall") and "--python" not in args:
args = [args[0], "--python", sys.executable, *args[1:]]
if using_uv and not _in_virtualenv() and args and args[0] in ("install", "uninstall") and "--system" not in args:
args = [args[0], "--system", *args[1:]]
cmd = base + args
+9
View File
@@ -1151,6 +1151,13 @@ _LAZY_REGISTRY: dict[str, tuple[str, str]] = {
# this module for TTSBackend). The class is resolved on first
# attribute access via the LazyRegistry below.
"supertonic3": ("engines.supertonic3", "Supertonic3Backend"),
# Issue #498: MOSS-TTS-v1.5 (8B) and dots.tts (2B) — both opt-in,
# subprocess-isolated with their own venv because each pins a
# transformers version that conflicts with the parent's >=5.3
# (MOSS == 5.0.0, dots.tts == 4.57.0). Same dedicated-venv pattern as
# IndexTTS2. Lazy for the same import-cycle reason as the entries above.
"moss-tts-v15": ("engines.moss_tts_v15", "MossTTSV15Backend"),
"dots-tts": ("engines.dots_tts", "DotsTTSBackend"),
}
@@ -1244,6 +1251,8 @@ _INSTALL_HINTS: dict[str, str] = {
"sherpa-onnx": "pip install sherpa-onnx (universal ONNX runtime, WASM-ready)",
"omnivoice-gguf":"Bundled — runs the C++ omnivoice-tts binary in bin/. Quants download lazily from Serveurperso/OmniVoice-GGUF on first generate.",
"supertonic3": "uv sync --extra supertonic (CPU-only ONNX, 31 langs, ~400 MB model on first use; OpenRAIL-M model license)",
"moss-tts-v15": "git clone OpenMOSS/MOSS-TTS + set OMNIVOICE_MOSS_TTS_V15_DIR (own venv, transformers==5.0; 8B, ~16 GB weights; CUDA/CPU, no MPS; Apache-2.0)",
"dots-tts": "git clone rednote-hilab/dots.tts + set OMNIVOICE_DOTS_TTS_DIR (own venv, transformers==4.57; 2B, ~9 GB weights; CUDA/CPU, Linux/macOS only — no Windows; Apache-2.0)",
}
+77 -2
View File
@@ -25,7 +25,17 @@ sys.modules["core.config"] = _config
whisperx = pytest.importorskip("whisperx")
from services.asr_backend import WhisperXBackend # noqa: E402
from services.asr_backend import ( # noqa: E402
WhisperXBackend,
_is_compute_type_error,
)
# The exact ValueError CTranslate2 raises at model construction on a GPU
# without efficient fp16 (older Maxwell/Pascal, GTX 16xx) or a cuDNN mismatch.
_FP16_ERR = (
"Requested float16 compute type, but the target device or backend do not "
"support efficient float16 computation"
)
def test_cuda_oom_falls_back_to_cpu(monkeypatch):
@@ -52,8 +62,13 @@ def test_cuda_oom_falls_back_to_cpu(monkeypatch):
def test_non_oom_runtime_error_still_raises(monkeypatch):
msg = "some other failure"
# A generic non-OOM, non-compute-type RuntimeError must still propagate —
# the new compute_type fallback must NOT swallow it.
assert _is_compute_type_error(msg) is False
def fake_load_model(name, device, compute_type, **kw):
raise RuntimeError("some other failure") # not an OOM → must propagate
raise RuntimeError(msg) # not an OOM, not compute-type → must propagate
monkeypatch.setattr(whisperx, "load_model", fake_load_model)
@@ -62,3 +77,63 @@ def test_non_oom_runtime_error_still_raises(monkeypatch):
be._allow_vad_pickle_globals = lambda: None
with pytest.raises(RuntimeError, match="some other failure"):
be._ensure_asr()
def test_float16_unsupported_falls_back_to_int8(monkeypatch):
"""#551: a GPU without efficient fp16 raises a ValueError at load for both
float16 AND int8_float16; the backend must degrade to int8 on the SAME
device (cuda) without raising not fall to CPU and not crash."""
calls = []
def fake_load_model(name, device, compute_type, **kw):
calls.append((device, compute_type))
if device == "cuda" and compute_type in ("float16", "int8_float16"):
raise ValueError(_FP16_ERR)
return object() # cuda int8 succeeds
monkeypatch.setattr(whisperx, "load_model", fake_load_model)
be = WhisperXBackend()
be._device, be._compute_type = "cuda", "float16"
be._allow_vad_pickle_globals = lambda: None
be._ensure_asr()
assert be._asr is not None # recovered, no raise
assert be._device == "cuda" and be._compute_type == "int8" # same device, int8
assert calls == [("cuda", "float16"), ("cuda", "int8_float16"), ("cuda", "int8")]
def test_faster_whisper_float16_unsupported_falls_back_to_int8(monkeypatch):
"""Mirror for FasterWhisperBackend: float16 + int8_float16 raise the fp16
ValueError, int8 succeeds loads on (cuda, int8) without raising."""
import services.asr_backend as asr_backend
from services.asr_backend import FasterWhisperBackend
calls = []
class FakeWhisperModel:
def __init__(self, name, device, compute_type, **kw):
calls.append((device, compute_type))
if device == "cuda" and compute_type in ("float16", "int8_float16"):
raise ValueError(_FP16_ERR)
# cuda int8 succeeds
fake_fw = types.ModuleType("faster_whisper")
fake_fw.WhisperModel = FakeWhisperModel
monkeypatch.setitem(sys.modules, "faster_whisper", fake_fw)
# Force the CUDA starting point regardless of the CI host's hardware by
# making torch.cuda.is_available() return True inside _ensure_model.
fake_torch = types.ModuleType("torch")
fake_torch.cuda = types.SimpleNamespace(
is_available=lambda: True, empty_cache=lambda: None
)
monkeypatch.setitem(sys.modules, "torch", fake_torch)
be = FasterWhisperBackend()
be._ensure_model()
assert be._model is not None # recovered, no raise
assert be._device == "cuda" and be._compute_type == "int8"
assert calls == [("cuda", "float16"), ("cuda", "int8_float16"), ("cuda", "int8")]
+12 -4
View File
@@ -1,10 +1,11 @@
# Engine venvs & disk usage
Most engines run in-process in OmniVoice's main environment. A few
(**IndexTTS2**, and any engine whose dependencies conflict with the parent's
`torch`/`transformers` pins) run in a **dedicated sidecar venv** so their pins
can't break the rest of the app. Those sidecars are where disk adds up — this
page explains why, and how the on-disk cost is kept down.
(**IndexTTS2**, **MOSS-TTS-v1.5**, **dots.tts**, and any engine whose
dependencies conflict with the parent's `torch`/`transformers` pins) run in a
**dedicated sidecar venv** so their pins can't break the rest of the app. Those
sidecars are where disk adds up — this page explains why, and how the on-disk
cost is kept down.
## Why a sidecar needs its own venv
@@ -48,6 +49,13 @@ the parent whenever the engine allows it.** When it doesn't (IndexTTS2's
`transformers<5` forces an older torch line), the second copy is the
unavoidable price of isolation — not a bug.
The opt-in #498 engines illustrate both sides: **dots.tts** pins
`torch==2.8.0` — the **same** build the parent constrains to — so it shares
almost all of torch with the main venv and only its `transformers==4.57` +
model deps are new. **MOSS-TTS-v1.5** pins `torch==2.9.1+cu128`, a **different**
build, so it pays a full extra multi-GB torch copy on CUDA hosts (the price of
running an 8B model whose stack pins `transformers==5.0`).
> On Linux, the `nvidia-*` CUDA packages are separate wheels, so even across
> *different* torch versions any `nvidia-*` whose pinned version happens to
> match is still shared. On Windows the CUDA DLLs live inside the one torch
+117
View File
@@ -0,0 +1,117 @@
# OmniVoice Studio — dots.tts Engine
dots.tts (rednote-hilab) is a **2B** fully-continuous autoregressive TTS,
widely cited as one of the strongest open zero-shot voice-cloning models. It
covers **24 languages**, emits **48 kHz** audio, and is released under
**Apache-2.0** (code + checkpoints).
It runs in its own subprocess **and its own Python venv** with
`transformers==4.57.0`, isolated from the OmniVoice parent process which
pins `transformers>=5.3` — the same isolation primitive used by
[IndexTTS-2](indextts.md) and [MOSS-TTS-v1.5](moss-tts-v15.md).
> **Opt-in, and never a default.** dots.tts is selected explicitly in
> **Settings → Engines** (or `OMNIVOICE_TTS_BACKEND=dots-tts`). It is not
> part of the default install.
## Platform support
- **Linux / macOS only.** dots.tts's upstream package declares Linux and
macOS classifiers and has **no Windows install path**. On Windows the
engine reports itself unavailable in **Settings → Engines** with a clear
reason — run OmniVoice under WSL2 or use a Linux/macOS host.
- **No MPS.** Upstream device selection is CUDA-or-CPU with no Metal branch,
so on Apple Silicon the official package runs on **CPU** (slow but
correct). A faster Apple-Silicon path exists only via community MLX ports,
which OmniVoice does not auto-wire.
- **VRAM:** ~9 GB checkpoint; a 1216 GB CUDA GPU is the realistic target.
## Install
dots.tts is **not** bundled (large checkpoint + conflicting `transformers`).
1. Clone the dots.tts repo on disk:
```bash
git clone https://github.com/rednote-hilab/dots.tts.git
```
2. Install the editable package into a fresh venv with the upstream
constraints. Use `uv pip install -e . -c constraints/recommended.txt`
**never** `uv sync --all-extras`, which would overwrite OmniVoice's lock
file with `transformers==4.57` and break the parent process:
```bash
cd dots.tts
uv venv .venv
uv pip install -e . -c constraints/recommended.txt
```
3. The ~9 GB checkpoint downloads from HuggingFace on first synthesize. The
parent forwards `HF_HOME` / `HF_HUB_CACHE` to the sidecar so the cache is
shared with the rest of OmniVoice's downloads.
4. Set `OMNIVOICE_DOTS_TTS_DIR` to the repo root (the directory that
contains `pyproject.toml` and `constraints/`):
```bash
# macOS / Linux
echo 'export OMNIVOICE_DOTS_TTS_DIR=$HOME/code/dots.tts' >> ~/.zshrc
source ~/.zshrc
```
5. Restart OmniVoice. dots.tts appears in **Settings → Engines** with
`available: true` and `isolation_mode: subprocess`.
## Venv resolution order
OmniVoice probes for a usable dots.tts Python interpreter in this priority
order (see `backend/engines/dots_tts/bootstrap.py`):
1. **`${OMNIVOICE_DOTS_TTS_DIR}/.venv/`** — your existing clone's venv.
2. **`backend/engines/dots_tts/.venv/`** — OmniVoice's own venv, created on
demand by step 3.
3. **Lazy bootstrap**`uv venv` then `uv pip install -e <clone> -c
<clone>/constraints/recommended.txt`. Requires `OMNIVOICE_DOTS_TTS_DIR`.
## Voice cloning
For best fidelity ("continuation cloning"), pass **both** a reference clip
(`ref_audio`) and its exact transcript (`ref_text`). A reference clip alone
does x-vector-only cloning. Keep the reference ~10 s. Upstream requires the
reference audio whenever a transcript is given, so OmniVoice drops a stray
`ref_text` that arrives without `ref_audio`.
## Optional env knobs
| Variable | Default | Purpose |
|----------|---------|---------|
| `OMNIVOICE_DOTS_TTS_DIR` | — | Path to the dots.tts clone (required). |
| `OMNIVOICE_DOTS_TTS_MODEL` | `rednote-hilab/dots.tts-soar` | Checkpoint override (`-base`, `-soar`, `-mf`). |
| `OMNIVOICE_DOTS_TTS_PRECISION` | `bfloat16` (CUDA) / `float32` (CPU) | Inference precision. |
| `OMNIVOICE_DOTS_TTS_OPTIMIZE` | `0` | `1` enables `torch.compile` (slower first call, faster after). |
> Using the `dots.tts-mf` (MeanFlow-distilled) checkpoint? It's tuned for
> **4** flow-matching steps — pass `num_step=4`.
## Common errors
### `dots.tts is not supported on Windows ...`
Upstream is Linux/macOS only. Use WSL2 or a Linux/macOS host.
### `dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR ...`
You haven't pointed OmniVoice at a dots.tts clone yet. Follow **Install**.
## License
Apache-2.0 (code and checkpoints). See the upstream
[README](https://github.com/rednote-hilab/dots.tts/blob/main/README.md).
---
dots.tts runs in a dedicated sidecar venv (it pins `transformers==4.57`,
which conflicts with the parent's `transformers>=5.3`). For why that adds
disk and how uv keeps the cost down, see
[Engine venvs & disk usage](disk-usage.md).
+131
View File
@@ -0,0 +1,131 @@
# OmniVoice Studio — MOSS-TTS-v1.5 Engine
MOSS-TTS-v1.5 (OpenMOSS) is an **8B** flagship zero-shot TTS — a Qwen3-8B
language backbone plus a 1.6B audio codec. It covers **31 languages**, does
zero-shot voice cloning, token-level duration control and inline
`[pause Ns]` markers. Released under **Apache-2.0** (code + weights).
It runs in its own subprocess **and its own Python venv** with
`transformers==5.0.0`, isolated from the OmniVoice parent process which
pins `transformers>=5.3`. This is the same isolation primitive used by
[IndexTTS-2](indextts.md): the two `transformers` pins cannot share one
interpreter, so MOSS runs behind
`backend/services/subprocess_backend.py::SubprocessBackend`.
> **Opt-in, and never a default.** MOSS-TTS-v1.5 is selected explicitly in
> **Settings → Engines** (or `OMNIVOICE_TTS_BACKEND=moss-tts-v15`). It is
> not part of the default install and does not change OmniVoice's
> out-of-the-box behaviour on any platform.
## Hardware
- **VRAM/RAM:** an 8B model. The upstream llama.cpp pipeline fits the 8B on
8 GB GPUs when quantized; the bf16 Transformers path used here is ~16 GB
of weights, so a 16 GB+ GPU is the realistic CUDA target. It also runs on
**CPU** (fp32) — correct but slow.
- **Device:** CUDA when present, else CPU. **There is no MPS path**
upstream documents only CUDA/CPU and the custom modelling code is
untested on Apple Silicon, so OmniVoice never routes MOSS to MPS. On a
Mac it runs on CPU.
## Install
MOSS-TTS-v1.5 is **not** bundled (the model is large and the package pins a
conflicting `transformers`). OmniVoice ships a sidecar runner that loads it
into an isolated venv on demand.
1. Clone the MOSS-TTS repo on disk:
```bash
git clone https://github.com/OpenMOSS/MOSS-TTS.git
```
2. Install the editable package into a fresh venv. Use
`uv pip install -e ".[torch-runtime]"`**never** `uv sync --all-extras`,
which would overwrite OmniVoice's lock file with `transformers==5.0` and
break the parent process. The `torch-runtime` extra is CUDA (`+cu128`):
```bash
cd MOSS-TTS
uv venv .venv
uv pip install -e ".[torch-runtime]"
```
On a **non-CUDA / CPU host** (e.g. Apple Silicon), install plain
`torch`/`torchaudio`/`transformers==5.0.0` into the venv instead of the
`+cu128` extra (the auto-bootstrap below only targets CUDA hosts).
3. The ~16 GB weights download from HuggingFace on first synthesize. The
parent forwards `HF_HOME` / `HF_HUB_CACHE` to the sidecar so the cache is
shared with the rest of OmniVoice's downloads.
4. Set `OMNIVOICE_MOSS_TTS_V15_DIR` to the repo root (the directory that
contains `pyproject.toml`):
```bash
# macOS / Linux
echo 'export OMNIVOICE_MOSS_TTS_V15_DIR=$HOME/code/MOSS-TTS' >> ~/.zshrc
source ~/.zshrc
```
```powershell
# Windows PowerShell
[Environment]::SetEnvironmentVariable("OMNIVOICE_MOSS_TTS_V15_DIR","$env:USERPROFILE\code\MOSS-TTS","User")
```
5. Restart OmniVoice. MOSS-TTS-v1.5 appears in **Settings → Engines** with
`available: true` and `isolation_mode: subprocess`.
## Venv resolution order
OmniVoice probes for a usable MOSS Python interpreter in this priority
order (see `backend/engines/moss_tts_v15/bootstrap.py`):
1. **`${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv/`** — your existing clone's venv.
Highest priority, so a power user who already set MOSS up gets zero
re-install.
2. **`backend/engines/moss_tts_v15/.venv/`** — OmniVoice's own venv,
created on demand by step 3.
3. **Lazy bootstrap** — if neither venv exists, OmniVoice runs `uv venv`
then `uv pip install --python <python> -e "${DIR}[torch-runtime]"`.
Requires `OMNIVOICE_MOSS_TTS_V15_DIR`; raises a clear error otherwise.
On a non-CUDA host the `+cu128` extra cannot resolve — set the venv up
manually per step 2.
## Voice cloning
Pass a reference clip as `ref_audio`. MOSS's zero-shot clone mode needs only
the audio (no transcript). Without a reference, MOSS synthesizes in its own
default voice. `duration` (seconds) maps to MOSS's `tokens` argument at
~12.5 tokens/second.
## Optional env knobs
| Variable | Default | Purpose |
|----------|---------|---------|
| `OMNIVOICE_MOSS_TTS_V15_DIR` | — | Path to the MOSS-TTS clone (required). |
| `OMNIVOICE_MOSS_TTS_V15_MODEL` | `OpenMOSS-Team/MOSS-TTS-v1.5` | HF repo id override (mirror / air-gapped). |
| `OMNIVOICE_MOSS_TTS_V15_ATTN` | `sdpa` | Attention impl; set `flash_attention_2` on Ampere+ CUDA with `flash-attn` installed. |
## Common errors
### `MOSS-TTS-v1.5 venv not found. Set OMNIVOICE_MOSS_TTS_V15_DIR ...`
You haven't pointed OmniVoice at a MOSS-TTS clone yet. Follow **Install**.
### `uv pip install -e failed ... '[torch-runtime]' extra (cu128) cannot resolve`
You're on a non-CUDA host. The upstream `torch-runtime` extra is CUDA-only;
set up the venv manually with plain `torch`/`transformers==5.0.0` (step 2).
## License
Apache-2.0 (code and weights) — no acceptance gate. See the upstream
[README](https://github.com/OpenMOSS/MOSS-TTS/blob/main/README.md).
---
MOSS-TTS-v1.5 runs in a dedicated sidecar venv (it pins `transformers==5.0`,
which conflicts with the parent's `transformers>=5.3`). For why that adds
disk and how uv keeps the cost down, see
[Engine venvs & disk usage](disk-usage.md).
+6
View File
@@ -46,6 +46,12 @@ tts_engines:
doc: docs/engines/indextts.md
- id: omnivoice-gguf
- id: supertonic3
- id: moss-tts-v15
readme: "**MOSS-TTS-v1.5**"
doc: docs/engines/moss-tts-v15.md
- id: dots-tts
readme: "**dots.tts**"
doc: docs/engines/dots-tts.md
# Same contract against backend/services/asr_backend.py _REGISTRY.
asr_engines:
+15 -9
View File
@@ -25,18 +25,24 @@ manifest:
Both manifests are signed with the same minisign key, so a tampered build is
rejected regardless of channel.
## For maintainers — cutting a preview build
## For maintainers — how previews are built
Preview builds are **manual** (no scheduled spend, nothing auto-published):
Preview builds come from **`main`**, two ways:
1. Go to **Actions → Desktop Release → Run workflow**.
2. Pick the branch to build (usually `main`).
3. Set **publish_preview = true** and run.
- **Nightly (automatic).** A scheduled job (07:00 UTC) rebuilds the rolling
`preview` prerelease from `main` — but only when `main` actually moved in the
last day, so idle days cost nothing. Preview is never more than ~24h behind
`main`.
- **On demand.** **Actions → Desktop Release → Run workflow**, pick a branch
(usually `main`), set **publish_preview = true**. Useful to cut a preview off
a feature branch, or to refresh immediately without waiting for the nightly.
This builds the matrix and publishes/updates a single rolling `preview`
**prerelease** with its own signed `latest.json`. The tagged `latest` stable
release is never affected. Preview users get the new build on their next check;
stable users see nothing.
Either way it builds the matrix and publishes/updates a single rolling
`preview` **prerelease** — always flagged prerelease, and carrying the same
platform set as stable (both verified in CI after each preview publish) — with
its own signed `latest.json`. The tagged `latest` stable release is never
affected. Preview users get the new build on their next check; stable users see
nothing.
To stop offering previews, delete the `preview` release/tag on GitHub — the
Preview channel then falls back to stable.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "omnivoice-studio",
"private": true,
"version": "0.3.6",
"version": "0.3.7",
"license": "AGPL-3.0-only",
"type": "module",
"scripts": {
+3 -3
View File
@@ -1,7 +1,7 @@
{
"raised": 137.5,
"raised": 10,
"goal": 200,
"currency": "USD",
"sponsorCount": 23,
"updated": "2026-06-16"
"sponsorCount": 1,
"updated": "2026-06-17"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "omnivoice-studio"
version = "0.3.6"
version = "0.3.7"
description = "OmniVoice Studio AI voice cloning & dubbing desktop app"
authors = ["Debpalash"]
license = "AGPL-3.0-only"
+25 -7
View File
@@ -501,14 +501,20 @@ fn quarantine_broken_venv(venv_dir: &Path) -> bool {
}
/// Whether a dead backend process looks like it failed because the venv
/// itself is structurally broken — the CPython venv launcher prints
/// "No pyvenv.cfg file" and exits with code 106 (`RC_NO_PYVENV_CFG`). Matches
/// either the message in the captured stderr tail or the exit code in the
/// `ExitStatus` display ("exit code: 106" on Windows, "exit status: 106" on
/// Unix). Kept deliberately narrow so ordinary backend crashes never trigger
/// a venv rebuild.
/// itself is structurally broken — either the CPython venv launcher's
/// "No pyvenv.cfg file" + exit 106 (`RC_NO_PYVENV_CFG`), OR a relocated/copied/
/// restored venv whose interpreter can't bootstrap its own stdlib and aborts
/// very early with "No module named 'encodings'" (exit 1). Both are
/// unrunnable-interpreter cases that `uv sync` cannot fix — only a venv rebuild
/// can — so both route into the rebuild-once self-heal. Matches the message in
/// the captured stderr tail or the exit code in the `ExitStatus` display
/// ("exit code: 106" on Windows, "exit status: 106" on Unix). Kept deliberately
/// narrow (full quoted phrases) so an ordinary backend crash — or an app-level
/// import error of some 'encodings'-named package — never triggers a rebuild.
pub fn backend_exit_indicates_broken_venv(exit_info: &str, err_tail: &str) -> bool {
err_tail.contains("No pyvenv.cfg file") || exit_info.trim_end().ends_with(": 106")
err_tail.contains("No pyvenv.cfg file")
|| err_tail.contains("No module named 'encodings'")
|| exit_info.trim_end().ends_with(": 106")
}
/// Prepare (and on first run, create) the Python venv that will host the
@@ -1204,6 +1210,18 @@ mod tests {
assert!(!backend_exit_indicates_broken_venv("exit status: 1060", ""));
assert!(!backend_exit_indicates_broken_venv("signal: 6 (SIGABRT)", ""));
assert!(!backend_exit_indicates_broken_venv("never started", ""));
// A relocated/copied venv whose interpreter can't bootstrap its stdlib
// aborts with this exact phrase (exit 1, not 106) — must rebuild.
assert!(backend_exit_indicates_broken_venv(
"exit status: 1",
"ModuleNotFoundError: No module named 'encodings'"
));
// ...but an app-level import of an 'encodings'-prefixed package must NOT
// (the full quoted phrase guards against this).
assert!(!backend_exit_indicates_broken_venv(
"exit status: 1",
"ModuleNotFoundError: No module named 'encodings_helper'"
));
}
/// #248: verify that the setuptools repair install uses the correct specifier.
+11 -3
View File
@@ -1,7 +1,7 @@
{
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
"productName": "OmniVoice Studio",
"version": "0.3.6",
"version": "../package.json",
"identifier": "com.debpalash.omnivoice-studio",
"build": {
"frontendDist": "../dist",
@@ -46,13 +46,21 @@
"csp": "default-src 'self' 'unsafe-inline' 'unsafe-eval'; connect-src 'self' ipc://localhost http://localhost:* ws://localhost:* http://127.0.0.1:* ws://127.0.0.1:* blob: data:; media-src 'self' blob: data: http://localhost:* http://127.0.0.1:* asset: https://asset.localhost; img-src 'self' blob: data: asset: https://asset.localhost http://localhost:* http://127.0.0.1:* https://fonts.gstatic.com; font-src 'self' data: https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com;",
"assetProtocol": {
"enable": true,
"scope": ["**"]
"scope": [
"**"
]
}
}
},
"bundle": {
"active": true,
"targets": ["dmg", "app", "msi", "deb", "appimage"],
"targets": [
"dmg",
"app",
"msi",
"deb",
"appimage"
],
"createUpdaterArtifacts": true,
"icon": [
"icons/32x32.png",
+26 -7
View File
@@ -1,4 +1,4 @@
import React, { useState, useRef, useEffect, useCallback, Suspense, lazy } from 'react';
import React, { useState, useRef, useEffect, useLayoutEffect, useCallback, Suspense, lazy } from 'react';
import './index.css';
import { useAppStore, FONT_STACKS } from './store';
import SearchableSelect from './components/SearchableSelect';
@@ -86,12 +86,12 @@ function App() {
// Responsive shell breakpoints driven off the app-container's OWN width, not
// the viewport. The shell is sized `width: calc(100vw / --ui-scale)` then
// `transform: scale(--ui-scale)` (the WebKitGTK fix, #407), so the grid lays
// out against `100vw/scale` which `el.clientWidth` reports (transforms don't
// change the layout box). Viewport `@media` queries fire on raw `100vw` and so
// collapse at the wrong threshold whenever the UI scale 1, cramming the
// content into a sliver. ResizeObserver fires on both window resize and scale
// change (the calc width changes), so this stays correct on every engine.
// `zoom: --ui-scale` (#504; the WebKitGTK no-op case is handled by the
// data-zoom-layout probe below), so the grid lays out against `100vw/scale`
// which `el.clientWidth` reports. Viewport `@media` queries fire on raw
// `100vw` and so collapse at the wrong threshold whenever the UI scale 1,
// cramming the content into a sliver. ResizeObserver fires on both window
// resize and scale change (the calc width changes), so this stays correct.
const shellRef = useRef(null);
const [shellWidth, setShellWidth] = useState(Infinity);
useEffect(() => {
@@ -102,6 +102,25 @@ function App() {
setShellWidth(el.clientWidth);
return () => ro.disconnect();
}, []);
// Engine capability probe (#523/#524): does this WebView honor `zoom` as a
// LAYOUT transform? Chromium (WebView2 / macOS WebKit) and modern WebKitGTK
// do; older WebKitGTK (Linux) treats it as a no-op. The .app-container sizing
// branches on the result (index.css) so the shell fills the window on BOTH
// no black band on WebKitGTK, no clipped Generate/Settings CTAs on Chromium.
// Measuring a real zoomed element is robust where @supports(zoom)/UA-sniffing
// aren't (both report "supported" on WebKitGTK even when zoom doesn't lay out).
useLayoutEffect(() => {
let honored = true;
try {
const probe = document.createElement('div');
probe.style.cssText = 'position:absolute;left:-9999px;top:0;width:100px;height:100px;zoom:2';
document.body.appendChild(probe);
honored = Math.round(probe.getBoundingClientRect().width) >= 150;
probe.remove();
} catch { honored = true; } // safe default: the existing zoom path
document.documentElement.dataset.zoomLayout = honored ? 'on' : 'off';
}, []);
const shellSizeClass = shellWidth <= 600 ? 'shell-mini' : shellWidth <= 1100 ? 'shell-narrow' : '';
const theme = useAppStore(s => s.theme);
+3 -3
View File
@@ -30,11 +30,11 @@ export interface DonationProgress {
* bump both a test asserts the bundled fallback is well-formed.
*/
export const BUNDLED_PROGRESS: DonationProgress = {
raised: 137.5,
raised: 10,
goal: 200,
currency: 'USD',
sponsorCount: 23,
updated: '2026-06-16',
sponsorCount: 1,
updated: '2026-06-17',
};
/** Where the runtime-refreshable copy lives (served from `public/`). */
+5 -5
View File
@@ -26,7 +26,7 @@ import { exportStems } from '../utils/storyExport';
import { storyToSpans } from '../utils/storyToSpans';
import { consumeLongformStream } from '../utils/longformStream';
import { reorder } from '../utils/storyReorder';
import { effectiveProfile, castMember, nextCastColor } from '../utils/storyCast';
import { effectiveProfile, effectiveSpeed, castMember, nextCastColor } from '../utils/storyCast';
import './StoriesEditor.css';
// Trigger a browser download for a Blob.
@@ -302,7 +302,7 @@ export default function StoriesEditor({ profiles = [] }) {
const raw = (track.text || '').trim();
if (!raw) return;
const pid = effectiveProfile(track, cast);
const spd = track.speed || 1.0;
const spd = effectiveSpeed(track, globalSpeed);
setTracks((prev) => prev.map((tk) => (tk.id === track.id ? { ...tk, generating: true } : tk)));
if (!hasStoryMarkers(raw)) {
@@ -349,7 +349,7 @@ export default function StoriesEditor({ profiles = [] }) {
console.warn('Stories chained preview failed:', err);
setTracks((prev) => prev.map((tk) => (tk.id === track.id ? { ...tk, generating: false } : tk)));
}
}, [fetchChunkAudio, cast, setTracks]);
}, [fetchChunkAudio, cast, globalSpeed, setTracks]);
// Deliver a stitched WAV in the chosen format. MP3 routes through the backend
// ffmpeg endpoint; if that fails (e.g. no ffmpeg), fall back to the raw WAV.
@@ -416,7 +416,7 @@ export default function StoriesEditor({ profiles = [] }) {
try {
const stems = await exportStems(
usable,
(tk) => ({ profileId: effectiveProfile(tk, cast), speed: tk.speed || 1.0 }),
(tk) => ({ profileId: effectiveProfile(tk, cast), speed: effectiveSpeed(tk, globalSpeed) }),
fetchChunkBlob,
(d, total) => setExportPct(total ? Math.round((d / total) * 100) : 0),
);
@@ -431,7 +431,7 @@ export default function StoriesEditor({ profiles = [] }) {
} finally {
setExporting(false);
}
}, [tracks, cast, fetchChunkBlob, exporting, deliver, t]);
}, [tracks, cast, fetchChunkBlob, exporting, deliver, globalSpeed, t]);
// Stats
const totalChars = tracks.reduce((acc, tk) => acc + tk.text.length, 0);
+15 -8
View File
@@ -16,9 +16,10 @@
*/
import React, { useEffect, useRef, useState } from 'react';
import WaveSurfer from 'wavesurfer.js';
import { Play, Pause, Loader } from 'lucide-react';
import { Play, Pause } from 'lucide-react';
import { claimPlayback } from '../utils/playback';
import { isTauri, fileToMediaUrl } from '../utils/media';
import { unlockAudio } from '../utils/audioUnlock';
import { useAppStore } from '../store';
import './WaveformPlayer.css';
@@ -196,11 +197,19 @@ export default function WaveformPlayer({
};
}, [resolvedUrl, failed, height, source, onEnded]);
const togglePlay = () => {
const togglePlay = async () => {
// Browser autoplay policy (Linux FF/Chrome, Android Chrome): WaveSurfer's
// AudioContext starts suspended until a user gesture. This click IS the
// gesture explicitly resume so the subsequent play() succeeds. No-op
// on macOS where the context was never blocked.
try { await unlockAudio(); } catch { /* ignore — play() will surface errors */ }
// playPause is async a swallowed rejection here is exactly how the
// "click does nothing" bug hid; log it so playback failures are visible.
Promise.resolve(wsRef.current?.playPause())
.catch((e) => console.warn('WaveformPlayer: play failed:', e));
try {
await wsRef.current?.playPause();
} catch (e) {
console.warn('WaveformPlayer: play failed:', e);
}
};
if (!resolvedUrl) return null;
@@ -240,12 +249,10 @@ export default function WaveformPlayer({
type="button"
className="wf-player__btn"
onClick={togglePlay}
disabled={!ready}
disabled={!resolvedUrl}
aria-label={isPlaying ? 'Pause' : 'Play'}
>
{!ready
? <Loader size={compact ? 13 : 15} className="wf-player__spin" />
: isPlaying ? <Pause size={compact ? 13 : 15} /> : <Play size={compact ? 13 : 15} />}
{isPlaying ? <Pause size={compact ? 13 : 15} /> : <Play size={compact ? 13 : 15} />}
</button>
<div className="wf-player__wave" ref={containerRef} style={{ height }} />
<span className="wf-player__time">{fmt(currentTime)} / {fmt(duration)}</span>
@@ -8,6 +8,7 @@
*/
import React from 'react';
import { Palette } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { useAppStore, FONT_OPTIONS, FONT_STACKS } from '../../store';
import './AppearancePanel.css';
@@ -21,6 +22,7 @@ const THEMES = [
];
export default function AppearancePanel() {
const { t } = useTranslation();
const uiScale = useAppStore(s => s.uiScale);
const setUiScale = useAppStore(s => s.setUiScale);
const theme = useAppStore(s => s.theme);
@@ -28,14 +30,18 @@ export default function AppearancePanel() {
const font = useAppStore(s => s.font);
const setFont = useAppStore(s => s.setFont);
const scaleLabel = t('settings.ui_scale', { defaultValue: 'UI scale' });
const themeLabel = t('settings.color_theme', { defaultValue: 'Color theme' });
const fontLabel = t('settings.font', { defaultValue: 'Font' });
return (
<section className="appearance-panel" aria-labelledby="appearance-panel-heading">
<h3 id="appearance-panel-heading" className="appearance-panel__title">
<Palette size={14} /> Appearance
<Palette size={14} /> {t('settings.appearance', { defaultValue: 'Appearance' })}
</h3>
<div className="appearance-panel__row">
<span className="appearance-panel__label">UI scale</span>
<span className="appearance-panel__label">{scaleLabel}</span>
<div className="appearance-panel__scale">
<input
type="range"
@@ -44,7 +50,7 @@ export default function AppearancePanel() {
step="0.05"
value={uiScale}
onChange={(e) => setUiScale(Number(e.target.value))}
aria-label="UI scale"
aria-label={scaleLabel}
aria-valuetext={`${Math.round(uiScale * 100)}%`}
/>
<span className="appearance-panel__scale-val">{Math.round(uiScale * 100)}%</span>
@@ -52,18 +58,18 @@ export default function AppearancePanel() {
</div>
<div className="appearance-panel__row">
<span className="appearance-panel__label">Color theme</span>
<div className="appearance-panel__themes" role="radiogroup" aria-label="Color theme">
{THEMES.map(t => (
<span className="appearance-panel__label">{themeLabel}</span>
<div className="appearance-panel__themes" role="radiogroup" aria-label={themeLabel}>
{THEMES.map(th => (
<button
key={t.id}
key={th.id}
type="button"
className={`appearance-panel__theme-dot ${theme === t.id ? 'is-active' : ''}`}
style={{ '--dot-color': t.dot }}
onClick={() => setTheme(t.id)}
title={t.label}
aria-label={`${t.label} theme`}
aria-checked={theme === t.id}
className={`appearance-panel__theme-dot ${theme === th.id ? 'is-active' : ''}`}
style={{ '--dot-color': th.dot }}
onClick={() => setTheme(th.id)}
title={th.label}
aria-label={th.label}
aria-checked={theme === th.id}
role="radio"
/>
))}
@@ -71,8 +77,8 @@ export default function AppearancePanel() {
</div>
<div className="appearance-panel__row appearance-panel__row--stack">
<span className="appearance-panel__label">Font</span>
<div className="appearance-panel__fonts" role="radiogroup" aria-label="Font">
<span className="appearance-panel__label">{fontLabel}</span>
<div className="appearance-panel__fonts" role="radiogroup" aria-label={fontLabel}>
{FONT_OPTIONS.map(f => (
<button
key={f.id}
@@ -93,9 +99,10 @@ export default function AppearancePanel() {
</div>
<p className="appearance-panel__help">
These controls used to live in the bottom logs bar moved here so
the footer can stay focused on logs. Changes apply instantly and
persist across launches.
{t('settings.appearance_help', {
defaultValue:
'These controls used to live in the bottom logs bar — moved here so the footer can stay focused on logs. Changes apply instantly and persist across launches.',
})}
</p>
</section>
);
+6 -1
View File
@@ -5,6 +5,7 @@ import { createProfile, deleteProfile as apiDeleteProfile, lockProfile, unlockPr
import { generateSpeech, audioUrlWithCacheBust } from '../api/generate';
import { playBlobAudio } from '../utils/media';
import { PRESETS } from '../utils/constants';
import { instructToFormValue } from '../utils/voiceInstruct';
import { askConfirm } from '../utils/dialog';
import { toast } from 'react-hot-toast';
import { evaluateDonationPrompt } from '../components/donate/evaluateDonationPrompt';
@@ -93,7 +94,11 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
fd.append('name', profileName);
fd.append('kind', 'design');
fd.append('vd_states', JSON.stringify(vdStates || {}));
fd.append('instruct', instruct || '');
// Defensive: instruct must be the STRING. buildDesignInstruct() returns an
// object — appending it coerced to "[object Object]", poisoning the profile
// (#550 et al). instructToFormValue extracts .instruct if an object slips
// through, so the field is never garbage.
fd.append('instruct', instructToFormValue(instruct));
fd.append('language', language || 'Auto');
try {
await createProfile(fd);
+3
View File
@@ -277,6 +277,9 @@
"about": "About",
"ui_scale": "UI Scale",
"theme": "Theme",
"color_theme": "Color theme",
"font": "Font",
"appearance_help": "These controls used to live in the bottom logs bar — moved here so the footer can stay focused on logs. Changes apply instantly and persist across launches.",
"language": "Language",
"language_desc": "Select the interface language",
"engines": "Engines",
+29 -18
View File
@@ -190,23 +190,25 @@ samp,
/* ═══ LAYOUT ═══ */
.app-container {
/* UI scale (#21 black-band fix PERMANENT): scale via `zoom`, and the shell
ALWAYS occupies the full viewport (`100vw`/`100vh`, NOT `calc(/scale)`).
Why this never black-bands on any engine:
Chromium (macOS/Windows): `zoom` magnifies AND fills standard browser
zoom (the same `style={{zoom:uiScale}}` the bootstrap/wizard wrappers
already use here).
WebKitGTK (Linux): `zoom` is a no-op the UI renders at 1.0× but the
shell is still a plain `100vw × 100vh` element it FILLS the window.
The previous approach paired `width: calc(100vw/scale)` with `transform:
scale(var(--ui-scale))`; on WebKitGTK the transform wasn't magnifying the
shrunk shell, so `calc(100vw/1.3)` (the default scale) left ~ of the
window black. Dropping the `/scale` shrink makes a missed magnification
degrade to "unscaled but full", never to "shrunk + black band".
Do NOT reintroduce `width: calc(100vw / var(--ui-scale))` or
`transform: scale(var(--ui-scale))` here a regression test
(src/test/appShellScale.test.js) fails CI if either pattern returns. */
width: 100vw;
/* UI scale (#504 clipping fix + #21 black-band fix): the layout box is
shrunk by the scale factor, then `zoom` magnifies it back to the viewport.
This guarantees the scaled UI never overflows the window and never leaves
black bands:
Chromium (macOS/Windows): `zoom` magnifies the shrunk layout box back
to exactly `100vw × 100vh`, so content fits with no clipping.
Older WebKitGTK (Linux): `zoom` is a layout no-op, so the shrunk
`100vw/scale` box would leave a black band on the right/bottom AND push
the bottom Generate/Settings CTAs off-screen (#523/#524). The App.jsx
zoom-layout probe sets `data-zoom-layout=off` on those engines, and the
override below renders the shell at 1.0 filling the window (no band, no
clip). Newer WebKitGTK that honors zoom keeps the scaled path.
The old `transform: scale(var(--ui-scale))` approach is what caused black
bands on WebKitGTK because the transform didn't magnify the shrunk shell.
Do NOT replace `zoom` with `transform: scale(var(--ui-scale))` a
regression test (src/test/appShellScale.test.js) fails CI if that pattern
returns. */
width: calc(100vw / var(--ui-scale, 1));
height: calc(100vh / var(--ui-scale, 1));
zoom: var(--ui-scale, 1);
max-width: none;
display: grid;
@@ -222,10 +224,19 @@ samp,
footer's state. Content columns (not the grid) yield to the fixed footer
via padding-bottom below, so expanding the logs panel never compresses
the side menubar. */
height: 100vh;
overflow: hidden;
transition: grid-template-columns var(--transition-smooth);
}
/* Older WebKitGTK (Linux), where the App.jsx probe found `zoom` is a layout
no-op html[data-zoom-layout=off]: the calc(100vw/scale) box can't be
magnified back, so it leaves a black band and clips the bottom Generate/
Settings CTAs (#523/#524). Fill the viewport at 1.0 instead no band, no
clip. The probe leaves the zoom path in place on engines that honor zoom. */
html[data-zoom-layout='off'] .app-container {
width: 100vw;
height: 100vh;
zoom: 1;
}
/* Keep scrollable content clear of the fixed footer. The footer writes its
current height (pre-zoom px same coordinate space as these children) to
--logs-footer-height whenever it collapses/expands/resizes. */
+11
View File
@@ -6,6 +6,17 @@ if (import.meta.env.DEV && !window.__vite_plugin_react_preamble_installed__) {
window.__vite_plugin_react_preamble_installed__ = true;
}
// AudioContext autoplay-policy unlock MUST install before any module that
// constructs an AudioContext (wavesurfer.js, the AEC tap, the dictation
// capture, etc.). The side-effecting import patches `window.AudioContext`
// to track every instance ever created; `installAudioUnlock()` then wires
// a one-time pointerdown/keydown listener that resumes them all on the
// first user gesture. Without this, Linux Firefox/Chrome and Android Chrome
// leave WaveSurfer's AudioContext suspended peaks decode hangs `ready`
// never fires play button stays disabled no /audio/ request ever fires.
import { installAudioUnlock } from './utils/audioUnlock.js';
installAudioUnlock();
const { bootstrapApp } = await import('./main-app.jsx');
bootstrapApp();
+1 -1
View File
@@ -606,7 +606,7 @@ export default function CloneDesignTab(props) {
onChange={e => setProfileName(e.target.value)}
/>
<Button variant="subtle" size="sm"
onClick={() => handleSaveDesignProfile(vdStates, buildDesignInstruct(vdStates, instruct), language)}>
onClick={() => handleSaveDesignProfile(vdStates, buildDesignInstruct(vdStates, instruct).instruct, language)}>
{t('clone.save')}
</Button>
<Button variant="ghost" size="sm" onClick={() => setShowSaveProfile(false)}>{t('clone.cancel')}</Button>
+22 -1
View File
@@ -8,6 +8,7 @@ import {
import { apiFetch } from '../api/client';
import { loadTranscriptions, TRANSCRIPTION_EVENT } from '../utils/transcriptionsStore';
import { audioUrl } from '../api/generate';
import { playBlobAudio } from '../utils/media';
import './Projects.css';
/**
@@ -51,6 +52,26 @@ function fmtDuration(sec) {
return `${m}m ${s}s`;
}
/**
* Preview a finished render (audiobook/story) inside the app.
*
* Previously the card did `window.open(url, '_blank')`. Under Tauri's WebView2
* on Windows that handed the file to a new webview/OS media surface, spawning a
* separate black playback window with centered controls that the user couldn't
* close without force-quitting the whole app (#532). Fetch the file and route
* it through the shared single-playback manager instead identical, in-app
* behavior on macOS/Windows/Linux, and starting another preview stops this one.
*/
async function playRenderInApp(url) {
try {
const resp = await fetch(url, { cache: 'no-store' });
if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
await playBlobAudio(await resp.blob());
} catch (e) {
console.error('[Projects] render playback failed:', e);
}
}
function Card({ kind, accent, title, subtitle, trailing, onClick, IconC }) {
return (
<button
@@ -201,7 +222,7 @@ export default function Projects({
ts: (j.created_at || 0) * 1000,
accent: '#d3869b',
Icon: BookMarked,
onClick: () => j.output && window.open(audioUrl(j.output), '_blank'),
onClick: () => j.output && playRenderInApp(audioUrl(j.output)),
});
}
for (const tr of transcriptions) {
+9 -1
View File
@@ -2,7 +2,11 @@
Chrome tokens only: every surface here should rhyme with the status bar. */
/* Tighter page chrome — reclaim whitespace around title and section padding. */
.settings-page { padding: 10px 24px 24px; }
/* Fill the scroll container so short tabs (Appearance/About/Privacy) render as a
full-height panel instead of a stunted box floating in a void. min-height:100%
is a FLOOR tall tabs (Models/Logs) grow past it and scroll as before; if the
parent height is ever indeterminate it simply no-ops. */
.settings-page { padding: 10px 24px 24px; display: flex; flex-direction: column; min-height: 100%; box-sizing: border-box; }
.settings-page h1 { font-size: 1.15rem; margin: 0 0 2px; }
.settings-page .settings-subtitle{ font-size: 0.72rem; margin-bottom: 14px; }
.settings-tabs-ui { margin-bottom: var(--space-3); }
@@ -97,6 +101,10 @@
sit. The border is tinted by the active tab's accent so the active tab and
its panel read as connected by a shared border. */
margin-top: 0;
/* Grow to fill the flex-column .settings-page so the bordered panel reaches
the viewport bottom on short tabs (only adds space when there's slack
tall tabs keep their natural height + internal scroll). */
flex: 1 1 auto;
padding: var(--space-5) var(--space-5) var(--space-4);
border: 1px solid color-mix(in srgb, var(--settings-accent) 38%, var(--chrome-border) 62%);
/* Stronger top: a 2px full-accent edge at the seam with the bar, so the
+3 -2
View File
@@ -28,6 +28,7 @@ import { useQueryClient } from '@tanstack/react-query';
import { selectEngine } from '../api/engines';
import { setupDownloadStreamUrl } from '../api/setup';
import { getFrontendLogs, clearFrontendLogs } from '../utils/consoleBuffer';
import { resolveAboutVersion } from '../utils/appVersion';
import { Tabs, Segmented, Button, Badge, Table, Progress, Select } from '../ui';
import { useAppStore } from '../store';
import ApiKeysPanel from '../components/settings/ApiKeysPanel';
@@ -1242,7 +1243,7 @@ export default function Settings() {
const lines = [
'### OmniVoice Studio diagnostics',
'',
`- **App version:** ${appVersion || '—'}`,
`- **App version:** ${resolveAboutVersion(appVersion, info)}`,
`- **Tauri runtime:** ${tauriVersion || (isTauri() ? '—' : 'web preview')}`,
`- **Platform:** ${info?.platform || '—'}`,
`- **Architecture:** ${nav.userAgentData?.platform || nav.platform || '—'}`,
@@ -1505,7 +1506,7 @@ export default function Settings() {
<section className="settings-section">
<h2><Info size={16} color="#8ec07c" /> {t('settings.about')}</h2>
<Row label={t('about.app')} value="OmniVoice Studio" />
<Row label={t('about.version')} value={appVersion || info?.app_version || '—'} mono />
<Row label={t('about.version')} value={resolveAboutVersion(appVersion, info)} mono />
<Row label={t('about.tauri_runtime')} value={tauriVersion || (isTauri() ? '—' : t('about.web_preview'))} mono />
<Row label={t('about.platform')} value={info?.platform || '—'} />
<Row label={t('about.architecture')} value={info?.arch || '—'} mono />
@@ -0,0 +1,54 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
// Regression guard for #532: a finished audiobook/story render must preview
// IN-APP. The card used to call window.open(url, '_blank'), which under Tauri's
// WebView2 on Windows spawned a separate, un-closeable black media window. The
// fix routes the render through the shared in-app playback path (playBlobAudio).
vi.mock('../utils/media', () => ({ playBlobAudio: vi.fn() }));
vi.mock('../api/generate', () => ({ audioUrl: (f) => `http://test.local/audio/${f}` }));
vi.mock('../api/client', () => ({
apiFetch: vi.fn(async (path) => {
if (path === '/longform/jobs') {
return {
json: async () => ({
jobs: [{ job_id: 'jb1', output: 'book.wav', title: 'My Audiobook', type: 'audiobook', created_at: 1 }],
}),
};
}
return { json: async () => ({}) };
}),
}));
import Projects from '../pages/Projects';
import { playBlobAudio } from '../utils/media';
describe('Projects — audiobook playback (#532)', () => {
let fetchMock;
beforeEach(() => {
localStorage.clear();
vi.clearAllMocks();
fetchMock = vi.fn(async () => ({ ok: true, blob: async () => new Blob(['x']) }));
vi.stubGlobal('fetch', fetchMock);
});
afterEach(() => { vi.unstubAllGlobals(); vi.restoreAllMocks(); });
it('plays the render in-app and never opens a separate window', async () => {
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => null);
render(<Projects />);
// The longform job loads via apiFetch('/longform/jobs') in an effect.
const card = (await screen.findByText('My Audiobook')).closest('button');
expect(card).toBeTruthy();
fireEvent.click(card);
await waitFor(() => expect(playBlobAudio).toHaveBeenCalledTimes(1));
// In-app fetch of the render file, never a new window/OS media surface.
expect(fetchMock).toHaveBeenCalledWith('http://test.local/audio/book.wav', { cache: 'no-store' });
expect(openSpy).not.toHaveBeenCalled();
});
});
+29 -18
View File
@@ -3,36 +3,47 @@ import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
/**
* PERMANENT regression guard for the app-shell black-band bug (#21).
* Regression guard for the app-shell scale behavior (#21 black bands, #504
* bottom-button clipping).
*
* The shell must scale via `zoom` and ALWAYS occupy the full viewport. The old
* `width: calc(100vw / var(--ui-scale))` + `transform: scale(var(--ui-scale))`
* approach left ~ of the window black on WebKitGTK (the default scale is 1.3,
* and the transform wasn't magnifying the shrunk shell). This test fails CI if
* either foot-gun pattern is reintroduced, so a future change can't silently
* bring the black band back.
* Rule: the shell's layout box must be shrunk by `--ui-scale`, then magnified
* back with `zoom`. On Chromium this gives a scaled UI that exactly fits the
* viewport; on WebKitGTK `zoom` is a no-op so the UI renders smaller but never
* leaves black bands.
*
* Forbidden pattern (caused black bands on WebKitGTK):
* `transform: scale(var(--ui-scale))` paired with the shrunk layout box,
* because WebKitGTK didn't magnify the shrunk shell.
*/
// vitest runs from the frontend/ package dir. Strip /* … */ comments so the
// guard checks real declarations, not the warning comment that quotes the
// forbidden patterns.
// patterns.
const raw = readFileSync(resolve(process.cwd(), 'src/index.css'), 'utf8');
const css = raw.replace(/\/\*[\s\S]*?\*\//g, '');
describe('app shell scale (black-band regression guard)', () => {
it('does NOT shrink the shell with calc(100vw / --ui-scale)', () => {
expect(css).not.toMatch(/calc\(\s*100vw\s*\/\s*var\(--ui-scale/);
expect(css).not.toMatch(/calc\(\s*100vh\s*\/\s*var\(--ui-scale/);
});
describe('app shell scale (black-band + clipping regression guard)', () => {
it('does NOT scale the shell via transform: scale(--ui-scale)', () => {
expect(css).not.toMatch(/transform:\s*scale\(\s*var\(--ui-scale/);
});
it('scales the shell via zoom and fills the viewport (100vw/100vh)', () => {
// .app-container block must use zoom + full-viewport sizing.
it('scales the shell via zoom', () => {
const block = css.slice(css.indexOf('.app-container {'));
expect(block).toMatch(/zoom:\s*var\(--ui-scale/);
expect(block).toMatch(/width:\s*100vw/);
expect(block).toMatch(/height:\s*100vh/);
});
it('shrinks the layout box by --ui-scale so zoomed content fits the viewport', () => {
// width: calc(100vw / var(--ui-scale)) × zoom: var(--ui-scale) ⇒ rendered 100vw.
const block = css.slice(css.indexOf('.app-container {'));
expect(block).toMatch(/width:\s*calc\(\s*100vw\s*\/\s*var\(--ui-scale/);
expect(block).toMatch(/height:\s*calc\(\s*100vh\s*\/\s*var\(--ui-scale/);
});
it('falls back to 100vw/100vh where zoom is a layout no-op (WebKitGTK, #523/#524)', () => {
// The App.jsx zoom-layout probe sets data-zoom-layout=off on engines that
// treat zoom as a layout no-op; this override must then fill the window at
// 1.0 so the shell never leaves a black band or clips the bottom CTAs.
expect(css).toMatch(
/\[data-zoom-layout=['"]?off['"]?\][^{]*\.app-container\s*\{[^}]*width:\s*100vw[^}]*height:\s*100vh/,
);
});
});
+21
View File
@@ -0,0 +1,21 @@
/* global __APP_VERSION__ -- injected by Vite (vite.config define) at build time */
// The build's own version, always present regardless of Tauri presence or
// backend liveness — the always-correct source per the version-lockstep rule
// (it comes from package.json, kept in lockstep with the other version files).
export const APP_VERSION =
(typeof __APP_VERSION__ !== 'undefined' && __APP_VERSION__) || 'unknown';
/**
* Resolve the version shown in Settings About / the diagnostics block. Prefer
* the authoritative Tauri-config version, then the live backend's
* `/system/info` `app_version`, then the build-time constant so it is NEVER
* blank, even in the web/Pinokio build where Tauri is absent and the backend
* may be idle (the About Version field rendered empty there).
*
* @param {string|null|undefined} appVersion Tauri `getVersion()` result
* @param {{ app_version?: string }|null|undefined} info `/system/info` payload
* @returns {string}
*/
export function resolveAboutVersion(appVersion, info) {
return appVersion || info?.app_version || APP_VERSION;
}
+21
View File
@@ -0,0 +1,21 @@
import { describe, it, expect } from 'vitest';
import { resolveAboutVersion, APP_VERSION } from './appVersion';
describe('resolveAboutVersion (About → Version blank-in-web-build fix)', () => {
it('prefers Tauri appVersion, then backend app_version, then the build constant', () => {
expect(resolveAboutVersion('1.2.3', { app_version: '9.9.9' })).toBe('1.2.3');
expect(resolveAboutVersion(null, { app_version: '9.9.9' })).toBe('9.9.9');
expect(resolveAboutVersion('', { app_version: '9.9.9' })).toBe('9.9.9');
});
it('falls back to the build-time version and is NEVER blank/dash (web/Pinokio build)', () => {
// Vite injects __APP_VERSION__ at build time (incl. under vitest), so the
// constant is always a real version string.
expect(APP_VERSION).toBeTruthy();
expect(resolveAboutVersion(null, undefined)).toBe(APP_VERSION);
expect(resolveAboutVersion(null, null)).toBe(APP_VERSION);
expect(resolveAboutVersion(null, {})).toBe(APP_VERSION);
expect(resolveAboutVersion(null, undefined)).not.toBe('');
expect(resolveAboutVersion(null, undefined)).not.toBe('—');
});
});
+94
View File
@@ -0,0 +1,94 @@
/**
* Browser autoplay-policy unlock for AudioContext.
*
* WaveSurfer.js (and several utils here) construct an `AudioContext` at
* component-mount time i.e. before any user gesture. On Linux Firefox/Chrome
* and Android Chrome, browsers leave such a context in `"suspended"` state;
* `decodeAudioData` then hangs WaveSurfer's `ready` event never fires
* the play button stays disabled no `/audio/` request is ever made.
*
* macOS Safari/Chrome are more lenient (typically auto-resume on first
* interaction) which is why the bug only manifests cross-platform.
*
* Fix: monkey-patch `window.AudioContext` to track every instance ever
* created, then resume all of them on the first user gesture (pointerdown /
* keydown). The patch MUST install before any module constructs an
* AudioContext so this file is imported once at the top of main.jsx.
*
* Once unlocked, the document stays unlocked we don't need to re-resume
* on every subsequent gesture.
*/
const _tracked = new WeakSet();
const _resumeQueue = new Set();
const _patch = (Ctor) => {
if (!Ctor || Ctor.__omnivoiceTracked) return Ctor;
class TrackedAudioContext extends Ctor {
constructor(...args) {
super(...args);
_tracked.add(this);
// Some browsers create the context already suspended; queue a resume
// attempt so that when the user gesture arrives, we sweep them all.
if (this.state === 'suspended') _resumeQueue.add(this);
}
}
TrackedAudioContext.__omnivoiceTracked = true;
return TrackedAudioContext;
};
if (typeof window !== 'undefined') {
if (window.AudioContext) window.AudioContext = _patch(window.AudioContext);
if (window.webkitAudioContext && window.webkitAudioContext !== window.AudioContext) {
window.webkitAudioContext = _patch(window.webkitAudioContext);
}
}
let _unlocked = false;
export function unlockAudio() {
if (_unlocked) return Promise.resolve();
_unlocked = true;
// Snapshot then clear — new contexts created post-unlock will start in
// "running" state on their own (the document is now gesture-activated).
const pending = Array.from(_resumeQueue);
_resumeQueue.clear();
return Promise.all(
pending.map((ac) =>
ac.state === 'suspended' ? ac.resume().catch(() => {}) : Promise.resolve()
)
).then(() => {});
}
let _installed = false;
export function installAudioUnlock() {
if (_installed || typeof window === 'undefined') return;
_installed = true;
const opts = { once: true, capture: true };
const handler = () => {
unlockAudio().catch(() => {});
window.removeEventListener('pointerdown', handler, opts);
window.removeEventListener('keydown', handler, opts);
window.removeEventListener('touchstart', handler, opts);
};
// pointerdown beats click — fires earlier, so the resume completes before
// any click handler that depends on the AudioContext running.
window.addEventListener('pointerdown', handler, opts);
window.addEventListener('keydown', handler, opts);
// touchstart for mobile Safari/Chrome which sometimes don't synthesize
// pointerdown fast enough on the very first tap.
window.addEventListener('touchstart', handler, opts);
}
// Test-only escape hatch. Not for production use — the unlock is meant to be
// a one-shot per page load. Resetting lets unit tests exercise the unlock
// path repeatedly against the same module instance.
export function __resetForTesting() {
_unlocked = false;
_installed = false;
_resumeQueue.clear();
// Clear any listeners installAudioUnlock may have wired (capture-phase,
// once: true). removeEventListener is a no-op if the listener isn't
// registered, so call unconditionally — there's no way to reach the
// handler reference from outside, so we accept that already-fired listeners
// are gone (which is what `once:true` guarantees anyway).
}
+108
View File
@@ -0,0 +1,108 @@
import { describe, it, expect, beforeAll, beforeEach } from 'vitest';
// Regression guard for the AudioContext autoplay-policy fix.
// On Linux Firefox/Chrome and Android Chrome, AudioContexts created before
// a user gesture stay suspended — decodeAudioData hangs, WaveSurfer's `ready`
// never fires, play button stays disabled. The fix patches `window.AudioContext`
// to track every instance and exposes unlockAudio() to resume them.
//
// Test strategy: install a fake AudioContext BEFORE importing audioUnlock.js
// (the patch wraps window.AudioContext at module-load time). Each test resets
// the singleton's unlocked flag so the resume path runs fresh.
class FakeAudioContext {
constructor() {
this.state = 'suspended';
this.resumeCalls = 0;
this.resumeImpl = () => {
this.state = 'running';
return Promise.resolve();
};
}
resume() {
this.resumeCalls += 1;
return this.resumeImpl();
}
close() {
this.state = 'closed';
return Promise.resolve();
}
}
let audioUnlock;
beforeAll(async () => {
// Install fake BEFORE the module's top-level patch runs.
window.AudioContext = FakeAudioContext;
audioUnlock = await import('./audioUnlock.js');
});
describe('audioUnlock', () => {
beforeEach(() => audioUnlock.__resetForTesting());
it('AudioContexts are wrapped at construction (proves patch is applied)', () => {
const ctx = new AudioContext();
// The patched class adds the __omnivoiceTracked marker and extends the
// fake (so state is 'suspended' from FakeAudioContext's constructor).
expect(window.AudioContext.__omnivoiceTracked).toBe(true);
expect(ctx.state).toBe('suspended');
expect(ctx.resumeCalls).toBe(0);
});
it('unlockAudio() resumes all suspended tracked contexts', async () => {
const ctx1 = new AudioContext();
const ctx2 = new AudioContext();
const ctx3 = new AudioContext();
await audioUnlock.unlockAudio();
expect(ctx1.state).toBe('running');
expect(ctx2.state).toBe('running');
expect(ctx3.state).toBe('running');
expect(ctx1.resumeCalls).toBe(1);
expect(ctx2.resumeCalls).toBe(1);
expect(ctx3.resumeCalls).toBe(1);
});
it('unlockAudio() is idempotent — repeated calls do not re-resume', async () => {
const ctx = new AudioContext();
await audioUnlock.unlockAudio();
expect(ctx.resumeCalls).toBe(1);
await audioUnlock.unlockAudio();
await audioUnlock.unlockAudio();
await audioUnlock.unlockAudio();
expect(ctx.resumeCalls).toBe(1);
});
it('contexts created AFTER unlock are not re-resumed by a second unlock', async () => {
const before = new AudioContext();
await audioUnlock.unlockAudio();
expect(before.resumeCalls).toBe(1);
const after = new AudioContext();
await audioUnlock.unlockAudio(); // no-op now (idempotent)
expect(after.resumeCalls).toBe(0);
});
it('resume() rejections are swallowed — one bad context does not block others', async () => {
const good = new AudioContext();
const bad = new AudioContext();
bad.resumeImpl = () => Promise.reject(new Error('policy blocked'));
// Should NOT throw — the catch in unlockAudio isolates failures.
await audioUnlock.unlockAudio();
expect(good.state).toBe('running');
expect(bad.state).toBe('suspended'); // its resume rejected; state unchanged
});
it('installAudioUnlock is idempotent — repeated calls are a no-op', () => {
// We can't directly enumerate window listeners, but the internal _installed
// gate guarantees one-time wiring. Calling repeatedly must not throw.
expect(() => {
audioUnlock.installAudioUnlock();
audioUnlock.installAudioUnlock();
audioUnlock.installAudioUnlock();
}).not.toThrow();
});
});
+12
View File
@@ -24,6 +24,18 @@ export function effectiveProfile(track, cast) {
return (member && member.profileId) || null;
}
/**
* Resolve the reading speed a track should use: per-line override the global
* Stories speed (#415) null (the /generate engine default of 1.0×). Mirrors
* the span-speed resolution in storyToSpans (`tk.speed || gspeed || null`) so
* preview, stem export, and longform export all agree on one speed. A global
* of 1 counts as "at rest" null, matching the export path.
*/
export function effectiveSpeed(track, globalSpeed) {
if (track && track.speed) return track.speed;
return (globalSpeed && globalSpeed !== 1) ? globalSpeed : null;
}
/** Find a cast member by id (falls back to the first member). */
export function castMember(cast, id) {
return (cast || []).find((c) => c.id === id) || (cast || [])[0] || null;
+19 -1
View File
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest';
import { nextCastColor, effectiveProfile, castMember, CAST_COLORS } from './storyCast';
import { nextCastColor, effectiveProfile, effectiveSpeed, castMember, CAST_COLORS } from './storyCast';
describe('nextCastColor', () => {
it('returns the first unused palette color', () => {
@@ -27,6 +27,24 @@ describe('effectiveProfile', () => {
});
});
describe('effectiveSpeed', () => {
it('prefers a per-line speed override over the global', () => {
expect(effectiveSpeed({ speed: 1.25 }, 0.7)).toBe(1.25);
});
it('applies the global speed when the track has none (#508)', () => {
// Regression: preview + stem export must follow the global, not 1.0.
expect(effectiveSpeed({ speed: null }, 0.7)).toBe(0.7);
expect(effectiveSpeed({}, 0.7)).toBe(0.7);
});
it('treats a global of 1 as at-rest → null (engine default)', () => {
expect(effectiveSpeed({ speed: null }, 1)).toBeNull();
});
it('returns null when neither override nor a non-default global is set', () => {
expect(effectiveSpeed({ speed: null }, null)).toBeNull();
expect(effectiveSpeed({}, undefined)).toBeNull();
});
});
describe('castMember', () => {
it('finds by id, else first', () => {
const cast = [{ id: 'a' }, { id: 'b' }];
+16
View File
@@ -66,6 +66,22 @@ export function buildDesignInstruct(vdStates = {}, freeText = '') {
return { instruct: Object.values(byCategory).join(', '), unsupported, duplicates };
}
/**
* Coerce an instruct value to the STRING that belongs in the FormData/payload.
* `buildDesignInstruct()` returns `{ instruct, unsupported, duplicates }`, and
* passing that object to `FormData.append` string-coerced it to the literal
* `"[object Object]"`, poisoning saved design profiles (#550 #545 #542 #537
* #530 #525). Always run instruct through this before sending it.
*
* @param {string | { instruct?: string } | null | undefined} instruct
* @returns {string}
*/
export function instructToFormValue(instruct) {
if (typeof instruct === 'string') return instruct;
if (instruct && typeof instruct === 'object') return String(instruct.instruct ?? '');
return '';
}
/**
* Project the backend's "describe your voice" result (#317) onto a fresh
* vdStates object. The description drives the *whole* parameter set matched
+19 -1
View File
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest';
import { buildDesignInstruct } from './voiceInstruct';
import { buildDesignInstruct, instructToFormValue } from './voiceInstruct';
// plan-05 (#132): the Voice Design payload must be a validator-safe instruct —
// one valid tag per category, no unsupported free-text — so Synthesize stops
@@ -57,3 +57,21 @@ describe('buildDesignInstruct', () => {
expect(unsupported).toEqual([]);
});
});
describe('instructToFormValue (#550 [object Object] guard)', () => {
it('extracts the string from a buildDesignInstruct() object, never "[object Object]"', () => {
const built = buildDesignInstruct({ Gender: 'male' }, '');
// the bug: appending the raw object to FormData string-coerces to this
expect(String(built)).toBe('[object Object]');
expect(typeof instructToFormValue(built)).toBe('string');
expect(instructToFormValue(built)).toBe('male');
expect(instructToFormValue(built)).not.toBe('[object Object]');
});
it('passes a plain string through and coerces null/garbage to ""', () => {
expect(instructToFormValue('male, high pitch')).toBe('male, high pitch');
expect(instructToFormValue(null)).toBe('');
expect(instructToFormValue(undefined)).toBe('');
expect(instructToFormValue({})).toBe('');
});
});
+5 -1
View File
@@ -1412,7 +1412,11 @@ def _resolve_instruct(
# Split on both half-width and full-width commas
raw_items = re.split(r"\s*[,]\s*", instruct_str)
raw_items = [x for x in raw_items if x]
# Tolerate the "[object Object]" sentinel a buggy frontend build could have
# persisted into voice_profiles.instruct (#550 et al): drop it instead of
# 400-ing the whole generation. The 0006 migration heals stored values; this
# guards any that slip through (e.g. generation_history rows).
raw_items = [x for x in raw_items if x and x.strip().lower() != "[object object]"]
# Validate each item
unknown = []
+10 -1
View File
@@ -202,7 +202,16 @@ class RuleDurationEstimator:
return self.weights["default"]
def calculate_total_weight(self, text):
"""Sums up the normalized weights for a string."""
"""Sums up the normalized weights for a string.
Text is NFC-normalized first so decomposed (NFD) input a base letter
followed by a combining tone/diacritic mark (common for Vietnamese and
any diacritic script) collapses onto its precomposed form. Combining
marks (U+0300036F) weigh 0.0, so NFD text under-allocated frames and
produced rushed/garbled audio (#502); NFC composition is a no-op for
already-precomposed text.
"""
text = unicodedata.normalize("NFC", text)
return sum(self._get_char_weight(c) for c in text)
def estimate_duration(
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "omnivoice"
version = "0.3.6"
version = "0.3.7"
description = "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models"
readme = "README.md"
# Free and open-source under the GNU Affero General Public License v3 (see
+16
View File
@@ -106,3 +106,19 @@ def test_every_personality_instruct_is_accepted_by_resolve_instruct():
f"Personality {p['id']!r} instruct {p['instruct']!r} normalised "
"to nothing — pick at least one taxonomy token."
)
def test_resolve_instruct_tolerates_object_object_sentinel():
"""A pre-fix Voice Studio build persisted the literal "[object Object]" into
voice_profiles.instruct (#550 et al). _resolve_instruct must DROP that
sentinel and not 400 the whole generation, while still rejecting a genuine
unsupported token."""
resolve_instruct = _import_resolver()
# sentinel alone → treated as empty (no ValueError); returns falsy
assert not resolve_instruct("[object Object]")
assert not resolve_instruct("[OBJECT object]") # case-insensitive
# sentinel mixed with a valid token → the valid token survives, no raise
assert resolve_instruct("male, [object Object]") == "male"
# a real unsupported token must STILL raise (the #114/#115 user feedback)
with pytest.raises(ValueError):
resolve_instruct("frobnicate")
+15 -1
View File
@@ -42,7 +42,21 @@ def load_tauri_config(platform: str | None = None) -> dict:
override_path = _SRC_TAURI / f"tauri.{platform}.conf.json"
if override_path.exists():
base = _deep_merge(base, json.loads(override_path.read_text(encoding="utf-8")))
return base
return _resolve_version(base)
def _resolve_version(config: dict) -> dict:
"""Resolve a package.json `version` reference the way Tauri does.
package.json is the single source of truth, so tauri.conf.json carries
``"version": "../package.json"`` (a path Tauri reads at build time) rather
than a literal. Resolve it here to the effective bundle version so the
config-integrity checks see and assert parity against the real value."""
v = config.get("version")
if isinstance(v, str) and v.endswith("package.json"):
pkg = json.loads((_SRC_TAURI / v).resolve().read_text(encoding="utf-8"))
config = {**config, "version": pkg.get("version", v)}
return config
def pyproject_version() -> str:
+33
View File
@@ -655,3 +655,36 @@ def test_system_info_rejects_non_loopback():
res = non_loopback_client.get("/system/info")
assert res.status_code == 403
assert "loopback" in res.json().get("detail", "").lower()
def test_static_audio_served_with_canonical_mime():
"""Regression: `.wav` files served by `/audio` StaticFiles must come back
with the IANA-canonical `audio/wav` Content-Type, NOT Python's default
`audio/x-wav`.
The `x-` prefix is vendor-experimental and never IANA-registered. macOS
Chrome/Safari MIME-sniff leniently via CoreAudio so playback works there,
but Linux Chrome/Firefox (FFmpeg) and Android Chrome (ExoPlayer) strictly
honor the declared type and treat `audio/x-wav` as download-only which
silently broke the play button in the web app on those platforms.
"""
from pathlib import Path
from fastapi.testclient import TestClient
from main import app
from core.config import OUTPUTS_DIR
# Drop a wav into the real OUTPUTS_DIR — the mount serves from there.
tmp_wav = Path(OUTPUTS_DIR) / f"__mime_test_{uuid.uuid4().hex[:8]}.wav"
tmp_wav.write_bytes(make_wav_bytes(0.1))
try:
client = TestClient(app)
res = client.get(f"/audio/{tmp_wav.name}")
assert res.status_code == 200, res.text
ct = res.headers.get("content-type", "")
assert ct == "audio/wav", (
f"Expected audio/wav (IANA canonical), got {ct!r}. "
f"Linux/Android browsers reject audio/x-wav as download-only."
)
finally:
tmp_wav.unlink(missing_ok=True)
+64 -13
View File
@@ -16,27 +16,78 @@ def test_app_version_matches_installed_package_metadata():
assert APP_VERSION == version("omnivoice")
def test_all_version_files_in_lockstep():
"""The FOUR version files must agree: pyproject.toml,
frontend/src-tauri/{tauri.conf.json,Cargo.toml}, and frontend/package.json.
package.json drives the runtime ``__APP_VERSION__`` (vite.config.js), which
shows in the first-run footer and EVERY auto bug report so a drift ships a
v0.3.6 build that calls itself v0.3.5. The release.yml version-bump job was
bumping only the first three; package.json drifted unnoticed. Catch it in CI.
"""
def test_tauri_version_derives_from_package_json():
"""tauri.conf.json must NOT carry its own version literal — it derives from
package.json (Tauri v2 ``"version": "../package.json"``). package.json is the
single source of truth; a re-hardcoded literal here is exactly the drift that
shipped a 0.3.6 bundle calling itself 0.3.5."""
import json
from pathlib import Path
root = Path(__file__).resolve().parents[1]
tauri_conf = json.loads((root / "frontend/src-tauri/tauri.conf.json").read_text())
assert tauri_conf["version"] == "../package.json", (
"tauri.conf.json must derive its version from package.json "
f'(expected "../package.json", got {tauri_conf["version"]!r})'
)
def test_all_version_files_in_lockstep():
"""``frontend/package.json`` is the SINGLE SOURCE OF TRUTH for the app
version: vite injects ``__APP_VERSION__`` from it (first-run footer + every
bug report), and tauri.conf.json reads its bundle version from it
(``"version": "../package.json"``).
The other three declarations are toolchain-required CI-guarded mirrors
Cargo.toml + pyproject.toml (cargo/uv need a literal) and
backend/core/version.py's ``_FALLBACK_VERSION`` (the frozen-backend last
resort, whose drift to "0.3.5" is why the v0.3.6 build reported 0.3.5). The
release.yml version-bump job bumps the canonical and these mirrors together;
catch any drift here in CI.
"""
from pathlib import Path
root = Path(__file__).resolve().parents[1]
def _toml_version(p: Path) -> str:
return re.search(r'(?m)^version\s*=\s*"([^"]+)"', p.read_text()).group(1)
versions = {
def _named_literal(p: Path, name: str) -> str:
return re.search(rf'(?m)^{name}\s*=\s*"([^"]+)"', p.read_text()).group(1)
import json
canonical = json.loads((root / "frontend/package.json").read_text())["version"]
mirrors = {
"pyproject.toml": _toml_version(root / "pyproject.toml"),
"Cargo.toml": _toml_version(root / "frontend/src-tauri/Cargo.toml"),
"tauri.conf.json": json.loads((root / "frontend/src-tauri/tauri.conf.json").read_text())["version"],
"package.json": json.loads((root / "frontend/package.json").read_text())["version"],
"core/version.py": _named_literal(root / "backend/core/version.py", "_FALLBACK_VERSION"),
}
assert len(set(versions.values())) == 1, f"version files drifted: {versions}"
drifted = {k: v for k, v in mirrors.items() if v != canonical}
assert not drifted, f"version mirrors drifted from package.json={canonical!r}: {drifted}"
def test_fallback_version_resolves_to_pyproject():
"""When package metadata is unavailable (frozen build / raw checkout), the
version must still resolve to pyproject never the stale literal that made
the v0.3.6 build report "0.3.5"."""
from pathlib import Path
from core.version import _fallback_version
root = Path(__file__).resolve().parents[1]
pyproject = re.search(
r'(?m)^version\s*=\s*"([^"]+)"', (root / "pyproject.toml").read_text()
).group(1)
assert _fallback_version() == pyproject
def test_frozen_build_collects_package_metadata():
"""backend.spec must copy_metadata('omnivoice') so the frozen backend reads
its real version via importlib.metadata instead of the fallback literal."""
from pathlib import Path
spec = (Path(__file__).resolve().parents[1] / "backend.spec").read_text()
assert (
"copy_metadata('omnivoice')" in spec or 'copy_metadata("omnivoice")' in spec
), "backend.spec must copy_metadata('omnivoice') (frozen-build version reporting)"
+9
View File
@@ -76,6 +76,15 @@ def test_plan_to_dict_shape():
assert d["chapters"][0]["spans"][0]["text"] == "Hi there."
def test_plan_chapter_count_property():
# Regression for #543: the /audiobook/import endpoint reads plan.chapter_count
# directly (not via to_dict), so the attribute must exist and stay in lockstep
# with the serialized key.
plan = parse_audiobook_script("# A\nHi.\n# B\nBye.")
assert plan.chapter_count == 2
assert plan.chapter_count == plan.to_dict()["chapter_count"]
# ── FFMETADATA ───────────────────────────────────────────────────────────────
def test_ffmetadata_cumulative_offsets():
+89
View File
@@ -0,0 +1,89 @@
"""The runtime schema must self-heal additive columns even when `alembic
upgrade head` can't run — the "no such column: consent_audio_path" 500
(#552/#547) and its whole class (kind/vd_states/is_demo/...).
A DB whose alembic_version is stamped at a revision no longer in versions/
(common after running a preview/main build) makes alembic raise; the failure is
swallowed, and CREATE TABLE IF NOT EXISTS never adds columns to a pre-existing
table so without reconciliation the new columns never land.
"""
import sqlite3
from core.db import _BASE_SCHEMA, _reconcile_additive_columns
def _cols(db_path, table="voice_profiles"):
with sqlite3.connect(str(db_path)) as conn:
return {r[1] for r in conn.execute(f"PRAGMA table_info({table})")}
def _base_schema_cols(table="voice_profiles"):
canon = sqlite3.connect(":memory:")
try:
canon.executescript(_BASE_SCHEMA)
return {r[1] for r in canon.execute(f"PRAGMA table_info({table})")}
finally:
canon.close()
# A pre-consent / pre-unification voice_profiles — missing every alembic-era
# additive column.
_LEGACY_PROFILES = """
CREATE TABLE voice_profiles (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
ref_audio_path TEXT,
ref_text TEXT DEFAULT '',
instruct TEXT DEFAULT '',
language TEXT DEFAULT 'Auto',
created_at REAL
);
"""
def test_init_db_self_heals_missing_columns_when_alembic_fails(tmp_path, monkeypatch):
db = tmp_path / "legacy.db"
with sqlite3.connect(str(db)) as conn:
conn.executescript(_LEGACY_PROFILES)
conn.execute("INSERT INTO voice_profiles(id, name) VALUES ('vp-1', 'Alice')")
# alembic stamped at a revision that no longer exists → command.upgrade
# raises 'Can't locate revision', exercising the swallowed-failure path.
conn.execute("CREATE TABLE alembic_version (version_num VARCHAR(32) NOT NULL)")
conn.execute("INSERT INTO alembic_version VALUES ('0003_preview_removed_rev')")
conn.commit()
monkeypatch.setattr("core.db.DB_PATH", str(db))
from core.db import init_db
init_db() # must NOT raise, and must converge the schema
cols = _cols(db)
for col in ("verified_own_voice", "consent_text", "consent_audio_path",
"consent_recorded_at", "kind", "vd_states", "is_demo"):
assert col in cols, f"schema reconcile did not add {col} (the #552 symptom)"
# the existing row survives
with sqlite3.connect(str(db)) as conn:
assert conn.execute("SELECT name FROM voice_profiles WHERE id='vp-1'").fetchone()[0] == "Alice"
def test_reconcile_converges_voice_profiles_to_base_schema(tmp_path):
db = tmp_path / "stripped.db"
with sqlite3.connect(str(db)) as conn:
conn.executescript(_LEGACY_PROFILES)
conn.commit()
_reconcile_additive_columns(conn)
assert _cols(db) == _base_schema_cols(), "reconcile must match the canonical column set"
def test_reconcile_is_idempotent_and_additive_only(tmp_path):
db = tmp_path / "twice.db"
with sqlite3.connect(str(db)) as conn:
conn.executescript(_LEGACY_PROFILES)
conn.commit()
_reconcile_additive_columns(conn)
after_first = _cols(db)
_reconcile_additive_columns(conn) # second run must be a clean no-op
after_second = _cols(db)
assert after_first == after_second
# additive only — the original legacy columns are never dropped
assert {"id", "name", "instruct", "language"} <= after_second
+163
View File
@@ -0,0 +1,163 @@
"""Tests for the dots.tts engine (issue #498).
dots.tts runs in a dedicated subprocess venv (transformers==4.57.0), so
these tests never import dots.tts itself. They exercise the parent-side
wiring that ships in the default install: registry resolution, subprocess
isolation, the Windows-unsupported gate (cross-platform parity rule),
hardware honesty, and the generate() kwarg arbitration. No network, no
optional deps, no subprocess spawn.
"""
from __future__ import annotations
import importlib
import sys
import pytest
# ── registry wiring ────────────────────────────────────────────────────────
def test_registry_contains_dots_tts():
from services.tts_backend import _REGISTRY, get_backend_class
assert "dots-tts" in _REGISTRY, (
"_REGISTRY is missing 'dots-tts'; check _LAZY_REGISTRY in "
"services/tts_backend.py"
)
cls = _REGISTRY["dots-tts"]
assert cls.__name__ == "DotsTTSBackend"
assert get_backend_class("dots-tts") is cls
assert getattr(cls, "_is_subprocess_isolated", False), (
"DotsTTSBackend should be subprocess-isolated"
)
for name in ("is_available", "generate", "sample_rate", "supported_languages"):
assert hasattr(cls, name), f"DotsTTSBackend missing {name!r}"
def test_pep562_lazy_import():
mod = importlib.import_module("services.tts_backend")
cls = mod._REGISTRY["dots-tts"]
assert cls.__name__ == "DotsTTSBackend"
def test_install_hint_present():
from services.tts_backend import _INSTALL_HINTS
hint = _INSTALL_HINTS.get("dots-tts", "")
assert "OMNIVOICE_DOTS_TTS_DIR" in hint
assert "rednote-hilab" in hint
def test_sidecar_script_ships():
from engines.dots_tts.bootstrap import DOTS_TTS_SIDECAR_SCRIPT
assert DOTS_TTS_SIDECAR_SCRIPT.name == "main.py"
assert DOTS_TTS_SIDECAR_SCRIPT.is_file()
# ── hardware + platform honesty ────────────────────────────────────────────
def test_gpu_compat_cuda_cpu_no_mps():
from engines.dots_tts import DotsTTSBackend
assert DotsTTSBackend.gpu_compat == ("cuda", "cpu")
def test_sample_rate_is_48k():
from engines.dots_tts import DotsTTSBackend
b = DotsTTSBackend()
assert b.sample_rate == 48000
assert b.supported_languages == ["multi"]
def test_windows_is_gated_off(monkeypatch):
"""Cross-platform rule: dots.tts upstream is Linux/macOS-only, so on
Windows is_available() must refuse cleanly (not advertise a dead engine)."""
monkeypatch.setattr(sys, "platform", "win32")
from engines.dots_tts import DotsTTSBackend
ok, msg = DotsTTSBackend.is_available()
assert ok is False
assert "Windows" in msg
assert "mps" not in msg.lower()
def test_is_available_not_installed_is_honest(monkeypatch):
"""On a supported OS without the venv, gate cleanly with an actionable
hint and never claim MPS."""
monkeypatch.setattr(sys, "platform", "linux")
monkeypatch.setattr(
"engines.dots_tts.bootstrap.is_dots_tts_installed",
lambda: False,
)
from engines.dots_tts import DotsTTSBackend
ok, msg = DotsTTSBackend.is_available()
assert ok is False
assert "OMNIVOICE_DOTS_TTS_DIR" in msg
assert "docs/engines/dots-tts.md" in msg # actionable pointer
# No-MPS honesty is enforced by gpu_compat; the message may disclaim MPS.
# ── generate() parent-side arbitration ─────────────────────────────────────
def test_clone_with_transcript_and_overrides(monkeypatch):
"""ref_audio+ref_text → continuation cloning; num_step/guidance forwarded."""
captured: dict = {}
def fake_super_generate(self, text, **kw):
import torch
captured["text"] = text
captured.update(kw)
return torch.zeros(1, 8)
from engines.dots_tts import DotsTTSBackend
# Patch the exact SubprocessBackend in this backend's MRO (not via a
# module-path string): survives the sys.modules['services.*'] reloads
# other tests perform, so super().generate() hits the fake instead of
# dispatching to the real class and trying to spawn a sidecar. (#498)
_sub = next(c for c in DotsTTSBackend.__mro__ if c.__name__ == "SubprocessBackend")
monkeypatch.setattr(_sub, "generate", fake_super_generate)
DotsTTSBackend().generate(
"speak this",
ref_audio="/tmp/ref.wav",
ref_text="the reference transcript",
language="en",
num_step=20,
guidance_scale=1.5,
)
assert captured["ref_audio"] == "/tmp/ref.wav"
assert captured["ref_text"] == "the reference transcript"
assert captured["language"] == "en"
assert captured["num_steps"] == 20
assert captured["guidance_scale"] == 1.5
def test_orphan_ref_text_dropped_and_dots_defaults(monkeypatch):
"""ref_text without ref_audio is dropped (upstream would raise), and the
dots-appropriate defaults (num_steps=10, guidance=1.2) apply."""
captured: dict = {}
def fake_super_generate(self, text, **kw):
import torch
captured.update(kw)
return torch.zeros(1, 8)
from engines.dots_tts import DotsTTSBackend
# Patch the exact SubprocessBackend in this backend's MRO (not via a
# module-path string): survives the sys.modules['services.*'] reloads
# other tests perform, so super().generate() hits the fake instead of
# dispatching to the real class and trying to spawn a sidecar. (#498)
_sub = next(c for c in DotsTTSBackend.__mro__ if c.__name__ == "SubprocessBackend")
monkeypatch.setattr(_sub, "generate", fake_super_generate)
DotsTTSBackend().generate("no reference", ref_text="orphan transcript")
assert "ref_text" not in captured
assert "ref_audio" not in captured
assert captured["num_steps"] == 10
assert captured["guidance_scale"] == 1.2
+66
View File
@@ -146,6 +146,72 @@ def test_transcribe_stream_surfaces_model_load_failure(tmp_path, monkeypatch):
assert "CUDA driver init failed: simulated" in body, body
def test_transcribe_stream_never_closes_without_terminal_event(tmp_path, monkeypatch):
"""Regression #516: an unanticipated exception INSIDE the stream body (one
that escapes the per-chunk handler, e.g. segmentation blowing up) must still
end the stream with a terminal `error` then `done` never a silent
disconnect (which the UI can only report as "stream dropped, likely ASR
failed", hiding the real cause)."""
import asyncio
import numpy as np
from api.routers import dub_core as dc
job_id = "t_bodycrash"
audio = tmp_path / "a.wav"
_make_wav(audio, seconds=1.0)
dc._dub_jobs[job_id] = {
"audio_path": str(audio), "vocals_path": None, "scene_cuts": [],
}
# Model + ASR backend load fine (preflight passes), so the failure happens
# mid-body where the terminal-event guard is the only safety net.
fake_model = MagicMock()
fake_model._asr_pipe = MagicMock()
async def _ok_model():
return fake_model
class _FakeASR:
id = "fake"
def transcribe(self, path, *, word_timestamps=True):
return {"chunks": [{"text": "hi", "timestamp": (0.0, 0.5)}],
"segments": [], "language": "en"}
def unload(self):
pass
monkeypatch.setattr(dc, "get_model", _ok_model)
monkeypatch.setattr(
"services.asr_backend.get_active_asr_backend",
lambda *a, **k: _FakeASR(),
)
# Make the post-chunk segmentation (outside the per-chunk try/except) blow
# up — the exact class of "unanticipated escape" the guard must catch.
def _boom_segment(*a, **k):
raise RuntimeError("segmentation exploded: simulated")
monkeypatch.setattr(dc, "segment_transcript", _boom_segment)
# Don't touch the GPU/TTS during the test.
monkeypatch.setattr(dc, "offload_tts_for_asr", lambda *a, **k: None)
async def _collect():
resp = await dc.dub_transcribe_stream(job_id)
parts = []
async for chunk in resp.body_iterator:
parts.append(chunk.decode() if isinstance(chunk, (bytes, bytearray)) else str(chunk))
return "".join(parts)
try:
body = asyncio.run(_collect())
finally:
dc._dub_jobs.pop(job_id, None)
# The stream must end with a terminal error followed by done.
assert "event: error" in body, body
assert "segmentation exploded: simulated" in body, body
err_idx = body.rfind("event: error")
done_idx = body.rfind("event: done")
assert done_idx > err_idx >= 0, f"error must precede the terminal done: {body}"
@pytest.mark.xfail(
reason="dub_core._transcribe was refactored to route through "
"services.asr_backend.get_active_asr_backend; the MagicMock fixture "
+84
View File
@@ -0,0 +1,84 @@
"""Issue #502 (partial) — the duration estimator must be Unicode-normalization
-form independent.
``RuleDurationEstimator`` weights combining marks (U+0300036F, category Mn) at
0.0. Decomposed (NFD) text a base letter followed by combining tone/diacritic
marks, the canonical on-disk form for Vietnamese and many diacritic scripts
therefore mis-allocates frames versus the precomposed (NFC) form of the *same*
text, producing rushed/garbled audio for the affected syllables.
The fix NFC-normalizes at the estimator's text entry point
(``calculate_total_weight``), so NFC and NFD inputs always yield the *same*
estimate. These tests fail before the fix (NFD diverges) and pass after.
"""
import unicodedata
import pytest
from omnivoice.utils.duration import RuleDurationEstimator
def _raw_weight(est, text):
"""Pre-fix weighting: sum per-char weights WITHOUT NFC normalization.
Calls the lru_cache-wrapped ``_get_char_weight`` via ``__wrapped__`` so the
fix's normalization step is bypassed — this reproduces the *old*
``calculate_total_weight`` behavior to prove the bug existed.
"""
return sum(est._get_char_weight.__wrapped__(est, c) for c in text)
# A single Korean Hangul syllable: precomposed (1 syllable block, weight 2.5)
# vs decomposed (lead/vowel/tail jamo). This is the clearest member of the
# normalization-divergence *class* — the bug is not Vietnamese-specific.
_KOREAN = "" # 한
# A Vietnamese phrase exercising stacked tone + vowel-quality marks — the
# issue's named script. Stays stable across forms once normalized.
_VIETNAMESE = "Tiếng Việt nặng hỏi ngã sắc huyền"
def test_combining_marks_weigh_zero():
"""Guards the precondition the bug rests on: combining marks contribute 0.0,
so NFD text that splits a syllable into base+marks loses that weight."""
est = RuleDurationEstimator()
for cp in (0x0300, 0x0301, 0x0302, 0x0303, 0x0309, 0x0323): # Vietnamese tone marks
assert est._get_char_weight(chr(cp)) == 0.0
def test_nfd_diverges_from_nfc_before_normalization():
"""Fail-before guard: the OLD (un-normalized) weighting gives NFD a
different weight than NFC, which is exactly the defect."""
est = RuleDurationEstimator()
nfc = unicodedata.normalize("NFC", _KOREAN)
nfd = unicodedata.normalize("NFD", _KOREAN)
assert nfc != nfd # the syllable really does have distinct forms
assert _raw_weight(est, nfc) != _raw_weight(est, nfd)
@pytest.mark.parametrize("sample", [_KOREAN, _VIETNAMESE, "한국어 문장입니다"])
def test_calculate_total_weight_is_normalization_independent(sample):
"""After the fix, NFC and NFD inputs weigh identically (the fix normalizes
to NFC first). Non-zero, so the estimate stays meaningful."""
est = RuleDurationEstimator()
nfc = unicodedata.normalize("NFC", sample)
nfd = unicodedata.normalize("NFD", sample)
w_nfc = est.calculate_total_weight(nfc)
w_nfd = est.calculate_total_weight(nfd)
assert w_nfc == w_nfd
assert w_nfc > 0
@pytest.mark.parametrize("sample", [_KOREAN, _VIETNAMESE])
def test_estimate_duration_is_normalization_independent(sample):
"""End-to-end: the public estimate is identical for NFC vs NFD input.
Before the fix the Hangul case diverges ~3x (NFD over-counts as jamo);
after the fix both forms produce the same, correct estimate."""
est = RuleDurationEstimator()
ref_text, ref_dur = "Hello, world.", 1.5
nfc = unicodedata.normalize("NFC", sample)
nfd = unicodedata.normalize("NFD", sample)
est_nfc = est.estimate_duration(nfc, ref_text, ref_dur)
est_nfd = est.estimate_duration(nfd, ref_text, ref_dur)
assert est_nfc == est_nfd
assert est_nfc > 0
+68
View File
@@ -0,0 +1,68 @@
"""ASR-robustness failure classification (#551 / #549).
The dub/transcribe "no segments" toast is only actionable if `classify()` names
the failure class so `build_failure()` can attach a hint. These assert the two
new taxonomy classes added for the ASR-robustness fix map to a non-empty hint.
"""
from core import failure
def test_classify_compute_type_unsupported():
# The exact CTranslate2 message on a GPU without efficient fp16 (#551).
reason = (
"Requested float16 compute type, but the target device or backend do "
"not support efficient float16 computation"
)
assert failure.classify(reason) == "COMPUTE_TYPE_UNSUPPORTED"
evt = failure.build_failure(reason, stage="transcribe", include_diagnostic=False)
assert evt["docs_topic"] == "COMPUTE_TYPE_UNSUPPORTED"
assert evt["hint"], "compute-type failure must carry an actionable hint"
def test_classify_transformers_import():
# The transformers ASR-pipeline import failure (#549).
assert failure.classify("Could not import module 'AutoFeatureExtractor'") == (
"TRANSFORMERS_IMPORT"
)
# Substring match on the bare class name too (case-insensitive).
assert failure.classify("AutoFeatureExtractor failed to load") == "TRANSFORMERS_IMPORT"
evt = failure.build_failure(
"Could not import module 'AutoFeatureExtractor'",
stage="transcribe",
include_diagnostic=False,
)
assert evt["hint"], "transformers-import failure must carry an actionable hint"
def test_classify_video_download_classes():
# #554: a non-downloadable link shape → actionable "paste a direct video URL".
assert failure.classify("Unsupported URL: https://www.douyin.com/discover") == (
"UNSUPPORTED_VIDEO_URL"
)
# #536: a transient mid-download drop → "just retry".
assert failure.classify("Unable to download video: [Errno 32] Broken pipe") == (
"VIDEO_DOWNLOAD_NETWORK"
)
assert failure.classify("Connection reset by peer") == "VIDEO_DOWNLOAD_NETWORK"
for cls, reason in (
("UNSUPPORTED_VIDEO_URL", "Unsupported URL: x"),
("VIDEO_DOWNLOAD_NETWORK", "Unable to download video: Broken pipe"),
):
evt = failure.build_failure(reason, stage="download", include_diagnostic=False)
assert evt["docs_topic"] == cls
assert evt["hint"], f"{cls} must carry an actionable hint"
def test_classify_broken_venv_encodings():
# The relocated/corrupted-venv stdlib-bootstrap failure → BROKEN_VENV (the
# Rust self-heal rebuilds it; this names the class for the toast).
assert failure.classify("ModuleNotFoundError: No module named 'encodings'") == (
"BROKEN_VENV"
)
# ...but an app-level import of an 'encodings'-prefixed package must NOT.
assert failure.classify("No module named 'encodings_helper'") == ""
def test_classify_generic_still_empty():
# A genuinely unknown reason must still classify to "" (no false hint).
assert failure.classify("some totally unrelated failure") == ""
+1 -1
View File
@@ -39,7 +39,7 @@ def _resolve(_voice_id):
def _stub_build_synth(*, fail_on=None):
"""Return a drop-in for `audiobook._build_synth` whose `synth` emits 0.1s of
silence per span. `fail_on(text)` raise, to exercise per-chapter faults."""
def _factory(default_voice=None):
def _factory(default_voice=None, language=None):
def synth(text, voice_id, speed=None):
if fail_on is not None and fail_on(text):
raise RuntimeError("stub synth deliberately failed")
+28
View File
@@ -5,10 +5,12 @@ EPUB zip in memory (no fixture file, no new dep).
"""
from __future__ import annotations
import asyncio
import io
import zipfile
import pytest
from fastapi import UploadFile
from services.longform_import import (
chapterize_plaintext,
@@ -16,6 +18,7 @@ from services.longform_import import (
pdf_to_chapter_script,
)
from services.audiobook import parse_audiobook_script
from api.routers.audiobook import audiobook_import
# ── PDF fixture builder ───────────────────────────────────────────────────
@@ -219,3 +222,28 @@ def test_pdf_too_many_pages_guard():
data = _make_pdf(["Chapter 1", "Hi."])
with pytest.raises(ValueError, match="too many pages"):
pdf_to_chapter_script(data, max_pages=0)
# ── import endpoint ────────────────────────────────────────────────────────
# Calls the handler directly (no TestClient → no main+torch import), mirroring
# tests/test_audiobook_cover.py.
def _upload(name: str, data: bytes) -> UploadFile:
return UploadFile(io.BytesIO(data), filename=name)
def test_import_endpoint_returns_chapter_count():
# Regression for #543: parsing succeeded but the endpoint then read
# plan.chapter_count, which didn't exist → 500 AttributeError. Covers the
# whole class — every import format hits this same return path.
pdf = _make_pdf(["Chapter 1", "Once upon a time.", "Chapter 2", "The end."])
for name, data in [
("book.pdf", pdf),
("book.md", b"# One\nhello\n\n# Two\nworld"),
("book.txt", b"just a flat blob of narration with no headings"),
]:
res = asyncio.run(audiobook_import(_upload(name, data)))
assert isinstance(res["chapters"], int) and res["chapters"] >= 1
assert res["text"].strip()
# The two-chapter inputs parse to exactly two chapters.
assert asyncio.run(audiobook_import(_upload("book.pdf", pdf)))["chapters"] == 2
+59
View File
@@ -0,0 +1,59 @@
"""Migration 0006 heals voice_profiles.instruct poisoned with the
"[object Object]" sentinel (#550 #545 #542 #537 #530 #525). Drives alembic on a
temp SQLite DB, mirroring tests/test_profile_consent.py's migration harness."""
import os
import sqlite3
# A pre-0003 voice_profiles — the shape the alembic chain expects to upgrade
# (matches tests/test_profile_consent.py::_PRE_CONSENT_PROFILES).
_BASE_PROFILES = """
CREATE TABLE voice_profiles (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
ref_audio_path TEXT,
ref_text TEXT DEFAULT '',
instruct TEXT DEFAULT '',
language TEXT DEFAULT 'Auto',
locked_audio_path TEXT DEFAULT '',
seed INTEGER DEFAULT NULL,
is_locked INTEGER DEFAULT 0,
personality TEXT DEFAULT '',
description TEXT DEFAULT '',
is_demo INTEGER DEFAULT 0,
created_at REAL
);
"""
def _run_alembic_upgrade(db_path: str, target: str = "head") -> None:
from alembic import command
from alembic.config import Config
root = os.path.abspath(os.path.dirname(__file__))
while root and root != "/" and not os.path.isfile(os.path.join(root, "alembic.ini")):
root = os.path.dirname(root)
assert os.path.isfile(os.path.join(root, "alembic.ini")), "alembic.ini not found"
cfg = Config(os.path.join(root, "alembic.ini"))
cfg.set_main_option("sqlalchemy.url", f"sqlite:///{db_path}")
command.upgrade(cfg, target)
def test_migration_0006_heals_object_object_instruct(tmp_path):
db = tmp_path / "poisoned.db"
with sqlite3.connect(str(db)) as conn:
conn.executescript(_BASE_PROFILES)
# a poisoned row (the #550 bug) + a healthy row that must be untouched
conn.execute(
"INSERT INTO voice_profiles(id, name, instruct) VALUES ('vp-bad', 'Bad', '[object Object]')"
)
conn.execute(
"INSERT INTO voice_profiles(id, name, instruct) VALUES ('vp-ok', 'Ok', 'male, high pitch')"
)
conn.commit()
_run_alembic_upgrade(str(db))
with sqlite3.connect(str(db)) as conn:
rows = dict(conn.execute("SELECT id, instruct FROM voice_profiles").fetchall())
assert rows["vp-bad"] == "", "0006 must clear the [object Object] sentinel"
assert rows["vp-ok"] == "male, high pitch", "0006 must not touch healthy instruct values"
+161
View File
@@ -0,0 +1,161 @@
"""Tests for the MOSS-TTS-v1.5 engine (issue #498).
MOSS-TTS-v1.5 runs in a dedicated subprocess venv (transformers==5.0.0),
isolated from the parent's transformers>=5.3 — so these tests never import
MOSS itself. They exercise the parent-side wiring that ships in the default
install: registry resolution, subprocess isolation, hardware honesty, the
not-installed gate, and the generate() kwarg arbitration. No network, no
optional deps, no subprocess spawn.
"""
from __future__ import annotations
import importlib
import sys
import pytest
# ── registry wiring ────────────────────────────────────────────────────────
def test_registry_contains_moss_tts_v15():
"""``_REGISTRY["moss-tts-v15"]`` resolves to ``MossTTSV15Backend`` and is
flagged subprocess-isolated."""
from services.tts_backend import _REGISTRY, get_backend_class
assert "moss-tts-v15" in _REGISTRY, (
"_REGISTRY is missing 'moss-tts-v15'; check _LAZY_REGISTRY in "
"services/tts_backend.py"
)
cls = _REGISTRY["moss-tts-v15"]
assert cls.__name__ == "MossTTSV15Backend"
assert get_backend_class("moss-tts-v15") is cls
# Duck-typed marker survives sys.modules['services.*'] purges (the same
# reason list_backends/test_supertonic3 rely on it instead of issubclass).
assert getattr(cls, "_is_subprocess_isolated", False), (
"MossTTSV15Backend should be subprocess-isolated"
)
for name in ("is_available", "generate", "sample_rate", "supported_languages"):
assert hasattr(cls, name), f"MossTTSV15Backend missing {name!r}"
def test_pep562_lazy_import():
"""The lazy registry resolves the class on first access."""
mod = importlib.import_module("services.tts_backend")
cls = mod._REGISTRY["moss-tts-v15"]
assert cls.__name__ == "MossTTSV15Backend"
def test_install_hint_present():
"""A Settings tooltip points users at the clone + env var."""
from services.tts_backend import _INSTALL_HINTS
hint = _INSTALL_HINTS.get("moss-tts-v15", "")
assert "OMNIVOICE_MOSS_TTS_V15_DIR" in hint
assert "OpenMOSS" in hint
def test_sidecar_script_ships():
"""The sidecar entrypoint ships with the install (the parent spawns it)."""
from engines.moss_tts_v15.bootstrap import MOSS_TTS_V15_SIDECAR_SCRIPT
assert MOSS_TTS_V15_SIDECAR_SCRIPT.name == "main.py"
assert MOSS_TTS_V15_SIDECAR_SCRIPT.is_file()
# ── hardware honesty (cross-platform rule) ─────────────────────────────────
def test_gpu_compat_cuda_cpu_no_mps():
"""MPS is undocumented/untested upstream — we must not claim it."""
from engines.moss_tts_v15 import MossTTSV15Backend
assert MossTTSV15Backend.gpu_compat == ("cuda", "cpu"), (
f"expected ('cuda', 'cpu'), got {MossTTSV15Backend.gpu_compat!r}"
)
def test_is_available_not_installed_is_honest(monkeypatch):
"""When the venv isn't present, is_available() gates cleanly with an
actionable hint and never claims MPS."""
monkeypatch.setattr(
"engines.moss_tts_v15.bootstrap.is_moss_tts_v15_installed",
lambda: False,
)
from engines.moss_tts_v15 import MossTTSV15Backend
ok, msg = MossTTSV15Backend.is_available()
assert ok is False
assert "OMNIVOICE_MOSS_TTS_V15_DIR" in msg
assert "docs/engines/moss-tts-v15.md" in msg # actionable pointer
# Honesty on hardware is enforced by gpu_compat (no 'mps' entry); the
# message is free to *disclaim* MPS ("no MPS"), which is the opposite of
# claiming it.
def test_sample_rate_and_languages():
from engines.moss_tts_v15 import MossTTSV15Backend
b = MossTTSV15Backend()
assert b.sample_rate == 24000
assert b.supported_languages == ["multi"]
# ── generate() parent-side arbitration ─────────────────────────────────────
def test_duration_maps_to_tokens(monkeypatch):
"""``duration`` (seconds) → ``tokens`` at 12.5 tokens/sec; engine-specific
kwargs are forwarded and the generic ``num_step`` is dropped."""
captured: dict = {}
def fake_super_generate(self, text, **kw):
import torch
captured["text"] = text
captured.update(kw)
return torch.zeros(1, 8)
from engines.moss_tts_v15 import MossTTSV15Backend
# Patch the exact SubprocessBackend in this backend's MRO (not via a
# module-path string): survives the sys.modules['services.*'] reloads
# other tests perform, so super().generate() hits the fake instead of
# dispatching to the real class and trying to spawn a sidecar. (#498)
_sub = next(c for c in MossTTSV15Backend.__mro__ if c.__name__ == "SubprocessBackend")
monkeypatch.setattr(_sub, "generate", fake_super_generate)
MossTTSV15Backend().generate(
"hello world",
ref_audio="/tmp/spk.wav",
language="fr",
duration=26.0,
num_step=16, # generic kwarg MOSS doesn't use
max_new_tokens=2048,
)
assert captured["text"] == "hello world"
assert captured["ref_audio"] == "/tmp/spk.wav"
assert captured["language"] == "fr"
assert captured["tokens"] == int(26.0 * 12.5) # 325
assert captured["max_new_tokens"] == 2048
assert "num_step" not in captured # not part of MOSS's surface
def test_generate_without_ref_audio_omits_reference(monkeypatch):
"""No ref_audio → plain TTS: neither ref_audio nor tokens leak in."""
captured: dict = {}
def fake_super_generate(self, text, **kw):
import torch
captured.update(kw)
return torch.zeros(1, 8)
from engines.moss_tts_v15 import MossTTSV15Backend
# Patch the exact SubprocessBackend in this backend's MRO (not via a
# module-path string): survives the sys.modules['services.*'] reloads
# other tests perform, so super().generate() hits the fake instead of
# dispatching to the real class and trying to spawn a sidecar. (#498)
_sub = next(c for c in MossTTSV15Backend.__mro__ if c.__name__ == "SubprocessBackend")
monkeypatch.setattr(_sub, "generate", fake_super_generate)
MossTTSV15Backend().generate("just text")
assert "ref_audio" not in captured
assert "tokens" not in captured
+200
View File
@@ -0,0 +1,200 @@
"""Non-English correctness: a voice profile's stored language must drive
generation, and the longform path must not hardcode language=None.
* #533 — POST /generate with a profile_id but no request language must thread
the *profile's* language into the engine (German archetype → German output,
not English). An explicit non-Auto request language still wins.
* #505 (B2) — the audiobook/longform synth callable hardcoded language=None,
so each chunk re-autodetected and a non-English clone drifted. The synth
must now carry the resolved language.
The engine layer is stubbed (no real model loads), matching
``tests/test_generate_engine.py``.
"""
import importlib
import os
import uuid
import pytest
import torch
os.environ.setdefault("OMNIVOICE_MODEL", "test")
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
def _tts_mod():
return importlib.import_module("services.tts_backend")
def _make_fake_engine(engine_id="fake-lang-engine", gpu_compat=("cpu",)):
_compat = gpu_compat
class _FakeEngine(_tts_mod().TTSBackend):
id = engine_id
display_name = "Fake Lang Engine (test)"
applies_own_mastering = False
gpu_compat = _compat
calls: list = []
@property
def sample_rate(self) -> int:
return 24000
@property
def supported_languages(self) -> list[str]:
return ["multi"]
@classmethod
def is_available(cls):
return True, "ready"
def generate(self, text, **kw) -> torch.Tensor:
type(self).calls.append((text, kw))
return torch.zeros(1, 24000)
return _FakeEngine
@pytest.fixture()
def client():
from fastapi.testclient import TestClient
from main import app
return TestClient(app, client=("127.0.0.1", 50000))
@pytest.fixture()
def _init_db():
from core.db import init_db
init_db()
@pytest.fixture()
def german_profile(_init_db):
"""Insert a clone profile whose stored language is 'German', then remove it.
No ref_audio_path on disk the resolver leaves ref_audio None, so the
engine still runs (the test asserts on the threaded `language`, not audio)."""
from core.db import db_conn
pid = f"vp-de-{uuid.uuid4().hex[:8]}"
with db_conn() as conn:
conn.execute(
"INSERT INTO voice_profiles (id, name, language, kind, created_at) "
"VALUES (?,?,?,?,?)",
(pid, "German Narrator", "German", "clone", 0.0),
)
yield pid
with db_conn() as conn:
# A successful /generate inserts a generation_history row FK-referencing
# the profile — clear dependents before the profile itself.
conn.execute("DELETE FROM generation_history WHERE profile_id=?", (pid,))
conn.execute("DELETE FROM voice_profiles WHERE id=?", (pid,))
# ── #533: profile language drives /generate ──────────────────────────────────
def test_generate_uses_profile_language_when_request_unset(client, monkeypatch, german_profile):
"""Request omits language → the profile's stored 'German' reaches the engine
(not None). Before the fix generation.py never read row['language']."""
fake = _make_fake_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
res = client.post("/generate", data={
"text": "Guten Tag", "profile_id": german_profile, "engine": fake.id,
})
assert res.status_code == 200, res.text
assert len(fake.calls) == 1
_, kw = fake.calls[0]
# The engine receives the profile's language string; the model's
# _resolve_language maps 'German' → 'de'. Critically: NOT None.
assert kw.get("language") == "German"
def test_generate_explicit_language_overrides_profile(client, monkeypatch, german_profile):
"""An explicit non-Auto request language wins over the profile language."""
fake = _make_fake_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
res = client.post("/generate", data={
"text": "Hello", "profile_id": german_profile, "engine": fake.id,
"language": "en",
})
assert res.status_code == 200, res.text
assert len(fake.calls) == 1
_, kw = fake.calls[0]
assert kw.get("language") == "en" # request wins, not 'German'
def test_generate_explicit_auto_falls_back_to_profile(client, monkeypatch, german_profile):
"""Explicit 'Auto' is request-unset → profile language still wins."""
fake = _make_fake_engine()
monkeypatch.setitem(_tts_mod()._REGISTRY, fake.id, fake)
fake.calls.clear()
res = client.post("/generate", data={
"text": "Guten Tag", "profile_id": german_profile, "engine": fake.id,
"language": "Auto",
})
assert res.status_code == 200, res.text
_, kw = fake.calls[0]
assert kw.get("language") == "German"
# ── #505 (B2): longform synth carries the language, never hardcoded None ──────
def test_resolve_default_language_request_wins():
from api.routers.audiobook import _resolve_default_language
assert _resolve_default_language("ja", None) == "ja"
assert _resolve_default_language("ja", "anything") == "ja"
def test_resolve_default_language_falls_back_to_profile(german_profile):
from api.routers.audiobook import _resolve_default_language
# No request language → the profile's stored language drives it.
assert _resolve_default_language(None, german_profile) == "German"
assert _resolve_default_language("Auto", german_profile) == "German"
def test_resolve_default_language_none_when_nothing(_init_db):
from api.routers.audiobook import _resolve_default_language
assert _resolve_default_language(None, None) is None
assert _resolve_default_language("Auto", None) is None
def test_build_synth_threads_language_into_generic_engine(monkeypatch):
"""The generic-engine synth callable must pass the resolved language to
backend.generate before the fix it hardcoded language=None (#505 B2)."""
import api.routers.audiobook as ab
captured = {}
class _Backend:
sample_rate = 24000
def generate(self, text, **kw):
captured["language"] = kw.get("language")
return torch.zeros(1, 24000)
monkeypatch.setattr(ab, "_resolve_voice",
lambda pid: {"ref_audio": None, "ref_text": None,
"instruct": None, "seed": None})
monkeypatch.setattr("services.tts_backend.active_backend_id", lambda: "fake")
# _Backend is not OmniVoiceBackend → _build_synth takes the generic path.
monkeypatch.setattr("services.tts_backend.get_backend_class", lambda eid: _Backend)
info = ab._build_synth(default_voice="vp1", language="ja")
assert info["mode"] == "generic"
info["synth"]("こんにちは", "vp1")
assert captured["language"] == "ja" # NOT None
+45
View File
@@ -0,0 +1,45 @@
"""The engine Install chip must target the interpreter the backend runs under.
#529/#527: the desktop spawns `<venv>/bin/python -m uvicorn` WITHOUT exporting
VIRTUAL_ENV, so bare `uv pip install` finds no venv and 500s with "No virtual
environment found". run_pip must pass `--python sys.executable`.
"""
import asyncio
import sys
from services import translation_engines as te
class _FakeProc:
returncode = 0
async def communicate(self):
return (b"ok", b"")
def _run_capturing(monkeypatch, args):
"""Force the uv branch + capture the spawned argv; return (rc, argv)."""
captured = {}
monkeypatch.setattr(te.shutil, "which", lambda name: "/usr/bin/uv" if name == "uv" else None)
async def fake_exec(*argv, **kwargs):
captured["argv"] = list(argv)
return _FakeProc()
monkeypatch.setattr(te.asyncio, "create_subprocess_exec", fake_exec)
rc, _out = asyncio.run(te.run_pip(args))
return rc, captured.get("argv", [])
def test_run_pip_pins_uv_install_to_sys_executable(monkeypatch):
rc, argv = _run_capturing(monkeypatch, ["install", "deep_translator"])
assert rc == 0
assert argv[:3] == ["uv", "pip", "install"], argv
assert "--python" in argv, argv
assert argv[argv.index("--python") + 1] == sys.executable
def test_run_pip_pins_uv_uninstall_to_sys_executable(monkeypatch):
_rc, argv = _run_capturing(monkeypatch, ["uninstall", "deep_translator"])
assert "--python" in argv, argv
assert argv[argv.index("--python") + 1] == sys.executable