Compare commits

..
320 Commits
Author SHA1 Message Date
29b6f30f2b release: freeze v0.3.15 — version bump, lockfiles, changelog (#1041)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:59:59 +05:30
72e979f1d9 fix(tts): model-load time stops eating the generate timeout budget (#1039)
* fix(tts): model-load time stops eating the generate timeout budget (#1033, #1037)

The generate guard (OMNIVOICE_GENERATE_TIMEOUT_S, 300s) wrapped the
adapter's lazy _ensure_loaded() — weight download included — together
with the synthesis. A cold first request burned the whole window on
the download and died with the VRAM-guidance 503; #1014's T4
verification measured it (0% GPU util for the full 300s), and #1033 +
#1037 match the signature.

New public TTSBackend.ensure_ready() (dispatches to the adapter's
_ensure_loaded when present) runs FIRST under the model-load budget
(OMNIVOICE_MODEL_LOAD_TIMEOUT, 1200s) in both /generate's adapter path
and /v1/audio/speech — the same load/generate split get_model()
already gave the native engine. Warm engines no-op. A load exceeding
its own budget 503s with load-specific text pointing at Settings →
Models, never the misleading 'too heavy for compute' guidance.

Tests: end-to-end class test (load slower than a tiny generate budget
but inside the load budget → succeeds; fail-before verified), the
stalled-load error path, and the base-hook dispatch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* changelog entry for the load-budget split (#1033, #1037)

* catch the builtin TimeoutError base — reload-proof class identity (CI-only miss)

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:35:17 +05:30
6c6207a132 docs: link the community Colab notebook (#1038) (#1040)
@shakib30 built and tested a working Colab notebook for the project
and offered it upstream. Linking it from the README (community-
maintained, credited) makes the no-local-GPU path discoverable without
taking on notebook maintenance in-repo.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 04:19:54 +05:30
0effe485f4 fix(studio): restore Clear History, stoppable auto-play preview, + cached ref transcripts (#1032) (#1036)
Three-part fix for the v0.3.5-comparison report:

1. Perf: since v0.3.6 (#308), a clone reference without a stored
   transcript triggered a FULL ASR model load + transcribe on every
   /generate — get_active_asr_backend() builds a fresh whisper backend
   per call. Measured live: 92.7s wall vs 14.9s of actual TTS. Now the
   first auto-transcript is persisted onto the (unlocked, clone-kind)
   profile row, and transcribe_reference caches results by audio
   content hash (bounded LRU, no model/VRAM held), so the cost is paid
   once per clip, not per request. User-typed transcripts are never
   overwritten; locked/design profiles are excluded from the persist.

2. Clear History: the workspace UX overhaul (#374) moved history into
   the right-side WorkspaceHistory panels and dropped the old Sidebar's
   clear-all control (the Sidebar is now hidden in every mode). Both
   the Voice and Dub panels get a scoped Clear History button wired to
   the existing DELETE /history and /dub/history endpoints, with the
   same confirm dialog the Sidebar used.

3. Auto-play: the finished-render playback (playBlobAudio) has no
   on-screen player and the only stop lived in the Voice ActionBar's
   CTA morph — unstoppable from the Dub workspace, profile pages, or
   after navigating away. A global PlaybackStopPill now appears for any
   'output' playback on every page. The existing Settings → Appearance
   "Auto-play preview" pref (#667) now also gates the generate path,
   as its label always promised (default ON — no behavior change).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:43:33 +05:30
32407bc781 fix(api): /v1/audio/speech honors num_step + guidance_scale instead of silently dropping them (#1014) (#1035)
A contributor's measured Tesla T4 verification (PR #1014) caught that
POST /v1/audio/speech accepted num_step/guidance_scale in the JSON
body with a 200 OK and discarded both (pydantic's default
extra=ignore) — API callers could never reach the model's documented
quality preset (num_step=32) through the OpenAI-compatible surface,
while the native /generate exposes both as form fields.

Both are now declared as validated optional extensions (num_step 1-128,
guidance_scale 0-20) and passed through to the engine's generate()
kwargs — omitted means absent (engines that don't accept the kwargs
never see a stray None), exactly like the existing duration/seed
extensions.

Tests: passthrough reaches the engine kwargs; omitted stays absent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:49:25 +05:30
df870e9ed3 docs(agents): add verified Tesla T4 (16GB) inference notes (#1014)
* docs: add AGENTS.md with verified Tesla T4 (16GB) inference notes

Documents two things found while verifying inference on a real T4:
1. Cold-cache first /v1/audio/speech call can hit the 300s
   OMNIVOICE_GENERATE_TIMEOUT_S because the checkpoint download happens
   inside that budget — workaround via existing POST /models/install or
   raising the timeout, no code change needed.
2. The OpenAI-compatible endpoint silently ignores num_step/guidance_scale
   (schema doesn't declare them) — use native /generate for those.

Also documents the T4 acceleration checklist (dtype/attention/int8/CUDA
graphs) and measured VRAM (peak 2.05GB). No code changes.

* fix(docs): make /models/install workaround command actually executable

Addresses Greptile review: the instruction omitted the required
repo_id body field (InstallModelRequest rejects an empty body).

* fix(docs): correct port in /models/install example (3900, not 8000)

The app serves on port 3900 (confirmed: /health returns 200 there,
connection refused on 8000). Verified the exact corrected curl command
returns 200 {"status":"install_started",...}.

* move T4 notes to docs/hardware-notes-tesla-t4.md — AGENTS.md is the auto-loaded agent-instructions filename

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:30:05 +05:30
stronghamjjiandstronghamjji 323892b36c fix(clone): bound the ref-text re-transcribe like every other ASR dispatch (#730) (#1031)
The re-transcribe added for the (ref_audio, ref_text) mismatch fix is
dispatched with a bare run_in_executor(_gpu_pool, ...), and refine_ref_text
calls asr_backend.transcribe() directly. Its try/except catches a raised
error but not a *hang* — a wedged whisperx/CTranslate2 transcribe (#730)
holds the GPU-pool worker forever. On a <=10 GB card the pool is 1 worker,
so that starves every later GPU job into the misleading "can't reach the
local backend", and there's no ping on the await so the EventSource drops.

Route both refine dispatches (per-speaker and per-segment) through the same
run_transcribe_guarded the rest of dub_core.py already uses (the chunk loop
and the whole-file "Dub" transcribe). On timeout it resets the pool and
raises ASRTimeoutError; keep the original clones, matching refine_ref_text's
own "failure is a strict no-op" fallback.

Adds a repro test: refine_ref_texts dispatched raw is unbounded on a hang;
through the guard it times out and falls back to the original ref_text.

Co-authored-by: stronghamjji <289942360+stronghamjji@users.noreply.github.com>
2026-07-09 17:10:23 +05:30
36ee06c7cc feat(skills): installable Agent Skills — npx skills add debpalash/omnivoice-studio (#1034)
Two skills in the standard skills/<name>/SKILL.md layout (vercel-labs/
skills CLI; listed on skills.sh via install telemetry):

- omnivoice — teaches any agent (Claude Code, Cursor, Codex, …) to
  speak and transcribe through the user's LOCAL install via the
  OpenAI-compatible API at localhost:3900: health preflight, TTS with
  cloned-voice-profile discovery via /v1/audio/voices, STT with
  srt/vtt subtitle formats, and the local-first rule (never silently
  fall back to a cloud API).
- oss-maintainer — the maintainer methodology this repo is actually
  run with, distilled from real sessions: absorbed-or-declined queue
  discipline, check-the-PR-queue-before-implementing, root-cause →
  fix-the-class → regression-test, structural merge gates with
  flaky-vs-real judgment, the release protocol, and
  thank-contributors-specifically.

Every endpoint/flag in the omnivoice skill verified against
backend/api/routers/openai_compat.py and the README's API section.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:06:14 +05:30
9bfd8f9a13 fix(update): app updates stop uninstalling user-added engines — drift sync goes --inexact (#1029) (#1030)
Every app update whose uv.lock changed ran `uv sync --frozen` to
reconcile the venv (#307 drift path) — and uv sync's exact mode
UNINSTALLS every package not in the lockfile. That silently deleted
user-pip-installed optional engines (voxcpm, kittentts — packages the
app's own Settings → Engines hints tell users to install into this
venv) on every single update. Reported as "VoxCPM2 is automatically
uninstalled after updating Studio."

Fix: the routine drift sync now carries --inexact — locked deps are
still installed/upgraded exactly per the lockfile, but extras the user
added on purpose are left alone. Deliberate asymmetry: the venv-REPAIR
sync stays exact, because repair runs when the venv is broken and a
user-installed extra is a plausible cause — healing must restore the
known-good locked state. First-run syncs are untouched (a fresh venv
has no extras; exact == inexact there).

Both sync arg sets are now named constants with contract tests pinning
the asymmetry, so neither side can silently regress.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 10:06:23 +05:30
cb6c72b409 docs(faq): honest ElevenLabs comparison — where each wins, and why dub quality varies (community question) (#1028)
Asked directly on Discord ('how it compares to something like 11 labs
in quality?'). The old answer ('yes, comparable for most use cases')
oversold — the honest version names where ElevenLabs still wins
(out-of-the-box English polish/consistency) and where OmniVoice is
genuinely competitive (cloning from clean references, 646 languages,
structural advantages), plus the dubbing-specific truth another
same-day report surfaced: a dub is a chain, and incoherent output
usually traces to transcription quality on the user's source audio —
with the check-the-original-text-first debugging step that actually
helps.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 09:13:11 +05:30
a721c1fdcf release: freeze v0.3.14 — version bump, lockfiles, changelog (#1027)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 09:12:24 +05:30
80f10289fe feat(asr): ASR engines get the same Settings picker TTS has (env var still wins) (#1026)
Settings → Engines now stacks one pinned Engine Compatibility Matrix per
family (TTS, ASR, LLM) instead of a single TTS-titled table with the other
families tucked behind a low-discoverability tab. The backend select/prefs
path (family="asr" → prefs.asr_backend, env > prefs > auto-detect) already
worked but was unexercised and undocumented — it's now locked by API and
resolution-order tests, and README + the openai-compat-asr doc stop
promising a picker that didn't exist / denying one that now does.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 04:43:16 +05:30
a0ad314736 docs: CLAUDE.md refresh — de-rot versions, compress shipped stack research, replace the dead GSD gate (#1025)
Three classes of staleness that actively misled agent sessions:

- The Project section hardcoded "latest stable v0.3.5 / main at v0.3.6"
  — eight releases behind. Now points at the sources of truth
  (frontend/package.json, the Releases page) and documents the current
  AUTO_VERSION_BUMP-off holding behavior instead of a version literal
  that rots every release.
- ~165 lines of May-2026 stack research for five capabilities that have
  ALL since shipped (HF-token panel, prefilled-URL bug reporting, uv
  mirror fallback, Supertonic-3, in-repo docs). Compressed to the
  durable don'ts it established (no telemetry endpoints, no app-side
  GitHub tokens, no setx, no MkDocs, no hf_transfer) plus a pointer to
  prefer what's already pinned.
- The GSD Workflow Enforcement gate referenced /gsd-quick//gsd-debug/
  /gsd-execute-phase skills that exist nowhere in this environment; the
  owner explicitly chose direct edits over restoring them (2026-07-08).
  It cost a real mid-task detour when a subagent correctly refused to
  work under an unsatisfiable rule. Replaced with the owner decision
  and the working conventions that actually bind (merge gating,
  check-the-PR-queue-first).

244 → 83 lines. GSD section markers preserved so the generating tool
can still find its blocks.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 04:25:14 +05:30
1808a373a1 fix(linux): AppRun workaround detection reads the BUNDLED WebKitGTK version, not the host's (#961 follow-up) (#1024)
The launcher decided whether to export WEBKIT_DISABLE_COMPOSITING_MODE
by asking the host's pkg-config — but LD_LIBRARY_PATH makes the
BUNDLED libwebkit2gtk the one that actually runs, so on any machine
where the two diverge the detection read the wrong number. This was
the second bug identified during #961's investigation (the reporter
built from source, so their dev packages answered pkg-config with a
healthy version while the shipped bundle ran an older lib) and was
explicitly deferred in #1007 as not-safely-fixable at runtime.

The fix makes it knowable by construction instead: inject-apprun.sh
runs at bundle time ON the build host whose libwebkit2gtk gets
bundled, so it stamps that version into .bundled-webkitgtk-version
inside the AppDir. AppRun reads the stamp first and only falls back to
host pkg-config for bundles predating it. Empty/unreadable stamp fails
safe (workaround on), same philosophy as the missing-pkg-config path.

Tests: 3 new cases in AppRun.test.sh — marker-beats-host in both
directions (broken-marker/healthy-host and the #961 inversion,
healthy-marker/broken-host) plus empty-marker fail-safe. Also wires
AppRun.test.sh into pytest (tests/test_apprun_launcher.py) — it was
previously run by NO CI job, so the launcher could regress silently.

Also documents Windows install-to-another-drive behavior in
docs/install/windows.md (#938): local drives work via the wizard's
directory picker, mapped network drives are a Windows Installer
limitation, and the data directory moves independently of the app.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 03:53:46 +05:30
67789fb31c release: freeze v0.3.13 — version bump, lockfiles, changelog (#1023)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:52:25 +05:30
da4bef8e42 docs: correct troubleshooting §16 — mic bug was a missing entitlement, not an upstream limitation; changelog for #1016/#1020/#1021 (#1022)
troubleshooting.md §16 claimed the macOS microphone-permission bug was
an unresolved upstream Tauri/wry limitation with no available fix.
That was wrong: @MahdiHedhli read the wry/tauri sources more carefully
and found the real cause — Tauri's Hardened Runtime default blocks mic
hardware access without com.apple.security.device.audio-input in the
bundle's entitlements, which also explains why TCC never listed the
app. Their fix (#1016) is merged; §16 now documents the real mechanism,
credits the correction, and keeps the record-elsewhere workaround for
users on ≤0.3.12 builds.

Also brings CHANGELOG [Unreleased] current for the three merges that
lacked entries: #1016 (mic fix), #1020 (shutdown wait 3s→20s), #1021
(CI flaky-trio root cause + guard).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:27:00 +05:30
0076d0067e test(ci): root-cause and neutralize the flaky trio — a leaked torch fp16 default dtype (#1021)
test_effects_chain / test_generation_audio_guard / test_persona_bundle
failed intermittently on CI (never locally) with identical signatures
across three unrelated PRs today (#1002, #1019, #1016) — costing a
full CI cycle per occurrence and repeatedly muddying merge decisions.

Root cause, confirmed by local reproduction: a leaked
torch.set_default_dtype(torch.float16) from some earlier test in the
CI-Linux ordering. The smoking gun was test_generation_audio_guard's
observed 0.0999755859375 — exactly float16(0.1), i.e.
torch.tensor([0.1, …]) built under a leaked fp16 default. Reproducing
with a simulated polluter locally produced the trio's exact failures:
Pedalboard refuses fp16 audio outright ("only supports 32-bit and
64-bit floating point") and silently returns unmodified audio for
every preset, so test_effects_chain's preset outputs compare
identical; and the fp16 tensor value breaks the sanitize approx-check.

Fix: an autouse conftest guard (same philosophy as the existing
LLM-state isolation guard, #878) that checks torch's default dtype
after every test, resets any leak to float32, and emits a UserWarning
naming the offending test's nodeid — so the actual CI-only polluter
identifies itself in the next CI log instead of being chased blind.
Regression test: a deliberate-leak pair proving reset-between-tests.

Fail-before/pass-after verified: with the guard stashed, a simulated
polluter + the trio reproduced 2/3 failures locally with the exact CI
signatures; with the guard active, 73/73 pass and the warning names
the polluter.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:19:07 +05:30
Mahdi Hedhli faf34348c8 fix(macos): add microphone/camera entitlements so TCC ever sees a request (#1016)
Root cause of #1013 (macOS: "Microphone access denied" but OmniVoice never
appears in Privacy & Security → Microphone to enable it):

Tauri's macOS bundle config defaults `hardenedRuntime` to true, and Hardened
Runtime blocks camera/microphone hardware access unless the matching
entitlement is present — regardless of Info.plist's NSMicrophoneUsageDescription
(that only supplies the *prompt text*, it isn't itself the grant) and
regardless of wry's own WKUIDelegate already granting the request at the
WebKit/JS layer (WryWebViewUIDelegate::request_media_capture_permission
unconditionally calls WKPermissionDecision::Grant — confirmed by reading
wry 0.55.1's source; that part was never the problem). With Hardened Runtime
on and zero entitlements, TCC never registers a request at all, which is
exactly the reported symptom: nothing to enable because the OS never saw a
legitimately-entitled process ask. This also explains the workaround in
#1013 and its comments (launching the raw binary from Terminal works, but
as Terminal's identity, not the app's) — Terminal is a properly entitled,
hardened-runtime process; the ad-hoc/unentitled app binary isn't.

Adds src-tauri/entitlements.plist (com.apple.security.device.audio-input,
plus com.apple.security.device.camera matching the forward-looking
NSCameraUsageDescription already in Info.plist) and wires it in via
tauri.conf.json's bundle.macOS.entitlements. Also corrects the stale
"nothing to do here" module comment in lib.rs that documented the
incomplete assumption this bug falsified.

Verified: built a debug .app (`tauri build --debug --bundles app`) and
diffed `codesign -dv --entitlements -` before/after this change — the
entitlements dictionary goes from absent to containing exactly the two
keys added here, alongside the runtime (Hardened Runtime) flag that was
already on. `cargo test` — 60 passed, 0 failed.
2026-07-09 02:15:28 +05:30
69ce697ee5 fix(backend): shutdown wait bound 3s→20s — post-merge review finding on #1002; absorb #1015's design-path test (#1020)
Greptile's review of the merged #1002 flagged a real residual gap: a
cold transformers import alone can exceed the 3s shutdown wait, and
cancelling the asyncio task doesn't stop the underlying OS thread —
so quitting during an unusually slow preload could still let shutdown
report "done" while that thread was alive, the exact #1000 class with
lower odds. Python cannot forcibly kill a running thread, so no finite
bound eliminates this outright; 20s shrinks the window from "any
preload" to "an unusually slow cold-import," the practical ceiling
before a long shutdown becomes its own complaint. New source-level
contract test pins the production bound at ≥15s so a future edit
can't quietly shrink it back without deliberate consideration.

Also absorbs the one test case from community PR #1015 (superseded by
the earlier-merged #1017, which duplicated it — my fault for not
checking the PR queue) that the merged version lacked: the
design/instruct path with no ref kwargs at all stays untouched by the
ref_text forwarding fix.

Co-authored-by: mergetest <test@local>
Co-authored-by: MahdiHedhli <noreply@github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:10:06 +05:30
93a3cb260a fix(ui): dub editor play button no longer sticks disabled; remove donate heart from nav rail (#1019)
WaveformTimeline's play button (disabled={!ready}) stayed permanently
disabled whenever the initial WaveSurfer decode failed and the
component fell back to loading pre-computed peaks. The waveform still
rendered fine from those peaks (nothing looked visibly broken), but
`ready` was only ever flipped by the 'ready' event re-firing on that
recovery load — which this component's own error-handling never
actually confirmed, just assumed. Each of the three fallback ws.load()
calls now explicitly confirms readiness once it settles (via .then()/
.catch(), or the existing synchronous-throw catch), instead of hoping
the event fires again.

Regression test: WaveformTimeline.readyFallback.test.js — a
source-level contract guard (driving a real decode-failure/recovery
sequence through jsdom is brittle, same house pattern as the sibling
WaveformTimeline.unlock.test.js) asserting every fallback load in the
error handler is followed by an explicit setReady(true).

Also removes the "Support OmniVoice" heart button from NavRail — the
donate page stays reachable from Settings' footer and the Contact page.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:09:54 +05:30
33379890ad fix(voice): free-text instruct now filtered before every generate/save call (#1010) (#1018)
buildDesignInstruct() already keeps Studio's design/clone generate
calls (useTTS.js) from round-tripping a 400 "Unsupported instruct
items" — it filters free-text against the active engine's supported
vocabulary client-side, with a toast instead of a failed request. Three
other call sites built their own instruct string directly and skipped
it entirely:

- handleSegmentPreview (Dub tab's per-segment preview) — instruct comes
  straight from segment/preset data; a preset's raw attrs merged with a
  free-text style field can carry phrases outside the vocabulary.
- handleSaveProfile / handleSaveHistoryAsProfile — both always create a
  kind='clone' profile; the backend only sanitizes instruct on save for
  kind='design' (see profiles.py's heal_design_instruct branch), so a
  clone profile could silently persist an unusable instruct and then
  400 every single time it's later used to generate.

All three now filter through the same buildDesignInstruct({}, instruct)
call useTTS.js's own clone path already uses, with the same
unsupported/duplicate-item toasts.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:47:06 +05:30
7f8a42ce51 fix(tts): mlx-audio CSM cloning drops ref_text, breaking every clone attempt (#1012, #1013) (#1017)
MLXAudioBackend.generate() reads voice/ref_audio/language/speed from
its kwargs but never extracted ref_text — it was built, then silently
never passed through to self._model.generate(). CSM (sesame.py) only
builds its cloning context when BOTH ref_audio AND ref_text are
present; with ref_text missing, the context list stays empty and
indexing into it raises "IndexError: list index out of range" deep
inside mlx-audio, instead of the clone ever being attempted. Voice
cloning on the CSM engine could never have worked as shipped.

generation.py already threads ref_text all the way through — even
auto-transcribing it via the GPU pool when the caller supplies
ref_audio without one (~line 780) — so the value was always available
in kwargs; it just never survived the crossing into this specific
backend.

Reported with the precise root cause and a working fix (community
member independently diagnosed and patched it locally, confirmed
working on MPS/0.3.12). Two-line fix: extract ref_text and pass it
through when both ref_audio and ref_text are present (guards against
passing an orphaned ref_text with no accompanying audio to engines
that don't expect it).

Tests: tests/test_engines.py — ref_text is passed through when paired
with ref_audio, omitted when ref_audio is absent.

Also documents the second bug from the same report (#1013): macOS
microphone permission never prompts, so OmniVoice never appears in
System Settings to grant access. Root-caused to an unresolved upstream
Tauri/WebKit limitation (WKWebView's requestMediaCapturePermissionFor
delegate — wry#1195, tauri#11951, fix wry#1196 still open/unmerged, no
released version to bump to) — not something fixable here without an
unverified native Rust/WKWebView hack this session has no way to test.
Documented in docs/install/troubleshooting.md with the confirmed
workaround (record elsewhere, upload the file).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:34:18 +05:30
4de3c824d0 fix(gallery): surface the real error instead of a generic guess (#1009)
Found while triaging Discord: a community member (lehoangan227) hit
"Could not create that voice — the engine may be loading" trying to
use an archetype from the Gallery. That message is hardcoded and
shown for ANY failure — the actual cause (a 500, a validation error,
anything) is caught and discarded.

api/client.js's ApiError already builds a clean, user-facing message
for every failure mode (HTTP status + backend detail, a network
failure, or a detected backend crash) — this codebase's own
established convention elsewhere is to interpolate that message via
`{{message}}` (see BatchQueue.jsx, Settings.jsx, ToolsPage.jsx). The
Gallery's own catch blocks just weren't following it.

Fixed the whole class across VoiceGallery.jsx (use/preview),
CommunityZone.jsx (add-to-voices, whose catch clause didn't even bind
the error), and ImportsZone.jsx (search/upload/save/delete/trim —
handleDelete previously failed completely silently, no message at
all). All now interpolate the real error message, matching the
gallery.download_failed key that already did this correctly a few
lines away.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:45:34 +05:30
11fbbd3f6d fix(dub): cross-language dub no longer speaks source-language reference text verbatim (#1004) (#1008)
extract_speaker_clones/extract_segment_refs pair each audio slice (cut
at ASR segment timestamps) with that segment's own `text` field, on
the assumption the two agree. They routinely don't — Whisper (and
friends) frequently drift on segment boundaries: a trailing word
audible in [start, end] but missing from text, or vice versa. When the
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming
breaks down and the clone can emit the mismatched reference text
itself instead of the target-language line it was asked to speak —
reported with an exceptionally clear root-cause diagnosis and a
working A/B repro (matched pair: clean on the first try; mismatched
pair: wrong language 6/6 times).

Fix (as proposed in the report): re-transcribe each written reference
clip via the already-loaded, already-warm active ASR backend and use
that transcript as ref_text — this guarantees the pair matches by
construction, independent of whether the original segment text was
ever right. Falls back to the original text on any re-transcribe
failure or empty result — never a regression from current behavior,
only ever a fix.

New services.speaker_clone.refine_ref_text (single clip, unit-testable
against a duck-typed fake ASR backend) and refine_ref_texts (batch —
one executor round-trip per whole clones/seg_clones dict rather than
one per reference). Wired into dub_core.py's two clone-extraction call
sites, routed through _gpu_pool to match the established convention
for ASR-backend calls (the model is mid-lifecycle: TTS is offloaded,
ASR is loaded and exclusive, right where the existing per-chunk
transcribe calls already run on this same pool).

Tests: tests/test_speaker_clone_purity.py — 6 new cases covering the
mismatch-correction path, ASR-failure fallback, empty-transcript
fallback, no-backend no-op, and batch behavior (one failing entry
doesn't affect the others). Full backend suite: 2412 passed, 0 failed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:18:15 +05:30
f546f04c8e fix(ci): bump Linux release runner to ubuntu-24.04, fixing stale bundled WebKitGTK (#961) (#1007)
The AppImage bundles whatever libwebkit2gtk-4.1-dev the build runner's
apt repos resolve at build time (see the "Linux system deps" step) —
AppRun's LD_LIBRARY_PATH then makes that bundled copy take priority
over the host's system WebKitGTK at runtime. ubuntu-22.04's version
was stale relative to what current distros (Ubuntu 24.04+, Fedora 44)
ship, which is why a from-source build (linking straight against the
host's healthy system library) worked fine on the exact machine where
the shipped AppImage white-screened — the released binary was running
an older, buggier WebKitGTK under the hood regardless of the host.

Bumped the Linux release matrix entry to ubuntu-24.04, and ci.yml's
Tauri shell-check job to match (its own comment already says "Mirror
release.yml" — now it actually does, so a green PR check accurately
predicts the release build will also succeed).

Raises the AppImage's glibc floor from 2.35 to 2.39 (Ubuntu 24.04+) —
README's system-requirements table corrected from the now-false
"Ubuntu 20.04+" claim. No reports of anyone on a pre-2022 distro.

This does not fix the AppRun launcher's separate, related bug (its
WebKitGTK-version auto-detection reads the *system's* pkg-config
version, not the version actually bundled and running) — that would
need a reliable way to read the bundled .so's version from within the
AppImage, which isn't straightforward (WebKitGTK's soname doesn't map
1:1 to its release version) and isn't verifiable without a real Linux
build environment to test against. Left as a known, separate gap.

Cannot be verified from here on a real Ubuntu 26.04 machine — shipped
on the strength of the root-cause diagnosis, pending the reporter's
confirmation.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:21:50 +05:30
270b3b1c4c fix(backend): quitting mid-preload no longer reports a clean shutdown while a GPU-pool thread is still importing (#1000) (#1002)
A user-pasted backend log revealed the real cause behind a class of
'can't reach backend' reports: three rapid restart cycles, each ending
with 'Shutdown: done.' immediately followed by a 'Model loading failed:
Could not import module AutoFeatureExtractor' error. That error text is
transformers' own generic lazy-import wrapper (import_utils.py's
_LazyModule.__getattr__), not a real dependency problem — pyproject.toml
already pins transformers/torch/torchaudio/soundfile/librosa as core,
non-optional deps, and the same venv loaded the model successfully 90
seconds later in the same log.

Root cause: preload_task and capture_preload_task were created at
startup but never referenced in the lifespan shutdown block — idle_task
and worker_task got cancelled-and-awaited, the preload tasks were simply
abandoned. Cancelling an asyncio task awaiting run_in_executor() can't
stop the underlying OS thread once it's inside blocking import/load
work, so 'Shutdown: done.' logged while a GPU-pool thread was still
mid-Version: ImageMagick 7.1.2-25 Q16-HDRI aarch64 037e46295:20260604 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/license/
Features: Cipher DPC HDRI Modules
Delegates (built-in): bzlib freetype heic jng jpeg lcms ltdl lzma png tiff webp xml zlib zstd
Compiler: clang (21.0.0)
Usage: import [options ...] [ file ]

Image Settings:
  -adjoin              join images into a single multi-image file
  -border              include window border in the output image
  -channel type        apply option to select image channels
  -colorspace type     alternate image colorspace
  -comment string      annotate image with comment
  -compress type       type of pixel compression when writing the image
  -define format:option
                       define one or more image format options
  -density geometry    horizontal and vertical density of the image
  -depth value         image depth
  -descend             obtain image by descending window hierarchy
  -display server      X server to contact
  -dispose method      layer disposal method
  -dither method       apply error diffusion to image
  -delay value         display the next image after pausing
  -encipher filename   convert plain pixels to cipher pixels
  -endian type         endianness (MSB or LSB) of the image
  -encoding type       text encoding type
  -filter type         use this filter when resizing an image
  -format "string"     output formatted image characteristics
  -frame               include window manager frame
  -gravity direction   which direction to gravitate towards
  -identify            identify the format and characteristics of the image
  -interlace type      None, Line, Plane, or Partition
  -interpolate method  pixel color interpolation method
  -label string        assign a label to an image
  -limit type value    Area, Disk, Map, or Memory resource limit
  -monitor             monitor progress
  -page geometry       size and location of an image canvas
  -pause seconds       seconds delay between snapshots
  -pointsize value     font point size
  -quality value       JPEG/MIFF/PNG compression level
  -quiet               suppress all warning messages
  -regard-warnings     pay attention to warning messages
  -repage geometry     size and location of an image canvas
  -respect-parentheses settings remain in effect until parenthesis boundary
  -sampling-factor geometry
                       horizontal and vertical sampling factor
  -scene value         image scene number
  -screen              select image from root window
  -seed value          seed a new sequence of pseudo-random numbers
  -set property value  set an image property
  -silent              operate silently, i.e. don't ring any bells
  -snaps value         number of screen snapshots
  -support factor      resize support: > 1.0 is blurry, < 1.0 is sharp
  -synchronize         synchronize image to storage device
  -taint               declare the image as modified
  -transparent-color color
                       transparent color
  -treedepth value     color tree depth
  -verbose             print detailed information about the image
  -virtual-pixel method
                       Constant, Edge, Mirror, or Tile
  -window id           select window with this id or name
                       root selects whole screen

Image Operators:
  -annotate geometry text
                       annotate the image with text
  -colors value        preferred number of colors in the image
  -crop geometry       preferred size and location of the cropped image
  -encipher filename   convert plain pixels to cipher pixels
  -extent geometry     set the image size
  -geometry geometry   preferred size or location of the image
  -help                print program options
  -monochrome          transform image to black and white
  -negate              replace every pixel with its complementary color
  -quantize colorspace reduce colors in this colorspace
  -resize geometry     resize the image
  -rotate degrees      apply Paeth rotation to the image
  -strip               strip image of all profiles and comments
  -thumbnail geometry  create a thumbnail of the image
  -transparent color   make this color transparent within the image
  -trim                trim image edges
  -type type           image type

Miscellaneous Options:
  -debug events        display copious debugging information
  -help                print program options
  -list type           print a list of supported option arguments
  -log format          format of debugging information
  -version             print version information

By default, 'file' is written in the MIFF image format.  To
specify a particular image format, precede the filename with an image
format name and a colon (i.e. ps:image) or specify the image type as
the filename suffix (i.e. image.ps).  Specify 'file' as '-' for
standard input or output., and interpreter finalization tore down module
state under it — producing exactly this misleading error.

Fix: extract the existing cancel+bounded-await pattern into
_cancel_and_await_tasks() and apply it to all four background tasks, not
just two. An early-stage load (still importing, not yet mid weight-
download) now gets a real chance to finish before shutdown proceeds; a
load genuinely deep in blocking work still times out at the same 3s
bound, and _reset_gpu_pool() abandons it same as before. Also: both
error handlers around this path logged only str(exc), discarding
__cause__ — added exc_info so a future incident (even one this fix
doesn't fully prevent) surfaces the real underlying error instead of the
misleading generic wrapper text.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:09:18 +05:30
5a7d9cc05c feat(asr): generic OpenAI-compatible transcription backend (#877) (#1003)
First slice of the community's two-track proposal for #877: a generic
OpenAI-compatible ASR backend that works TODAY, without waiting on
transformers to ship a direct Qwen3-ASR integration (tracked separately,
still blocked upstream). Points OmniVoice's transcription at any server
exposing POST /v1/audio/transcriptions — a self-hosted Qwen3-ASR/
FunASR/SenseVoice server, or OpenAI's own API.

- New OpenAICompatASRBackend (backend/services/asr_backend.py): a pure
  network client, no local model, no install. Prefers
  response_format=verbose_json for real per-segment timestamps,
  degrades to plain text (matching MoonshineASRBackend's shape) when a
  minimal server rejects that format. Never leaks a raw SDK/httpx
  exception to the caller (#977 convention) — wraps network/auth
  failures in a clean, actionable RuntimeError naming the server.
- Settings persist via the same encrypted-secret convention as
  services/llm_providers.py (settings_store.set_secret for the API key
  — Fernet-encrypted, never a .env row, never echoed back; get_text/
  set_text for base_url/model). New GET/PUT /api/settings/
  asr-openai-compat, loopback-gated like every other settings route.
- Frontend: a small settings panel (Settings → Models) mirroring
  HFMirrorPanel's exact structure. No ASR engine picker exists yet for
  ANY ASR backend (only TTS has one) — activating this engine still
  needs OMNIVOICE_ASR_BACKEND=openai-compat-asr; documented plainly
  rather than pretending otherwise.
- README's ASR Engines table (9 → 10 engines) and docs/features.yaml's
  drift-checker inventory updated; the '9 engines, all fully local'
  claim corrected since this one genuinely isn't.
- docs/engines/openai-compatible-asr.md: setup steps + an explicit
  privacy note (unlike every other ASR engine, audio leaves the
  machine to whatever server is configured).

Regression tests: tests/test_asr_openai_compat_877.py (12 tests) —
is_available() gating, verbose_json + plain-text response adaptation,
network-failure error hygiene, SDK retry disabling, and the settings
endpoints' persist/mask/clear-vs-unchanged semantics.

Fixed two real full-suite-only failures found during verification (not
brushed aside): the API route inventory snapshot needed regenerating
for the two new routes, and this file's own tests had a module-
staleness bug — a collection-time settings_store import went stale
relative to a test-time-fresh fixture when another test elsewhere in
the ~2400-test suite reimports the module — fixed by making
settings_store itself a fixture resolved at test-run time, same
lesson already applied to tests/test_mm2_lifecycle.py earlier this
session.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 08:33:03 +05:30
5871ec9d28 release: freeze v0.3.12 — version bump, lockfiles, changelog (#1001)
19-issue GitHub sweep: 16 PRs merged since v0.3.11, most fixing reports
filed same-day or in the days prior. Highlights: MLX-Audio's 6 other
curated models are finally selectable (was silently stuck on Kokoro
regardless of what was downloaded), first-run no longer dead-ends behind
restricted networks or corporate TLS-inspecting proxies, dubbing/batch
TTS honor your active engine selection, and a run of sharp community
diagnoses (ROCm wheel index, Windows dictation focus-steal, a genuine
frontend crash regression) got fixed largely because reporters did the
hard diagnostic work themselves.

Full backend suite: 2390 passed, 0 failed. Full frontend suite: 918
passed, 0 failed. Version lockstep (tests/test_app_version.py): 6/6
passed. Docker frozen-lockfile parity (bun install --frozen-lockfile):
clean, no drift.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 07:28:41 +05:30
58b023ff16 fix(diagnostics): TTS generate timeout message tells you to Flush/Unload (#939) (#999)
The guard itself (#851/#896) is working as designed — this is a message
gap, not a logic bug. The GPU-branch timeout guidance explained VRAM
contention but never mentioned the Flush/Unload action that actually
resolves it, even though: (a) that action already exists (POST
/system/flush-memory, wired to the header's Flush button), and (b) the
sibling ASR-timeout guard's message already recommends it verbatim
(asr_backend.py's _CUDA_VRAM_BUDGET_GB guidance). The maintainer ended up
manually explaining 'Settings → Models → Flush caches / Unload' in an
issue thread reply — information the error message should have carried
itself.

String-only change, no control-flow touched, mirrors the exact precedent
of #896 (a guidance-only change to this same function).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 05:18:00 +05:30
b61d17dd61 fix(diagnostics): filter Tauri's benign IPC-fallback warning from frontend log capture (#975) (#998)
On some Windows configurations, Tauri's custom-protocol IPC probe fails
once at startup and Tauri logs a console.warn before silently — and
successfully — falling back to postMessage + WebSocket. Fully functional,
happens at most once per launch, and not a bug in our code (confirmed:
this is Tauri's own internal fallback mechanism, structurally intentional
across its recent 2.11.x releases, not something being actively patched
upstream — so not bumping the framework speculatively for this).

It IS real noise though: as a captured console.warn it spuriously flips
the Settings > Logs footer's Frontend pill to "1 warning" on every
affected Windows launch. Filtered at the capture source (consoleBuffer.js)
rather than the display layer, so it never enters the ring buffer or a
copied diagnostic dump either — narrowly scoped to this one known message
prefix, not a general warning-suppression mechanism.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 05:17:12 +05:30
93aab6dadc docs(linux): mention yt-dlp as an optional prerequisite (#973) (#997)
The preflight system check already warns in-app when yt-dlp is missing
(Voice Gallery/Dub YouTube downloads fail without it), but the install
docs never mentioned it — a user has to hit the in-app warning first
instead of seeing it up front alongside the other optional prereqs.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:42:00 +05:30
7098be1c8b fix(dictation): Windows pill no longer steals foreground focus (#982) (#996)
Root-caused and verified against the actual code (community diagnosis in
#982 was accurate): showing the dictation pill — an always-on-top top-level
WebviewWindow — grants it Win32 foreground activation by default on
Windows, which macOS doesn't do for a shown window. Nothing marked the pill
non-activating, so on Windows the synthesized Ctrl+V from `simulate_paste`
landed back in the pill instead of the app being dictated into. Code review
also found a second, more direct culprit at the same call sites: the
global-shortcut Pressed handler's `win.set_focus()` was only skipped on
macOS (`#[cfg(not(target_os = "macos"))]`), so Windows explicitly focused
the pill on top of the implicit Win32 activation; the tray "dictate" menu
handler called `win.show(); win.set_focus();` unconditionally on every
platform, including Windows.

This is the Windows counterpart of #287 (macOS auto-paste — don't steal
focus): same class of bug, same "pill must stay unfocused so the paste
lands in the target app" intent (already stated in the `grant_webview_
media_permissions` module comment), different OS mechanism.

Fix, mirroring #287's platform-cfg pattern:
  1. WS_EX_NOACTIVATE applied to the pill's HWND once, right after creation
     (`mark_pill_noactivate`), via the `windows` crate pinned to the exact
     0.61.x tauri itself already resolves to — Cargo.lock confirms this
     unifies to the same `windows 0.61.3` already in the graph, so
     `WebviewWindow::hwnd()`'s HWND and our Win32 calls share one type and
     no new crate version was added.
  2. `ShowWindow(SW_SHOWNOACTIVATE)` (`show_pill_noactivate`) in place of
     `.show()` at the two dictation-trigger call sites (global shortcut +
     tray "Start Dictation"), since `.show()` alone still raced the style
     bit on some paths.
  3. The explicit `set_focus()` calls at those same two sites are now
     skipped on Windows too, the same way they already were on macOS.

macOS and Linux are untouched: the macOS cfg branch is unchanged, and the
Linux branch of the `set_focus()` guard still runs exactly as before.

The pill's auto-dismiss (`scheduleDismiss`/`dismiss` in CaptureWidget.jsx)
was checked and is a plain unconditional setTimeout chain — it is not
gated on any native focus-loss/paste-completion signal, so there's no
independent bug to fix there. The "never dismisses" symptom is a
consequence of the focus-steal, not a separate stall: once the pill wrongly
held foreground for the whole session, hiding it later left Windows'
foreground state inconsistent. With the pill never taking focus, the
target app stays foreground throughout and there's nothing to reconcile.

Win32 window-activation syscalls (`#[cfg(target_os = "windows")]`) can't
run under `cargo test`/`cargo build` on this non-Windows CI runner, so the
new `pill_noactivate_tests` module tests the pure flag math
(`with_noactivate_style`) instead — platform-agnostic, runs everywhere,
verified passing here. The actual HWND-touching code is logic-reviewed but
UNVERIFIED on real Windows; the reporter offered to test a patched build,
which is the recommended next step before this ships in a release.

`cargo build` and `cargo test --lib` both pass (60/60 tests, including the
3 new ones); the pre-existing `setup.rs` unreachable_code warning (#286) is
unrelated.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:41:25 +05:30
8520b84b68 fix(clone): voice-design panel no longer crashes on a partial vd_states shape (#983) (#995)
Crash: DesignMethodPanel's optLabel() called val.replace(...) on an
undefined category value — a regression from f678e33, which swapped a
safe plain template literal for an i18n lookup that assumes vdStates[key]
is always a string. Both occurrences (the label kicker and the chip
list) are now guarded, falling back to 'Auto' the same way the rest of
the component treats an unset category.

Root cause: vdStates could actually go partial in real usage. Selecting
a design profile (useProfiles.js) or restoring legacy localStorage state
(useAppData.js) applied the backend/stored vd_states object as-is, with
no check that all 6 CATEGORIES keys were present — so an older client,
hand-edited payload, or partial API write reproduced the crash on
selection. Both call sites now run the restored object through
mergeDescribedAttrs() (voiceInstruct.js), the existing completion helper
already used for the "describe your voice" path, which fills any
missing/unknown category with 'Auto'. useAppData.js also gained the
typeof === 'object' guard useProfiles.js already had.

Closes the class at the source: POST /profiles now completes vd_states
against CATEGORY_ORDER (core/describe_voice.py, the same list the
frontend's CATEGORIES mirrors) before persisting, so a design profile
can never be *saved* with an incomplete shape regardless of which
client wrote it — updated two existing tests whose fixtures asserted
the old (partial) persisted shape.

Regression tests: DesignMethodPanel render test with a partial vdStates
input, a mergeDescribedAttrs unit test for the exact partial shape from
the issue, and a backend test asserting POST /profiles fills all 6 keys.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:34:14 +05:30
549fa4009f feat(engines): expose MLX-Audio's curated model picker (#981) (#994)
mlx-audio multiplexes 7+ curated models (Kokoro, CSM, Qwen3-TTS, Dia,
Chatterbox, MeloTTS, OuteTTS) behind a single "mlx-audio" backend id, but
MLXAudioBackend resolved its active model ONLY from the
OMNIVOICE_MLX_AUDIO_MODEL env var — invisible to Settings and unreachable
without restarting the packaged app with that var set. A user who
downloaded e.g. Llama-OuteTTS via Settings → Models had no way anywhere
in the UI or API to actually load it; the backend silently kept using
Kokoro.

Fix:
- MLXAudioBackend.__init__ now resolves its model via
  prefs.resolve("mlx_audio_model_id", env=..., default=...), mirroring
  active_backend_id()'s env > prefs > default order exactly.
- get_active_tts_backend()'s switch-detection now also tracks the
  resolved mlx-audio model key, so a model-only change (same backend id)
  invalidates the cached instance and reconstructs it — no app restart
  needed to pick up a different curated model.
- POST /engines/select gained an optional model_id field; for
  family=tts/backend_id=mlx-audio it validates against
  MLXAudioBackend.CURATED_MODELS (or a raw HF repo id, matching the
  class's existing tolerance) and persists it via prefs.
- GET /engines now includes a curated_models roster + active_model_id on
  the mlx-audio entry only.
- Settings → Engines renders a small model dropdown on the mlx-audio row,
  pre-selected to the active model, wired through selectEngine's new
  optional modelId argument.

Regression coverage: prefs resolution + env override, cache invalidation
on model-only switch, /engines/select 400s on an unknown model id and
persists a valid one, curated_models present only on mlx-audio, and a
new EngineCompatibilityMatrix vitest suite for the dropdown.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:32:24 +05:30
dfe2bd1bc3 fix(install): classify SSL handshake failures + trust the OS cert store (#976) (#992)
Windows users behind a corporate/antivirus TLS-inspecting proxy got a raw
`[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE]` on every model install — the TCP
connection reaches the server fine, but the handshake fails because the
OS trusts the proxy's re-signed root CA and Python's bundled certifi CA
list doesn't. A genuinely different failure mode from #984 (that was
TCP-level unreachability to a blocked host, before any TLS negotiation).

- backend/core/failure.py: new SSL_HANDSHAKE_FAILURE classification
  (handshake/cert-verify-failed/sslv3_alert/sslcertverificationerror
  substring markers) with an actionable hint, added to
  _CONTEXT_FREE_HINT_CLASSES so append_hint() (already called by
  setup/download.py's install worker) surfaces it without further wiring.
- backend/main.py: truststore.inject_into_ssl() at module level, before
  any huggingface_hub/requests/httpx network I/O — patches ssl.SSLContext
  to verify against the OS trust store instead of only certifi's bundled
  CA list. Not platform-gated (correctness improvement everywhere);
  wrapped in try/except so it never blocks startup.
- pyproject.toml/uv.lock: truststore>=0.9 — pure Python, MIT, PyPA-
  maintained, zero transitive deps, same class of fix as socksio.

Verified: uv lock --check + uv sync --frozen clean (lockfile diff is
just the one new package); main.py imports cleanly; full backend suite
passes; no hiddenimports entry needed (main.py is PyInstaller's direct
entry script per backend.spec, so a top-level import traces normally —
unlike socksio's case, which was httpx's internal lazy import).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:29:21 +05:30
324d417d27 fix(engines): mlx-audio no longer crashes on unsupported languages, error messages never leak raw exception internals (#977) (#993)
Root cause: MLXAudioBackend.generate() blindly truncated the full
language display name to two characters (language[:2].lower()),
assuming an ISO code — 'Dutch' -> 'du', which crashed Kokoro's vendored
pipeline's internal assertion (assert lang_code in LANG_CODES, (lang_code,
LANG_CODES)) for any language whose first two letters didn't coincidentally
match one of Kokoro's single-letter codes. The raw AssertionError's
tuple-containing-a-dict args then leaked straight into the user-facing
500 message via two stacked f"...{e}" formatters in generation.py.

- resolve_kokoro_lang_code() resolves against the AUTHORITATIVE
  ALIASES/LANG_CODES table read from the installed mlx_audio package
  (never a hardcoded guess), and only applies when Kokoro is the actual
  active curated model — other curated models (CSM, Dia, Qwen3-TTS,
  OuteTTS, ...) either ignore the kwarg or expect a different format, so
  Kokoro's strict validation doesn't wrongly reject them. Unsupported
  languages now raise a clear ValueError naming what Kokoro supports,
  which generation.py already converts to a clean 400.
- _safe_exc_text() hardens both generic exception formatters in
  generation.py: if any element of an exception's .args is a container
  (dict/list/tuple/set), never interpolate str(e) raw — name the
  exception type and point at the log instead. Protects every current
  and future engine's generate() from leaking a raw container repr, not
  just this one Kokoro assertion.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:29:01 +05:30
637020b82b fix(engines): nemo-parakeet install hint stops recommending a shared-venv-breaking pip install (#974) (#991)
The Engines page told users to run `pip install nemo_toolkit[asr]` for
the NeMo Parakeet ASR engine. nemo_toolkit[asr]==2.7.3 hard-pins
transformers>=4.57,<4.58, which is unsatisfiable alongside OmniVoice's
own transformers>=5.3 requirement (needed by
omnivoice/models/omnivoice.py for HiggsAudioV2TokenizerModel). A user
who followed the hint ended up with a backend that wouldn't start
(ImportError: cannot import name 'HiggsAudioV2TokenizerModel').

_INSTALL_HINTS["nemo-parakeet"] in backend/services/asr_backend.py now
states plainly that installing into the shared venv will break the
backend, names the transformers conflict, and tells users to use a
separate/dedicated Python environment instead — without implying a
safe one-line fix or an isolated-venv env var exists (unlike
dots-tts/moss-tts-v15/confucius4-tts, nemo-parakeet has no isolated
venv option yet; that's a separate, larger follow-up).

Also adds one sentence to docs/install/troubleshooting.md's existing
"engine venv clash" section (#11) pointing at the same class of issue
on the ASR side, and a regression test asserting the hint never again
contains the literal bare `pip install nemo_toolkit[asr]` string.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:26:12 +05:30
dab6456581 docs(linux): stop advertising a .deb package that isn't published (#990)
README's Quickstart badges linked a 'Download Debian .deb' button
straight to the releases page — but .deb bundling was deliberately
dropped from release.yml (tauri-cli bug, 'Failed to create control
scripts') and no release has ever shipped one. A community member
investigating #961 confirmed this by checking the actual release
assets. Users clicking that badge got a broken promise, not a package.

Removed the badge; docs/install/linux.md's '## Install (.deb)' section
now honestly states it's unavailable pending a tauri-cli fix, points to
the AppImage as the supported path, and keeps the historical pre-v0.3
.deb upgrade note (ffprobe conflict) since that's still relevant to
existing installs.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:40 +05:30
015d29cafb test(voice-design): lock in RTL/non-Latin script handling for the instruct field (#980) (#989)
Issue #980 reported a raw 400 for Hebrew text ('שמואל') typed into the
Clone tab's Style field. Investigated: this is the same failure class
as #612 (Vietnamese free-text) and was already fixed when #612 landed
in commit 10b9d69 (first released v0.3.8) — buildDesignInstruct() drops
any unsupported free-text client-side before it ever reaches the
backend's validator, regardless of script. The reporter was on v0.3.7,
which predates that fix.

No behavior change needed — only a regression test, since the existing
Vietnamese test case covered Latin-script-with-diacritics but nothing
exercised a right-to-left / non-Latin script specifically.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:24 +05:30
1a03c59f82 fix(install): AMD ROCm torch reinstall targets rocm6.4, not rocm6.2 (#988)
Community-diagnosed (issue #972, Kaihui-AMD): pyproject.toml pins
torch==2.8.0, but the rocm6.2 wheel index only ever published up to
2.5.1 — the reinstall silently failed to resolve and fell back to the
default CUDA build, which runs on CPU on an AMD GPU. The failure was
correctly logged (bootstrap.rs's emit_log warning), just never actioned
because the index itself couldn't succeed. rocm6.4 carries a matching
torch==2.8.0 build.

Docs updated with the corrected index plus a repo.radeon.com find-links
path for users who want a driver-matched ROCm 7.2.x build the PyTorch
index doesn't carry (OMNIVOICE_TORCH_INDEX only accepts a PEP 503 index,
not find-links, so that's documented as a manual step).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:00 +05:30
8281b7c798 fix(engines): dub and batch TTS honor the active-engine selection (#987)
* fix(engines): dub and batch TTS honor the active-engine selection — with a real capability gate, not a silent OmniVoice fallback

Dub generation and batch TTS hardcoded services.model_manager.get_model()
(OmniVoice) regardless of the engine picked in Settings → Engines. A user
selecting VoxCPM2 (or any other engine) still got OmniVoice output with no
error — the silent fallback IS the bug class, not just the one report.

Root-caused and fixed for the whole class:
  - New `TTSBackend.supports_cloning` capability flag (default True) marks
    engines that can only offer fixed preset voices — kittentts,
    supertonic3, sherpa-onnx set it False. MLXAudioBackend exposes it as an
    instance property (Kokoro doesn't clone, CSM does) since the adapter
    multiplexes multiple models with different capabilities.
  - `cloning_capable_engine_ids()` and a shared `resolve_generation_backend()`
    helper in services/tts_backend.py centralize engine resolution
    (id → is_available() → routing gate → optional cloning gate), mirroring
    generation.py's /generate resolution instead of inventing a third
    parallel mechanism. Both routers now standardize on the existing
    get_active_tts_backend() cache (unload-on-switch already handled).
  - dub_generate.py's two TTS-generate call sites (main run + OOM retry) and
    the /dub/preview-segment route resolve once, up front, with
    require_cloning=True — dub's ref_audio is populated for essentially
    every real job, so an engine that can't clone fails the whole job with
    one actionable message instead of mis-cloning per segment.
  - batch.py resolves once per job, require_cloning only when voice_id is
    pinned — an unpinned batch job runs fine on any engine.
  - Applied the three pre-existing TODO(#312) comments: mastering now skips
    via `applies_own_mastering` for both pipelines, matching generation.py.

Regression tests cover the capability-id list, the fail-fast gate (proving
no OmniVoice fallback), the success path on a selected non-OmniVoice
engine, batch's pinned-vs-unpinned voice_id behavior, and the mastering
skip for both pipelines. Three existing dub tests that mocked get_model()
directly were updated to mock the new resolver instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(engines): exclude model-dependent adapters from cloning_capable_engine_ids()

getattr(cls, "supports_cloning", True) at the CLASS level returns a
property descriptor object (always truthy) when the flag is an instance
@property, not a plain attribute — MLXAudioBackend uses exactly this
pattern because its cloning capability depends on which of its 7+ curated
models is loaded (only CSM clones; Kokoro etc. don't). Without this fix,
the dub/batch capability-gate error message would always recommend
'switch to mlx-audio' even when the user's configured MLX model can't
clone, sending them in a circle back to the same error.

isinstance(value, bool) distinguishes a resolved boolean from a
descriptor object, so mlx-audio is excluded from the suggestion list
until its actual per-instance capability can be checked (already handled
correctly by resolve_generation_backend()'s per-call instance check).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): engine-aware dub/batch entry (#987)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 03:42:14 +05:30
efe57ee49d fix(ui): loaded-models panel says when a resident model is not the active engine (#985)
* fix(ui): loaded-models panel says when a resident model is not the active engine (#TBD)

A switched-away TTS model stays resident in VRAM until Unload/Flush or the
idle evictor, so users who picked another engine (e.g. voxcpm2) saw
"OmniVoice TTS - cuda:0 - 1937 MB" in the LOADED MODELS flyout and concluded
synthesis was still routing to OmniVoice. It wasn't - the panel just gave no
hint that resident != active.

/model/loaded entries for TTS-family models (in-process OmniVoice +
subprocess sidecars) now carry engine_id + is_active_engine, computed against
active_backend_id(); attribution failure degrades to the old shape
(is_active_engine: null) and non-TTS entries (ASR, diarization) are left
unannotated. The flyout renders a muted "not active - safe to unload" tag
(i18n: header.model_not_active, en + zh-CN) on inactive entries; Unload/Flush
behavior is unchanged. Regression tests cover both attribution states, the
ASR non-label, and the degradation path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): loaded-models active-engine hint entry (#985)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(mm2): string-target monkeypatch for active_backend_id — immune to sys.modules reimports

The two attribution tests patched the collection-time module alias; other
suites pop+reimport services.* modules mid-run, so in full-suite order the
patch landed on a stale module object while _active_tts_id late-imported the
fresh one (CI-only failure). String targets resolve at patch time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:41:47 +05:30
7f77f4d7bf fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#986)
* fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#TBD)

Field report (Discord): baked-in echo/reverb on some voices. apply_mastering()
hardcoded a Reverb that ran on every non-raw synthesis before the user's
preset chain — broadcast shipped reverb it never declared, podcast broke its
"no reverb" promise, cinematic/warm got doubled reverb.

The mastering pre-stage is now data-driven (MASTERING_CHAIN: highpass +
compressor, same params as before) and reverb-free; cinematic/warm keep their
user-chosen reverb. Regression tests pin the contract, incl. a burst-then-
silence echo-tail check and pedalboard-missing passthrough.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): hidden mastering reverb entry (#986)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:26:19 +05:30
087309259b fix(setup): first-run network check is mirror-aware and never hard-blocks (#984)
* fix(setup): first-run network check is mirror-aware and never hard-blocks

Field report (Discord, China): the Launchpad preflight probed hardcoded
huggingface.co:443 and any failure disabled Continue outright — users behind
the GFW were stuck on the very first screen, before Settings (and its
HF mirror quick-pick) was even reachable.

- The probe now targets the HF endpoint actually in effect (HF_ENDPOINT /
  hf_endpoint pref via configured_hf_mirror), with the real port.
- An unreachable endpoint is a WARNING, not a blocker: local-first — cached
  models work offline, and downloads surface their own actionable errors.
- When huggingface.co is blocked but hf-mirror.com answers, the fix text says
  exactly that, and the wizard shows an inline mirror quick-pick (presets +
  custom URL) that applies via PUT /hf-mirror — effective immediately for
  downloads — then re-checks.
- Docs updated (downloading-models, install troubleshooting); regression
  tests cover warn-not-fail, mirror-host probing, and the mirror suggestion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): open [Unreleased] with the preflight mirror fix (#984)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:11:18 +05:30
453db55f12 release: freeze v0.3.11 — version bump, lockfiles, changelog (#970)
package.json + three mirrors -> 0.3.11 in lockstep; Cargo.lock/uv.lock/
bun.lock regenerated; CHANGELOG [Unreleased] -> [0.3.11] — 2026-07-05
with the multi-language-release headline; nine entries since v0.3.10.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 22:28:43 +05:30
620321a9cc feat(diagnostics): backend crashes become self-documenting — exit code + stderr tail surfaced and attached to bug reports (#969)
* feat(diagnostics): backend crashes become self-documenting — exit code + stderr tail surfaced and attached to bug reports

When the backend PROCESS died (native CUDA abort, OOM kill, DLL crash) the
user saw only "Can't reach the local OmniVoice backend" and the evidence
died with the process — every #941-class report needed a logs-please
round-trip nobody answers. The v0.3.9 guard fixed HANGS; this fixes the
class of invisible DEATHS:

- Rust (crash.rs): every unexpected child exit — detected by the startup
  health poll and the post-Ready supervisor — writes a rotating (last 3)
  JSON crash marker next to the backend logs: ts, exit code/signal,
  backend version, uptime, ~40-line stderr tail. Intentional shutdowns
  never forensicate: app-quit raises the quitting flag first (now also on
  macOS Cmd+Q via ExitRequested), and retry/clean-retry kills set a
  BACKEND_KILL_INTENDED flag cleared when the fresh child is tracked.
- Tauri commands get_last_backend_crash / acknowledge_backend_crash;
  ack is a persisted watermark, never a delete — bug reports still get
  the evidence after the user viewed it.
- Crash-loop escalation: the supervisor budget goes 5-in-60s → 3-in-10min
  so slow crash loops stop respawning and land on the Failed screen with
  the last exit code + stderr tail.
- Frontend: apiFetch's transport-failure path swaps the vague message for
  "the backend crashed (exit code X) N s ago…" when an unacknowledged
  marker exists, and BackendCrashNotice (banner + details dialog,
  i18n'd, ack-on-view) surfaces it even with no request in flight.
- Bug-report prefill gains a "Last backend crash" section (exit code +
  home-path-scrubbed stderr tail via the existing scrubText), so the next
  report arrives WITH the evidence.

Tests: cargo --lib 57 pass (marker rotation write-4-keep-3, ack
semantics, store IO, ExitStatus decomposition, 3-in-10min policy);
vitest 909 pass incl. crash-notice branch, client crash-message branch,
bug-report enrichment; legacy node:test 41 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add backend crash forensics under [Unreleased] (#969)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:36:47 +05:30
738d45f1c7 fix(ui): timeline box colors pre-blended in JS — visible on any WebView2 (#963) (#968)
* fix(ui): timeline box colors pre-blended in JS — visible on any WebView2, color-mix dependency removed (#963)

#951 moved the segment-box palette to `color-mix(in srgb, tint 45%,
var(--chrome-bg))` strings applied as inline styles. WebView2/Chromium
< 111 has no color-mix, so the CSSOM rejects the whole `background`
assignment — and since .seg-track__box declares no background of its
own, the boxes rendered fully transparent on pinned/enterprise WebView2
runtimes (the Windows installer never enforces a minimum runtime).

Fix the class, not the instance: no engine-dependent CSS may reach this
lane's inline styles. The 0.45·tint + 0.55·bg blend now happens in JS —
timeline.js keeps the tints as numeric [r,g,b], reads --chrome-bg off
the document root (fallback #0f1011), and emits literal `rgb(r, g, b)`
strings every engine parses. Pixel-identical to what color-mix painted.
Theme-awareness is preserved by re-blending when [data-theme] changes
on <html> (the seam App.jsx switches themes through), observed via
MutationObserver; SegmentTrack subscribes with useSyncExternalStore so
mounted boxes recolor live.

Guards updated: palette entries must match plain opaque rgb() (no
color-mix/var()/alpha), the default-theme blend is asserted against
independently computed literals, theme-change re-blend and rgb()/
garbage --chrome-bg parsing are covered, and SegmentTrack's rendered
inline background is asserted to be a literal rgb() — fails on any
reintroduction of engine-dependent CSS in this lane.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add WebView2 box-color fix under [Unreleased] (#968)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:28:19 +05:30
d959aae41b fix(dub): dialogue starts stop snapping to footsteps — sustained-energy onsets, bounded snap (#963) (#967)
* fix(dub): dialogue starts stop snapping to footsteps — sustained-energy onsets, bounded snap distance (#963)

Field report #963 (point 3): dubbed speakers start seconds early or late.
The reporter's own theory was right on the money — 'when a noise is heard
(a sigh or footsteps), it's interpreted as the start of the conversation.'
The #280 onset snapper took the FIRST 20 ms frame above an adaptive RMS
threshold as the speech onset, so any transient qualified; it also had no
snap-distance bound (a wrong onset could move a start by the whole segment
minus 0.3 s) and ran even when Demucs had failed and the 'vocals' track was
really the raw mix, where every ambient sound is a candidate.

Three layered guards, all pure NumPy (no new deps):

- Sustained energy: an onset only counts when >=160 ms of the following
  300 ms stays above the threshold. Footsteps/door thuds light up one or
  two frames and die; syllables keep the energy up.
- Bounded snap distance: shifts beyond 1.5 s are only trusted when the
  skipped span is (near-)silent — that is exactly the genuine #280
  whisper start-stretch on the vocals track (Demucs removed the music,
  leaving real silence), so long trims over silence still work in full.
  Long jumps over audible content (e.g. quiet speech under the relative
  threshold) are refused instead of playing the dub seconds late; an
  isolated transient in the span (<10% audible frames) doesn't block it.
- Source-aware: snapping now runs only on the separated vocals track.
  dub_core detects the Demucs fallback (vocals_path == audio_path, see
  dub_pipeline) at both call sites and passes separated_vocals=False on
  mixed audio, disabling snapping — whisper's own timestamps beat a
  confidently wrong snap when music/ambience is sustained energy too.

Tests (tests/test_onset_align.py, fail-before/pass-after): transient burst
rejected at detect- and snap-level, transient-only window yields no onset,
long jump over audible content refused, bounded shift over audible lead
still allowed, >1.5 s trim over true silence still snaps (#280 regression
guard), mixed-audio mode is a no-op. 28 pass in the file; full dub-adjacent
suites green.

Credit: theory and repro description by the #963 reporter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add onset-snap robustness under [Unreleased] (#967)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:15:14 +05:30
de99bc3bd5 fix(net): SOCKS-proxy users can synthesize again — ship socksio, cache-first model resolution (#959) (#966)
* fix(net): SOCKS-proxy users can synthesize again — ship socksio, cache-first model resolution, degrade LLM clients (#959)

Under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio installed, httpx
raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS proxy, but the
'socksio' package is not installed"). huggingface_hub's get_session()
builds exactly that client inside snapshot_download, so POST /generate
500'd with the bare message even for a fully installed model, and
preload_model's model_info probe hit the same error and silently
skipped warm-up. Latent since v0.3.5 — #947's fresh-process engine
spawning unmasked it in v0.3.10 by handing the user's proxy env
directly to a clean backend process.

Three layers, so the class (any session-construction failure) is dead,
not just the reported instance:

* Ship SOCKS support: socksio>=1.0 in [project] dependencies (pure
  Python, MIT, zero transitive deps) AND in backend.spec hiddenimports
  — httpx imports it lazily in try/except, so PyInstaller's tracer
  misses it and the frozen installers would stay broken without the
  explicit entry. uv.lock regenerated; `uv lock --check` and
  `uv sync --frozen` (the Docker/release bootstrap semantics) verified.

* Cache-first model resolution: from_pretrained's snapshot resolution
  extracted into _resolve_snapshot_dir() — local dir, else
  snapshot_download(local_files_only=True) (a complete cache resolves
  with NO HTTP session constructed), else the original network path.
  preload_model's failed network probe now falls back to a cache-only
  check and warms up anyway instead of silently skipping (honest log
  either way).

* Class guards: resolve_skill_client wraps OpenAI() construction —
  env-shaped construction failures degrade to the existing "LLM
  unavailable" contract instead of 500ing the calling feature; and
  core.failure learns SOCKS_PROXY_SUPPORT_MISSING with an actionable
  hint, appended on the raw-string surfaces (global 500 handler,
  model-install SSE) via the new append_hint().

Fail-before/pass-after verified by reverting the fix: 11 of the 12 new
tests fail pre-fix (the remaining one is the unchanged network-fallback
contract). 165 tests green across the touched suites.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add SOCKS-proxy resilience under [Unreleased] (#966)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 20:00:08 +05:30
c47633a409 fix(settings): a saved LLM provider survives restart — explicit save activates, stale TRANSLATE_* prefs stop hijacking (#963) (#965)
* fix(settings): a saved LLM provider survives restart — explicit save activates, stale TRANSLATE_* prefs stop hijacking (#963)

"Ollama works until I restart OmniVoice" had three stacked causes:

1. Only "Save & use for translation" persisted the selection. Plain
   "Save" and "Test" sent make_active:false, and on restart
   active_provider_id() deliberately excludes local providers
   (Ollama/LM Studio) from auto-select — so a saved-and-tested Ollama
   was never resolved active again. The PUT handler now also claims the
   active slot on an explicit save when the user has never chosen a
   provider (new llm_providers.stored_active_provider_id(): the stored
   row only — no env pin, no legacy fallback, no auto-detect). An
   explicit prior choice is never stolen; an unconfigured provider
   can't claim the slot; make_active:true still flips.

2. Users of the retired (≤v0.3.7) Translation-LLM panel had
   env.TRANSLATE_* rows in prefs.json, re-imported into os.environ
   every launch — and a live TRANSLATE_BASE_URL resolves the active
   provider to "custom" ahead of auto-select on every restart. New
   startup migration (llm_providers.migrate_legacy_translate_prefs,
   run in main.py BEFORE the prefs→env import) moves those values into
   the custom provider's own settings-store rows (only where the store
   has no value yet) and deletes the prefs rows. Real process env vars
   are never touched; a failed store write keeps the prefs row and
   retries next boot. The legacy endpoint keeps working — via the
   store, without hijacking the active slot.

3. The panel read as "done" after a green Test even when another
   provider stayed active. It now shows a notice after save/Test when
   the edited provider is not the effective active one (suppressed
   while LLM_DEFAULT_PROVIDER pins the choice — the env banner already
   covers that).

Tests (fail-before): 7 new backend tests fail on the old code
(save-activates, never-steals, migration semantics, env untouched,
end-to-end ollama-beats-legacy-env) and the new panel test fails
without the notice; all pass after. Full LLM/settings suites, frontend
vitest (890), typecheck:ci, oxlint, oxfmt and vite build are green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add LLM-provider persistence fix under [Unreleased] (#965)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 19:47:54 +05:30
4aa9abe22a docs+scripts: install fixes — desktop-prod tauri resolution, Ubuntu white-screen guidance, honest GPU/prereq docs (#960 #961 #962) (#964)
* docs+scripts: install fixes — desktop-prod tauri resolution, Ubuntu white-screen guidance, honest GPU/prereq docs (#960 #961 #962)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the install-fixes batch under [Unreleased] (#964)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 19:34:01 +05:30
d90cfde1bb feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1) (#958)
* feat(dub): per-language translations + per-track caches — switching languages stops destroying work (P1)

Multi-language dubbing translated per language (#957) but stored everything
in single-slot state, so tracks silently destroyed each other's work:

P1.2 — per-language translation storage (additive):
- Frontend keeps every translation in s.translations[langCode] alongside the
  legacy s.text slot (still = the shown language). Translate All writes both;
  the new store action switchDubLangCode swaps text through the map on a
  user-driven language switch (non-destructive; restore paths keep the plain
  setter); manual edits / restore-original update the current language's
  entry; merge joins per-language texts, split drops them. Rides project
  save/load inside dubSegments — legacy projects behave exactly as before.
- Backend mirrors it as job["segments_i18n"] = {lang: {segKey: text}}
  (segKey = stable id, index for id-less legacy rows), written by
  _sync_job_segments; job["segments"] stays byte-identical for every existing
  consumer. /dub/srt|vtt?lang= and subtitle burn-in now emit THAT language's
  text when present — ExportModal's "all dubs" batch stops producing N
  identical files. Legacy jobs without the field fall back to today's output.

P1.3 — per-track WAV cache + fingerprints:
- Per-segment WAVs are language-keyed (seg_{lang}_{id}.wav). The partial-regen
  read path falls back to legacy seg_{id}.wav ONLY while the job has no
  other-language track — single-language jobs keep their whole on-disk cache;
  multi-track jobs stop splicing the last-generated language into the current
  track. Read-only endpoints (segment preview, clips zip) gained ?lang= with
  the permissive legacy fallback they always had.
- Fingerprints include the track language (segment_fingerprint(track_lang=…),
  /tools/incremental lang=…) and live in job["seg_hashes_by_lang"]; the flat
  job["seg_hashes"] stays as the current track's mirror so the done event,
  history restore and older frontends read it unchanged. A legacy flat map is
  attributed to the job's last-generated language (dropped when unknown) and
  reads stale once — the safe direction. seg_wav_kind is per-track too.
- The frontend stores fingerprints per language and judges "Regen N changed"
  against the ACTIVE track; project save/load and dub-history restore carry
  all tracks' hashes (segHashesByLang / seg_hashes_by_lang, additive).

Tests: fail-before regression coverage — two-track regen never splices the
other language's audio (sample-level assert on the mixed track), legacy
single-track cache reuse + multi-track gate, per-lang seg_hashes with flat
mirror + migration semantics, /dub/srt|vtt?lang= emitting different text per
track with legacy fallbacks, per-lang burn-in, /tools/incremental lang
scoping, and 14 frontend tests for translations round-trips, per-track
fingerprints and legacy-project behaviour. Full backend + frontend suites,
typecheck, lint and format:check green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add per-language storage + per-track caches under [Unreleased] (#958)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 03:05:40 +05:30
bd92729623 feat(dub): multi-language generate translates each language first + picks persist (P1) (#957)
* feat(dub): multi-language generate translates each language first + picks persist with the project

P1.1 — the "Generate N dubs" loop never translated: the backend synthesizes
segment text verbatim, so every multi-language track rendered the same words
and at most one was actually in its language. The loop now runs
translate → generate per pick:

- handleTranslateAll(langOverride?) accepts an optional ISO-code override
  (no-arg Translate All behavior unchanged; a click-event first arg is
  guarded). It resolves true only when a translation actually landed, and
  both it and handleDubGenerate snapshot segments from the store at call
  time — the click-time closures went stale the moment the previous pick's
  translate pass rewrote the segments.
- A pick whose translate fails (request error or all-segments-errored) is
  SKIPPED — never a wrong-language track — the batch continues, and the
  skipped languages are reported in a final toast.
- The redundant first translate is skipped only when pick 1 targets the
  language the editor text is already translated into; every later pick
  always translates.
- Honest progress: the pill shows "Translating → {lang} (i/N)…" before each
  generate, and the header CTA is inert while translating so a re-click
  can't start a second batch (belt: a ref guard in the loop).

P1.4 — multiLangMode/multiLangs move from DubTab component state into the
dub store slice and ride the project save/load payload (exportTracks too).
Additive and back-compat: legacy payloads default to off/empty and leave the
in-session exportTracks untouched (utils/projectState.js).

Tests (fail-before verified: 9 failures on the pre-fix code):
- handleTranslateAll override targets + return semantics + call-time
  segment snapshot (dubTranslateAllOverride.test.jsx)
- per-language translate-before-generate call order, skip-on-failure with
  continuation + skip-report toast, first-pick skip heuristic, unchanged
  single-language path (dubMultiLangGenerate.test.jsx)
- slice defaults/setters/reset, payload round-trip, legacy-payload defaults,
  App.jsx wiring guards (dubMultiLangPersist.test.js)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add multi-lang auto-translate under [Unreleased] (#957)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:49:41 +05:30
mergetestandClaude Fable 5 afe819f498 fix(ci): oxfmt the two #956 test files (unbreak main format check)
#956 merged with a red Tests gate — my merge script ran unconditionally
instead of aborting on the gate value; the failure was oxfmt-only on the
two new test files. Whitespace-only fix, tests re-verified green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:23:03 +05:30
6b91205036 fix(dub): completed tracks always show their tabs + history keeps its language (P0) (#956)
* fix(dub): completed tracks always show their tabs + history keeps its language (P0)

Root cause chain: the track switcher's visibility expression required
dubLangCode !== 'und' and ended in a tautology (dubTracks?.length > 0 ||
!!dubTracks), so it was effectively keyed to the language dropdown, not
the persisted tracks. History restore always handed the frontend 'und'
because the dub_history language/language_code COLUMNS froze at the
ingest-time "" — the save_job UPSERT never updated them after generation
set them on the job dict (only the job_data JSON carried the real value).
Net effect: a restored project with finished tracks showed no track tabs
until the user re-picked a language.

- DubTab: hasDubbedTrack = done && dubTracks.length > 0 (tracks only;
  also stops the tautology from showing a trackless switcher).
- DubTab auto-jump: membership-guarded — the preview only jumps to a
  language that has a track, else tracks[0]. Kills the preview-404 class
  (restores falling back to 'en' with tracks ['bn'] pointed the player
  at /dub/preview-video?lang=en).
- dub_pipeline.save_job UPSERT: language/language_code now update when
  non-empty (same CASE guard as content_hash), so new saves heal the
  frozen columns and empty re-saves can't clobber them back.
- App.restoreDubHistory: falls back to job_data's language/language_code
  so EXISTING rows in users' DBs restore correctly with no migration.
- P0.2 polish: track pills get duration + timing-strategy tooltips,
  hydrated lazily and failure-silently from the existing
  GET /dub/tracks/{job_id} via new api/dub.dubListTracks; all new
  strings through i18n (en.json).

Tests (fail-before/pass-after): DubTab-level visibility + auto-jump
membership-guard tests (3 of 4 fail pre-fix), pill-tooltip hydration
tests, and save_job language heal/no-clobber tests (heal fails pre-fix).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): open [Unreleased] with the dub track-tabs fix (#956)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:21:23 +05:30
1dfa032be4 feat(dub): project title first in the editor header, pipeline stages below (#955)
Header reordered per owner: row 1 = title (+ duration/segments) with the
action buttons, row 2 = the Upload→Export pipeline spine directly beneath
with a tight 2px gap (was: stepper and title side-by-side on one row).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 01:13:15 +05:30
fd7d20fe1e release: freeze v0.3.10 — version bump, lockfiles, changelog (#954)
package.json (source of truth) + the three mirrors -> 0.3.10, in lockstep;
Cargo.lock/uv.lock/bun.lock regenerated (one line each; bun --frozen-lockfile
verified). CHANGELOG [Unreleased] -> [0.3.10] — 2026-07-05 with the release
headline; nine fixes since v0.3.9, mostly same-day field-report turnarounds.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:57:17 +05:30
27ff5846c2 fix(audiobook): chapter render no longer crashes on mixed 1-D/2-D audio chunks (#897) (#953)
* fix(audiobook): chapter render no longer crashes on mixed 1-D/2-D audio chunks (#897)

Root cause: synthesize_chapter (backend/services/audiobook.py) built
inter-span pause silence as bare 1-D torch.zeros(n) while every real
engine's synth returns (1, samples) per the TTSBackend.generate contract
(OmniVoice's model.generate(...)[0] included) — so the chapter's final
hard concat in chunked_tts.concatenate_audio_chunks hit
torch.cat with mixed ranks and died with
'RuntimeError: Tensors must have same number of dimensions: got 1 and 2'.
Any chapter containing a [pause] span (Stories/audiobook longform)
crashed; existing tests missed it because their stub synth returned 1-D.
The crossfade branch had the same latent bug for mixed-rank chunks.

Fix, both layers:
- concatenate_audio_chunks now normalizes chunk shapes before any cat
  (_normalize_chunk_shapes): lower-rank chunks gain leading singleton
  dims to the highest rank present, then singleton channel dims
  broadcast to the widest channel count (mono follows stereo). Covers
  both the hard-cut and crossfade branches; homogeneous input passes
  through untouched, so all-1-D / all-2-D callers keep their exact
  output shapes. No future backend's output rank can re-break the join.
- synthesize_chapter materializes silence AFTER the loop, matching the
  rendered audio's channel dims / dtype / device — the same pattern
  generation.py's _render_with_pauses already uses for the single-shot
  path — so the data is rank-consistent at the source too. A
  silence-only chapter stays 1-D float32 as before.

Regression tests: mixed-rank hard-cut (both orders), mixed-rank
crossfade, mono->stereo broadcast, all-1-D/all-2-D shape stability, a
2-D-engine + [pause] chapter through synthesize_chapter (the exact #897
scenario), and a spy asserting the parts reaching the concat are
rank-homogeneous. All fail before the fix with the reported error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the audiobook pause-span concat fix under [Unreleased] (#953)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:40:35 +05:30
d14b37fab2 fix(dub): the speaker-count hint is honored on every diarization path + clone-purity guard (#952)
* fix(dub): the speaker-count hint is honored on every diarization path + clone-purity guard

The dub "Speakers" count reached _diarize() and then died on 3 of its 4
branches, so setting it changed nothing, speakers blended, and auto-clones
were cut from mixed-speaker audio ("made up" voices):

- FunASR inline-turns shortcut returned before the hint was ever consulted
  → now an explicit num_speakers routes the job through pyannote (the one
  engine that honors an exact count); turns stay the fast path only when no
  hint is set, and remain the fallback (with an honest "hint ignored"
  warning) when pyannote can't load or crashes mid-run.
- pyannote-unavailable fallback used a hardcoded 2-speaker silence-gap
  heuristic → assign_speakers_heuristic now takes num_speakers and cycles N
  labels on gap boundaries (1 → single speaker; None → legacy alternation),
  and the existing diarization warning says the hint is only approximately
  honored.
- pyannote-crash fallback dropped the hint the same way → same treatment.

No branch drops the hint silently anymore: every degraded path extends the
existing `warning` SSE payload (detail + a machine-readable speaker_hint
field) that the frontend already renders.

Parity + purity:
- POST /dub/transcribe/{job_id} (the CLI's endpoint) gains the same clamped
  num_speakers query param, forwarded to pyannote and the heuristic; the
  omnivoice-dub CLI gains --speakers N.
- Clone-purity guard: _pick_reference_slices rejects sub-1.5s slices, prefers
  slices not temporally adjacent (<0.3s) to another speaker's turn (scoring
  preference, not a hard filter), and extract_speaker_clones skips extraction
  entirely when labels came from the heuristic (labels_source kwarg threaded
  from _diarize; missing kwarg keeps the old behavior) — with a user-facing
  warning pointing at Settings → Models → pyannote.

Tests: fail-before/pass-after coverage in tests/test_speaker_hint.py (all
four _diarize branches driven through the real SSE stream), clone-purity
guards in tests/test_speaker_clone_purity.py, heuristic hint semantics in
tests/test_segmentation.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the speaker-hint + clone-purity fix under [Unreleased] (#952)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:53:47 +05:30
edf86c1800 fix(ui): dub timeline boxes can no longer flash invisible during playback (#373 class, completes #381) (#951)
* fix(ui): dub timeline boxes can no longer flash invisible during playback (#373 class, completes #381)

Root cause: the segment lane animated an inline
`transform: translateX(-scrollLeft)` on every playback tick, so Chromium
promoted it to a compositor layer — and on some Windows GPU/WebView2
driver combos, composited semi-transparent paints (the 0.45-alpha box
fills) flash invisible/visible while the layer moves, settling only when
paused. PR #381 removed `will-change` and raised the alpha, which only
dampened the symptom; the animated transform kept the lane composited.

Fix the class — no composited translucent paints on the lane, ever:

- Position boxes in pure layout: the lane transform is gone; each box's
  `left` is start·pxPerSec − scrollLeft (viewport coordinates). The
  virtualization window already derives from the same scrollLeft, so
  windowing stays consistent by construction. The selfScroll WebKit
  fallback keeps lane coordinates (its viewport is a real scroll
  container), unchanged.
- Belt-and-braces: REGION_COLORS are now fully opaque — each entry
  pre-blends the old 45% tint against the surface behind the lane
  (`--chrome-bg`, the .studio-panel background) via color-mix, which is
  pixel-identical to the previous alpha compositing (0.45·tint + 0.55·bg)
  in every theme, with zero alpha.

Regression tests (fail on pre-fix code): lane carries no transform at
rest and after a scroll update, box lefts are viewport-relative for a
scrolled view, no double-shift in the selfScroll fallback, and every
REGION_COLORS entry is alpha-free with the 45% ratio preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the timeline-box compositor fix under [Unreleased] (#951)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:41:16 +05:30
437a995a0f fix(translate): Cinematic/Autofit can no longer invent dialogue — divergence guard + pinned temperature (#950)
Root cause (v0.3.9 field report): the refine paths had no-op output guards.
Cinematic's ADAPT step only checked _looks_like_target_script, which returns
True unconditionally for every Latin-script target (no _SCRIPT_RANGES entry)
— so any non-empty LLM reply (hallucinated dialogue, refusals, commentary,
or the REFLECT critique itself) shipped as the dub line. Autofit's
adjust_for_slot accepted ANY non-empty reply, and its best-candidate tracker
(closest rate_ratio to 1.0) actively selected the most-padded output, while
_EXPAND_PROMPT invited invention with no ceiling. Both call paths also ran
at the provider-default temperature 1.0, unlike the working Fast path which
pins 0.2.

The fix, class-level:
- Shared divergence guard translator.refine_output_ok (length window
  0.4–2.5x, env-tunable via OMNIVOICE_REFINE_RATIO_MIN/MAX, with an
  absolute cap for short references; target-script check; critique-echo
  detection). Rejected ADAPT output degrades to the literal with
  error="adapt-diverged" (wrong-script keeps its adapt-wrong-script:<lang>
  marker), riding the existing degradation machinery unchanged.
- Autofit validates every reply against the ORIGINAL input text (divergence
  compounds across attempts otherwise); rejected candidates are discarded
  (attempt burned, graceful degradation to the input preserved) with
  error="fit-diverged"; lines under 15% of their slot skip LLM expansion
  entirely (fit-skip-short) — they could only "fill" the slot with
  fabricated dialogue.
- temperature=0.2 pinned on the cinematic (_chat) and fit (llm.chat) calls;
  chat/chat_messages gained an optional temperature param that is only sent
  when set, so refinement/director/glossary callers keep provider defaults.
- Prompts hardened: ADAPT forbids introducing facts/names/dialogue not in
  the source line; EXPAND forbids inventing information and more than
  doubling the line.
- speech_rate strict-mode docstring made honest: strict changes only the
  upper tolerance bound; expansion still runs (now guard-bounded).

Fail-before/pass-after regression tests for the reported bugs (10x runaway
ADAPT on an es target, critique echo, hallucinated slot-fill expansion,
refusal replies, tiny-line expansion skip, pinned temperature) plus the
previously-untested wrong-script fallback and legit-output acceptance.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:28:37 +05:30
03eb8f7d04 docs(readme): community table lists the Discord server's real channels (#949)
The table described aspirational channels (#showcase/#help/#feature-requests/#dev)
that don't exist on the server; it now matches reality (#announcements,
#releases+#changelog, #issues and #ideas forums, #discuss-ideas, #general).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 21:39:12 +05:30
beca8cfea8 fix(launcher): replace a stale old-version backend instead of attaching to it (#947)
* fix(launcher): replace a stale old-version backend instead of attaching to it

After an update, an orphaned backend from the PREVIOUS version can survive
holding the port. It still answers /system/info, so both attach paths
(lib.rs launch bootstrap + bootstrap.rs retry) treated it as "already
serving OmniVoice — attaching" and the new UI silently ran OLD backend
code: every fix in the update appeared to change nothing. Reported on
Discord as "a bound port which blocked the newer versions"; the app
already knew how to kill_orphan_on_port on both Unix (lsof) and Windows
(netstat) — it just never applied it to a healthy-but-stale backend.

The attach decision now compares versions: running_backend_version()
reads app_version from /system/info (string-sniff, no new deps), and
same_app_version() compares BASE versions (pre-release -N suffix
stripped, so a preview build 0.3.10-4 still attaches to its 0.3.10
backend). Same version → attach exactly as before. Different or missing
version → the orphan is killed and the bundled backend spawns. Foreign
processes keep the existing port_in_use take-ownership path; the
post-spawn health polls are untouched (we spawned that backend
ourselves).

Rust unit tests cover the /system/info parse shape and the
match/preview/stale/unversioned decisions; 51 pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the stale-backend port-reclaim fix under [Unreleased] (#947)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:40:42 +05:30
32103ad7b7 docs(readme): charm + organization overhaul (Opal-style) — collapsibles + OpenAI-compatible API section (#945)
* docs(readme): charm + organization overhaul (Opal-style)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): restore inventory-exact feature names (docs-drift guard)

The charm pass sentence-cased five bold leads in the collapsed feature
list; scripts/check-docs-drift.py greps for the inventory's exact
title-case names. Restored: Vocal Isolation, Speaker Diarization,
Batch Queue, AI Watermark, GPU Auto-Detect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(readme): dubbing screenshot shows a real completed dub (37 segs, EN→BN)

Replaces the empty drop-zone shot with the populated editor — video +
waveform + cast, 37 Bengali segment rows, DUB COMPLETE banner — captured
live from the v0.3.9 app; caption updated to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:36:34 +05:30
46c4445258 fix(ui): relative timestamps no longer render as "20617d ago" (seconds-vs-ms class) (#946)
Backend rows store timestamps as Unix SECONDS (time.time() REAL columns:
generation_history, dub_history, exports, longform jobs, projects), while
frontend-local records carry milliseconds (Date.now() story projects) or ISO
strings (transcriptions). Projects/OmniDrive fed the seconds straight into a
millisecond-based diff (fmtTime), so every generation-history card rendered
as ~1970 ("20617d ago") — and, because the same raw value drove the recency
sort, history items also sank to the bottom of the drive.

Fix the class, not the label: a single shared, unit-tolerant normalizer
(frontend/src/utils/relativeTime.js) now backs every relative-time call site.

- toMillis(ts): numbers < 1e12 are seconds (×1000), >= 1e12 already ms; ISO
  and numeric strings parse; Date instances pass through; null/0/undefined/
  garbage -> null. Backend storage format is untouched (backward compat).
- timeAgo(ts): "—" for missing stamps (never an epoch age), "just now" for
  future stamps within 1 min of clock skew, s/m/h/d buckets, short absolute
  date beyond 7 days.
- absoluteTime(ts): unit-safe tooltip text, '' when missing (no more
  "Jan 1 1970" titles on null rows).

Converted call sites: pages/Projects.jsx (drop local fmtTime + per-source
*1000 juggling; normalize ts once so sort and label agree), components/
Sidebar.jsx + components/WorkspaceProjects.jsx (drop duplicated local
timeAgo copies and caller-side *1000), pages/BatchQueue.jsx (drop local
formatAge; missing created_at used to render an epoch date), pages/
Transcriptions.jsx + components/TranscriptionPicker.jsx (parse via
toMillis, keep their i18n labels; unparseable stamps no longer render
"Invalid Date").

Tests (fail-before/pass-after): utils/relativeTime.test.js covers seconds/
ms/ISO/numeric-string/Date inputs, null/0 -> "—", clock-skew "just now",
and the 1970 regression (a seconds stamp from today must not render as
thousands of days ago); test/ProjectsRelativeTime.test.jsx guards the
OmniDrive wiring end-to-end (seconds created_at renders "2h ago", null
renders "—", mixed-unit sort orders by real recency). Full frontend suite:
106 files / 843 tests green; oxlint, oxfmt, typecheck:ci, node:test green.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 20:03:41 +05:30
8bbab3fcc3 fix(translate): Dub LLM engine runs on the configured LLM provider (new dub_translation skill) (#944)
* fix(translate): the Dub LLM engine now runs on the configured LLM provider

Picking "LLM (OpenAI-compatible)" in the Dub tab read only the raw
TRANSLATE_* env vars — completely bypassing Settings → LLM Providers, so a
provider the user had configured AND tested in-app silently didn't power
the engine (empty key → raw 401 per segment). The Cinematic refiner was
already rewired through LLM Skills (#910/#912); this closes the gap for
direct LLM translation:

* new "dub_translation" LLM skill (Settings → LLM Skills) — per-skill
  provider override → global active provider, same resolution as every
  other skill; disabled == unconfigured, no new degradation modes
* the provider=openai branch resolves through resolve_skill_client();
  TRANSLATE_BASE_URL/TRANSLATE_API_KEY/TRANSLATE_MODEL stay working as
  the power-user override (env-only setups see zero behavior change,
  except the stale gpt-3.5-turbo default is now gpt-4o-mini, matching
  the cinematic path)
* per-segment calls are now bounded by the LLM timeout (45s default via
  OMNIVOICE_LLM_TIMEOUT) instead of the SDK's 600s default
* fully unconfigured → an up-front actionable 400 naming Settings → LLM
  Providers / LLM Skills instead of a per-segment 401
* provider-store keys are resolved into the error scrubber so a provider
  echoing the key can't leak it (parity with the env-key scrub)
* translation_engines registry: honest notes + a configured/configured_via
  stamp on LLM entries so the Engine dropdown can show ready-vs-needs-setup
  before the user clicks Translate

Tests: 4 new (skills-resolved client wins with its model+timeout; 400s
name the right settings page for no_provider vs disabled; env fallback
keeps working incl. TRANSLATE_MODEL); skills registry coverage updated;
existing openai-branch tests routed deterministically through the env
branch via the shared fake helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): add the dub-translation provider wiring under [Unreleased] (#944)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:22:12 +05:30
9ff5f38cb3 docs(readme): cross-promote the maker's other local-first projects (Opal, memxt) (#943)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:09:46 +05:30
359c70bc21 docs(readme): use the Launchpad screenshot as the hero image (#942)
Swap the static social-preview banner for the live v0.3.9 Launchpad shot
and drop the now-duplicate Launchpad row from the gallery (shown once).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 18:56:21 +05:30
0579ec91ad docs(readme): Opal-style restyle + fresh v0.3.9 screenshots (#937)
- Emoji section headers with explicit <a id> anchors. Emoji breaks
  GitHub's auto-generated heading slugs, so every in-page nav target keeps
  a stable explicit anchor (verified all href="#..." resolve).
- Refresh the screenshot gallery. The prior set was from April, predating
  the launchpad / settings / dictation UI overhaul, so it misrepresented
  the app. Captured fresh at retina from the live v0.3.9 UI and led the
  gallery with the new Launchpad home: launchpad, studio, voice design,
  voice gallery, dubbing, engine-compatibility matrix, model store,
  embedded API reference (Scalar), and the in-app changelog reader.
- Fix the stale engine count (11 -> 14 TTS engines) in the comparison
  table, FAQ, and roadmap to match the engine table + backend registry.
- Use <kbd> keycaps for the dictation shortcut (Opal detail).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 18:38:47 +05:30
b6ec4e23f3 fix(engines): snapshot lazy registry keys so /engines can't 500 under concurrency (#940)
* fix(engines): snapshot lazy registry keys so /engines can't 500 under concurrency

`list_backends()` runs in a FastAPI threadpool and iterates the lazy TTS/ASR
registries via `items()` → `__iter__`, which held a *live* `dict.__iter__(self)`
open across each engine's slow `is_available()` probe. Meanwhile the lazy
`__getitem__` resolves a deferred entry by mutating the dict (`self[key] = cls`).
A second concurrent `/engines` request (or any ASR op) materializing the lazy
`faster-whisper-isolated` entry therefore changed the dict size mid-iteration:

    RuntimeError: dictionary changed size during iteration
      asr_backend.py:1729 list_backends → _REGISTRY.items()
      asr_backend.py:1665 __iter__ → for k in dict.__iter__(self)

Both `_LazyRegistry` (TTS) and `_LazyASRRegistry` (ASR) now snapshot their live
keys up front with `list(dict.__iter__(self))` — consumed atomically under the
GIL — so a concurrent lazy insert can no longer trip the iteration. The slow
per-engine probes then run over the snapshot, not the live iterator.

Deterministic fail-before/pass-after regression for both registries:
tests/backend/services/test_lazy_registry_concurrency.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(changelog): add the /engines concurrency fix under [Unreleased] (#940)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 16:53:57 +05:30
eb188931b5 fix(dub): classify EINVAL transcribe failures so they stop dead-ending (#763) (#936)
A per-chunk temp-WAV write that fails with OSError EINVAL ("[Errno 22]
Invalid argument") — a missing/read-only/full temp dir, a removed drive,
or antivirus — collapsed into "Transcription produced no segments.
[Errno 22] Invalid argument" with no next step. classify() now names the
class (OS_INVALID_ARGUMENT) so build_failure attaches an actionable
temp-dir/disk/AV hint at the exact surface the streaming dub path already
feeds it (dub_core.py:672) — same treatment the ffmpeg and compute-type
classes get. Fail-before/pass-after regression added; the errno-22 token
keeps it from colliding with the errno-2 transformers-import class.

Also stamps the [0.3.9] CHANGELOG section with today's release date
(2026-07-04) ahead of tagging.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 05:35:51 +05:30
86f23326ff docs(changelog): add sherpa config-error fix (#919) to [0.3.9] (#935)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:27:42 +05:30
ba4f64240a fix(engines): classify sherpa "model not set" as a config error, gate the engine on its model dir (#919) (#934)
A user selected the sherpa-onnx TTS engine and got a 500 that read "TTS
engine stopped mid-generation. This usually means it ran out of memory.
Try the Flush button…" — when the real cause was a pure setup problem:
"OMNIVOICE_SHERPA_MODEL not set. Point it to a sherpa-onnx TTS model
directory (containing model.onnx + tokens.txt)." Same misclassification
class as #880/#893, which tightened the OOM catch-all on the generation
path — but the engine-not-configured case still fell through to memory.

Two layers, fixing the whole class:

1. Error classification (backend/api/routers/generation.py): a new
   `_is_config_failure()` recognizes "required engine model path / env
   var not set" over the whole exception chain (OMNIVOICE_* named with
   "not set"/"point it to"/"set omnivoice_…", sherpa's "no model.onnx
   found in", "not configured", "venv not found. set" for the dedicated-
   venv opt-ins). `_oom_friendly_reraise` checks it BEFORE the OOM branch
   and re-raises actionable setup guidance that names the variable, points
   at Settings → Engines, and never mentions memory or Flush. Generalizes
   to sherpa/Confucius4/dots/MOSS and any future env-gated engine.

2. Engine gating (backend/services/tts_backend.py): SherpaOnnxBackend
   ships no bundled model, so is_available() now gates on
   OMNIVOICE_SHERPA_MODEL (set + contains model.onnx) — like the other
   path-configured opt-in engines — returning False with an actionable
   reason instead of "ready", so the picker marks it unavailable-with-a-
   reason rather than selectable-but-broken. Added the copy-paste setup
   snippet for the Compat Matrix. Backward-compatible: a correctly
   configured OMNIVOICE_SHERPA_MODEL keeps the engine available.

Tests (fail-before/pass-after): config-classification of the sherpa
"model not set" error and the wider not-configured class (no "out of
memory"/"Flush"); is_available gating on the env var + model.onnx and the
setup-snippet registration.

Fixes #919

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:26:58 +05:30
8ca7de89eb fix(test): close the reload-induced test-isolation leak at its source (#932 follow-up) (#933)
Two sides of the same class the router-smoke leak (#932) traced to
test_pronunciation_api's importlib.reload teardown:
- test_pronunciation_api now re-runs init_db() on the restored data dir so
  the reloaded core.db/main.app is never left on a schema-less DB.
- test_db_migration_safety catches db_module.MigrationError dynamically
  instead of the collection-bound name, so a reload that rebinds the class
  can't make pytest.raises miss it.
Both orderings (real + reversed) now pass; no product change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:07:44 +05:30
ed0b0b63cf fix(test): router-smoke tests leak-proof against full-suite order (#932)
tests/test_router_smoke.py showed ~10 `sqlite3.OperationalError: no such
table: jobs` failures in the full suite (and in isolation on a clean data
dir), but passed when a schema-creating module ran first.

Root cause: the `client` fixture builds a bare `TestClient(app)` with no
`with` block, so the FastAPI lifespan never runs — and `init_db()` (the
only place the schema is created) lives in that lifespan (main.py). The
smoke tests therefore free-rode on whatever schema an earlier module left
on the active DB. A module that reloads `core.config`/`core.db` and leaves
`core.db.DB_PATH` pointed at a fresh, schema-less DB (test_pronunciation_api's
`importlib.reload` teardown restores the env var but never re-runs init_db
on the restored data dir) strands router-smoke on a DB with no tables ->
every DB-backed route 500s. Same class as #878 / #917.

Fix (test-only, zero blast radius): the `client` fixture now calls
`core.db.init_db()` against whatever DB is active at run time before serving
requests — the same `init_db()` pattern test_api.py / test_personas_api.py
use. Because it targets the live `core.db.DB_PATH`, it re-creates the schema
regardless of which path any prior module left active, making the suite
self-sufficient and order-independent.

Verify:
- `pytest tests/test_router_smoke.py` alone: 10 failed -> 24 passed
- `pytest tests/test_pronunciation_api.py tests/test_router_smoke.py`
  (deterministic reproducer): 10 failed -> 38 passed
- `pytest tests/` full suite: 2215 passed, 20 skipped, 10 xfailed,
  4 xpassed, 0 failed / 0 errors

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:47:45 +05:30
0d80ab2cb0 docs: backfill [0.3.9] batch bullets + add OSS sponsorship playbook (#931)
Bullets for #922 (release titles), #923+#924 (sponsors), #925 (contact),
#927 (models), #928 (openapi), #930 (engines) — the agents kept off
CHANGELOG.md during the merge chain. Plus a portable how-we-set-up-
sponsorship playbook for reuse on other projects.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:30:43 +05:30
6e3e014826 feat(settings): OpenAPI reference page — embedded Scalar (bundled, CDN-free) + footer button (#928)
Add a Settings → OpenAPI page that renders an interactive Scalar API
reference for OmniVoice's own local backend, plus a compact footer button
that opens it.

- New OpenApiPanel fetches the live spec from the resolved backend base
  (getApiBase()+"/openapi.json", via apiFetch so it follows remote-backend /
  LAN-share overrides), owns loading + unreachable-backend fallback (with
  Retry), and hands the parsed spec inline to Scalar.
- Scalar is bundled via the @scalar/api-reference-react npm package — NO CDN
  script tag. It is lazy-loaded (ScalarApiReference.jsx) so its ~heavy Vue
  bundle stays out of the initial load and only downloads when the page opens.
- CDN-free hardening: withDefaultFonts:false (drops the fonts.scalar.com
  @font-face rules), proxyUrl:'' (Test Request client goes direct to the local
  backend, not proxy.scalar.com), spec passed as inline content (no external
  spec fetch). The Tauri CSP is the hard backstop. Verified the built dist:
  external hosts appear only as inert/gated strings inside the on-demand Scalar
  chunks and are absent from the initial-load chunks.
- settingsCategories: new 'openapi' category (Braces icon, api/openapi/scalar/
  rest/swagger/docs keywords) in the System group; Settings render case wired.
- LogsFooter: compact Braces icon button (openSettingsTab('openapi')) next to
  the discord/mail cluster, chrome-muted → accent on hover, uniform 14px icon.
- i18n: all strings via t() with English defaultValue fallbacks (openapi.*,
  logs.open_api*, settings.openapi); keys added to en.json.
- Test: OpenApiPanel.test.jsx (mocks the spec fetch + stubs Scalar) — renders
  the reference container on success, shows the unreachable fallback on failure,
  recovers on Retry.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:27:05 +05:30
5bd8968aea feat(engines): real synthesis "Self-test" + copy-paste setup snippet for opt-in engines (#930)
* feat(engines): real synthesis "Self-test" + copy-paste setup snippet for opt-in engines

Builds on #905's Engines-settings fixes (verified still green: license dialog
mounts, matrix reloads on select, cpu_fallback routing toast, cpu-native →
cpu_only). Two enhancements, no #905 behavior touched.

Real "Self-test" for in-process TTS engines
-------------------------------------------
The existing /engines/{id}/health probe only imports the package and reports
"deps OK" for in-process engines — it never proves the engine can emit audio.
New POST /engines/{id}/selftest runs a *tiny real synthesis* from a fixed short
ASCII phrase and reports ok + duration + sample-rate + sample count, proving the
engine actually produces audio. Guardrails keep it cross-platform-identical and
CPU-cheap: TTS + available + in-process only, bounded wall-clock timeout
(OMNIVOICE_SELFTEST_TIMEOUT_S, default 90s) that returns ok=false/timed_out
instead of hanging the panel, a process-wide lock so a click-storm can't stack
model loads, loopback-gated, and only ever on user click (never on load). The
Compat Matrix gains a "Self-test" button (with cooldown) that renders
"0.82s @ 24 kHz in 820 ms". HF tokens in a synth error are redacted like the
health route. Verified end-to-end: kittentts synthesized 89,200 samples @ 24 kHz.

Copy-paste setup snippet for path-gated opt-in engines
------------------------------------------------------
IndexTTS / MOSS-v1.5 / dots.tts / Confucius4 gate on an OMNIVOICE_*_DIR env var.
list_backends() now emits a single-sourced `setup_snippet` (the exact
`export VAR=/path/...` line) surfaced with a Copy button inside the matrix's
"Why unavailable?" disclosure, so users don't reconstruct it from the docs.

Also tightened the incomplete SelectEngineResponse TS type to include the
routing echo (routing_status/effective_device/routing_reason) the post-select
toast already reads at runtime.

Tests: backend selftest success/subprocess-reject/unavailable/unknown/loopback/
exception-capture/timeout/HF-redaction + setup_snippet shape; frontend self-test
render, timeout marker, subprocess+ASR gating, setup-snippet render. New route
added to the API route snapshot. Full vitest (808) + backend engine/routing/asr/
route-inventory/no-CJK green; lint 0 errors; format + typecheck:ci clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(test): allow setup_snippet key in list_backends shape assertion

The engine self-test PR added setup_snippet to each backend entry but only
updated the route-shape test; test_list_backends_shape strict-asserts the key
set. Add setup_snippet there too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:02:08 +05:30
48a7154810 feat(models): one canonical HF-token path + surface incomplete cache (#927)
Two Model-management enhancements building on #908 (no re-do of its fixes).

Unify the two HF-token entry points. The Model Store toolbar saved the
token via /system/set-env (env var + HF-CLI file) while Settings →
Credentials saves to the encrypted app store — two stores with an
asymmetric clear path, so a toolbar-set token silently outlived the
Credentials "Clear" (a support-ticket generator). The toolbar now POSTs
the SAME canonical endpoint Credentials uses (/api/settings/hf-token →
encrypted store + huggingface_hub.login()), so there is one store with
one clear path. In-process parity is preserved (login() populates the HF
canonical file, so downloads pick it up immediately).

Surface an incomplete/partial cache. A truncated download (config landed,
weight shard didn't) occupies disk but used to read as a plain "not
installed". The backend already flags it as `incomplete`; the row now
shows an "incomplete · N MB" warn badge, relabels the primary action to
"Repair" (re-runs snapshot_download to finish the missing shard), and
offers a Delete to clear the partial bytes.

Tests: modelStoreTokenPath (toolbar hits /api/settings/hf-token, never
/system/set-env) + modelStoreIncomplete (badge, Repair→onInstall, Delete,
no false positives on normal not-installed/installed rows). Full vitest
green; lint + format clean. i18n keys added to en.json (models.incomplete,
incomplete_title, repair_btn, repair_title).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:59:43 +05:30
5e135b9655 feat(contact): make "Get in touch" a guided, well-typeset help page (#925)
* feat(contact): make "Get in touch" a guided, well-typeset help page

Replace the flat 4-row link list (Discord / Email / Issues / Website) with
five guidance cards, each an icon + heading + a "use this when…" sentence so
users pick the right channel instead of guessing:

- Report a bug → reuses ReportBugButton (prefilled GitHub issue + scrubbed
  diagnostics; nothing sent until the user reviews & submits)
- Request a feature / ask → GitHub Issues
- Get help & community → Discord (setup help, sharing dubs)
- Support the project → routes to the existing Support page (no Ko-fi
  duplication)
- Report a security issue → GitHub Security Advisories (private, per
  SECURITY.md)

Bigger, friendlier typography ("We'd love to hear from you" header, roomier
measure and spacing) and a container-reflow card grid (CSS grid auto-fit, no
viewport @media, so it stays correct under --ui-scale zoom). Email + website
kept as quieter direct channels. External CTAs are real <a rel="noreferrer">
links, keyboard-focusable, opened via the shared openExternal helper. All
strings go through i18n under contact.* with English defaultValues for locale
fallback.

Adds a ContactPage render test (sections render, each channel targets the
right URL, bug-report affordance present, Support routes to donate).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(i18n): prune 4 orphaned contact.* keys from 20 locales (Contact-page rewrite)

The Contact page rewrite renamed its i18n keys; the old keys lingered in the
20 non-English locales as orphans, failing the locale_no_orphan_keys probe.
Pruned; new keys fall back to English per i18n config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:59:30 +05:30
eeb3bf4452 feat(support): sponsor logo slot + "Become a sponsor" affordance (#923)
Adds a way for companies and people to visibly support OmniVoice.

- config/sponsors.js: single source of truth — an (empty) SPONSORS array with
  a documented { name, logoUrl, url, tier } shape + tier order, and a
  SPONSOR_CONTACT object whose githubIssue is a prefilled, zero-token
  "become a sponsor" issue (same pattern as the bug reporter) plus the Ko-fi
  link and a SPONSORS.md docs URL. Logos are added here + in SPONSORS.md.
- LogsFooter: a compact "Sponsors" link next to the donate heart (a link, not
  a logo strip in the 28px bar) that opens the in-app Support/Sponsors view.
- SupportPage: a Sponsors section — logo grid grouped by tier when populated,
  a tasteful outlined "be the first — your logo here" slot while empty, a
  primary "Become a sponsor" button opening the prefilled issue, and a
  one-line explainer linking to SPONSORS.md.
- SPONSORS.md: what sponsors get + how to become one, kept in lockstep with
  the config.
- All strings via i18n (support.sponsors_* / logs.sponsors) with English
  defaultValues so non-English locales fall back cleanly. Logo links are lazy,
  max-height capped, aria-labelled, rel="noreferrer", and open in the system
  browser via the app's external-open helper.
- Test: SupportPageSponsors renders the empty placeholder + asserts the
  become-a-sponsor CTA targets the contact URL, and (with injected sponsors)
  that each renders as an external logo link.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:46:03 +05:30
126f23fd3e docs(sponsors): add SPONSORS.md, README sponsors section, sponsor issue form + FUNDING link (#924)
Add a sponsorship home (SPONSORS.md) with Backer/Bronze/Silver/Gold tiers —
described as placements/benefits, with $ amounts left as `<!-- OWNER: set
amounts -->` placeholders (no invented prices). Primary "become a sponsor"
path is a prefilled GitHub issue form (.github/ISSUE_TEMPLATE/sponsor.yml:
name/org, logo URL, tier, contact), with Ko-fi/PayPal as direct paths and an
OWNER placeholder for a public contact email.

README gains a Sponsors subsection (logo-slot placeholder + SPONSORS.md link),
a Sponsors nav entry, and a note about GitHub's native Sponsor button.
FUNDING.yml adds the SPONSORS.md link alongside the existing ko_fi/PayPal.

Keeps the honest "agent bills" framing; sponsorship is a thank-you, not a
paywall — OmniVoice stays fully free and AGPL-3.0. Docs-only; no fabricated
sponsors, prices, or testimonials.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:33:11 +05:30
85b0db65bc ci(release): version-first release titles so the tag shows in GitHub's truncated release list (#922)
GitHub's release-list sidebar clips the title mid-string, hiding the version
when it trails 'OmniVoice Studio'. Name stable releases 'vX.Y.Z — OmniVoice
Studio' and the preview 'Preview — OmniVoice Studio'. Existing releases were
renamed to match.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:24:27 +05:30
45a5a5ce21 docs(changelog): add Launchpad full-width (#915) + migration-logging fix (#917) to [0.3.9] (#918)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:13:53 +05:30
d57ecea804 fix(test): DB migration-safety tests leak-proof against full-suite order (#909 follow-up) (#917)
The PR #909 data-safe-update tests passed in isolation but failed only in
full-suite CI order. Two independent, order-dependent leaks were at play:

1. Module-identity leak (the #878/#894 class). The `isolated_db`/`fresh_app`/
   `fresh_resolver` fixtures in tests/backend/** purge `core.*`/`services.*`
   from `sys.modules` and never restore them, so `sys.modules["core.db"]`
   afterward is a DIFFERENT object than the one the migration-safety tests
   imported at collection. `monkeypatch.setattr("core.db.DB_PATH", ...)`
   re-resolved the dotted string to the re-imported module, while
   `_run_alembic_upgrade`/`init_db` (bound at collection) kept reading the
   ORIGINAL module's globals — so the patch missed and the upgrade ran against
   the ambient session DB. Result: no backup at the asserted path, and the
   mid-flight-failure injection never hit the expected DB (DID NOT RAISE).
   The same divergence hit the lazy `from core import db_backup` inside
   `_run_alembic_upgrade`, so patching `MAX_BACKUP_DB_BYTES` was silently lost.

2. Logger-disable leak. Alembic's env.py called `fileConfig(...)` with the
   default `disable_existing_loggers=True`, which disabled the already-created
   `omnivoice.db.backup` logger the first time any earlier test ran a real
   `alembic upgrade` — so the oversized-DB "Skipping pre-migration DB backup"
   line was never emitted and the caplog assertion failed. This also silently
   mutes the live app's logging after a real startup migration.

Fixes:
- env.py: `fileConfig(..., disable_existing_loggers=False)` so a migration
  never mutes the app's (or another test's) loggers.
- core/db.py: import `db_backup`/`APP_VERSION` at module level so
  `_run_alembic_upgrade` uses a stable reference immune to a `sys.modules`
  purge, matching what tests patch at collection.
- test_db_migration_safety.py: patch DB_PATH on the imported `core.db` module
  object rather than the re-resolvable dotted string — the correct,
  self-contained seam.

Verified: the four migration-safety tests + the oversized-backup test pass in
full-suite order and in isolation; full `pytest tests/` is green
(2206 passed, 0 failed).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:13:12 +05:30
6feafbd3be fix(launchpad): full-width responsive feature-card grid (retire the fixed ~780px deck) (#915)
PR #904's deck-of-cards fan pinned the seven Launchpad feature cards
(Voice Clone, Voice Design, Video Dubbing, Stories, Audiobook, Voice
Gallery, Transcripts) inside a fixed ~780px box, leaving dead margins on
a maximized display. Replace it with one full-width grid that fills the
content edge-to-edge and reflows its column count from a maximized
~2560px display down to the 900x600 minimum.

- LaunchpadDeck renders a single `.lp-cards` grid at every shell width
  (no deck-vs-fallback split): `repeat(auto-fit, minmax(--lp-card-min,
  1fr))` derives the column count from the grid's OWN width, so columns
  reflow 7->1 with zero viewport @media (which fire at the wrong width
  under the shell's `zoom: --ui-scale` model). Every column stretches
  (1fr) -> no dead margins, no horizontal scroll.
- The only responsive knob is `--lp-card-min`, set inline from
  useShellNarrow (the `.app-container` shell-narrow/shell-mini own-width
  classes): 200px wide, 240px narrow -> fewer, comfier columns on narrow
  shells. No viewport media queries.
- Cards keep #904's character: animated waveform faces, cursor
  spotlight + eternal breath ring (phase-offset per card via --lp-i),
  and a hover/focus-forward raise (`lp-action-card--raised`) driven from
  React state so pointer and keyboard share one path. Reduced-motion
  freezes the waveform. All 7 navigation targets and i18n keys preserved.
- Removed the old `.lp-deck*` fan CSS, the ActionCard narrow fallback,
  and the launchpad viewport @media overrides. Rewrote the regression
  suite to assert full-width grid layout, the narrow-vs-wide floor, and
  the raise interaction for pointer AND focus.

Verified in a real browser (chromium): 7 cards fill the full width in
one row at 2472px content, reflow to 3 columns at 920/876px, and the
grid width equals the container at every size (no overflow).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:35:00 +05:30
8f71c90f20 feat(settings): LLM Skills — per-feature enable/route control for every LLM call (#912)
New Settings → System → LLM Skills area: every LLM-powered capability
(Cinematic & Autofit translation, speech-rate slot fitting, glossary
auto-extract, direction parsing, dictation cleanup) becomes a "skill" the
user can toggle or route to a specific provider (local Ollama/LM Studio vs
a remote key) instead of everything riding the one global active provider.

Backend:
- services/llm_skills.py — skill registry + settings_store persistence
  (llm_skill.<id>.enabled / .provider), resolution precedence
  override > active > none, resolve_skill_client() (OpenAI-compat client
  bound to the effective provider; None when disabled/unconfigured) and
  skill_backend() (OffBackend when disabled — the exact no-LLM object every
  caller already degrades on).
- All five consumption points wired through the registry; a disabled skill
  degrades exactly like "no LLM configured" today (Fast translation
  fallback, refinement pass-through, heuristic direction parse, no-llm slot
  fit, 503 on glossary auto-extract). No new degradation modes; defaults
  (enabled + no override) keep existing setups byte-identical.
- OpenAICompatBackend gains an optional bound provider (None = active, the
  historical behavior).
- GET /api/settings/llm-skills + PUT /api/settings/llm-skills/{skill_id}
  (404 unknown skill/provider); route snapshot updated.

Frontend:
- LLMSkillsPanel (Sparkles, next to LLM Providers): one row per skill —
  i18n name/description, enable toggle, provider Select ("Use active
  provider" + configured providers, local ones tagged), ready /
  needs-setup badge linking to LLM Providers. All strings via t()
  (settings.llmskills_*).

Tests: 30 backend (precedence, per-consumption-point disabled semantics,
endpoint round-trips, validation) + 4 panel render/PUT tests. Docs:
translation-engines.md gains an LLM Skills section.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:08:30 +05:30
a481d459bd docs(changelog): backfill the settings/features wave into [0.3.9] (#913)
Nine PRs (#904-912) shipped without their changelog bullets (agents were
kept off CHANGELOG.md to avoid merge conflicts across the wave); this
backfills them per the changelog hard rule.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:07:39 +05:30
af6690840e fix(translate): run Cinematic/Autofit on every engine (incl. default Argos), bound the fit pass, scrub provider errors (#910)
P0 — Cinematic/Autofit silently no-op'd on argos/nllb/openai. Those three
branches returned BEFORE _maybe_cinematic, so only the deep_translator
fall-through reached the refine/fit pass. A user on the DEFAULT Argos engine
who picked Cinematic/Autofit got plain Fast output with a success toast and
no quality_used/cinematic_skipped/rate_ratio. All three now route through
_maybe_cinematic. provider=openai is already an LLM translation, so it skips
the reflect/adapt re-refine (new already_llm flag) but still stamps
rate-ratio badges and runs the Autofit fit pass; the dialect it baked into
its translate prompt is now reported applied.

P1 — the Autofit fit pass ran one blocking adjust_for_slot per segment in the
merge loop, OUTSIDE any budget (a 50-seg dub vs a slow provider spun
~50×timeout unbounded). New speech_rate.adjust_for_slot_many fans it out
concurrently under a wall-clock deadline SHARED with the cinematic refine;
segments still running at the deadline degrade to their literal with
rate_error='fit-budget'. Also set max_retries=0 on the OpenAI clients used
for translate/refine/fit so a 429 + Retry-After can't sleep through the budget.

P2 — glossary auto-extract's no-LLM message now points at Settings → LLM
Providers (was the stale TRANSLATE_BASE_URL/TRANSLATE_API_KEY). Provider error
bodies on the glossary auto-extract, the OpenAI translate-segment path, and the
DeepL/Microsoft translate-segment path are now scrubbed
(core.scrub.scrub_provider_error) — they could echo the API key / a user_id.
DubTab re-polls LLM availability on window focus / visibility so configuring a
provider in Settings lifts the Cinematic gate without a remount. Documented
LLM_DEFAULT_PROVIDER in docs/dubbing/translation-engines.md.

Tests: fail-before/pass-after for argos+cinematic (refine runs), argos+cinematic
no-LLM (cinematic_skipped), argos Fast (rate_ratio stamped), openai+autofit
budget bound, and provider-error scrubbing on the translate + glossary paths.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:53:11 +05:30
75864a597f fix(dictation): refinement never stalls a final (~51s→≤4s), REST polish parity, real ASR preload reuse (#911)
P0 — Refinement blocked every dictation final with no timeout. With refinement
auto:true and a slow/dead LLM endpoint, maybe_refine ran unbounded and blocked
the final send in all three capture_ws handlers (~51s measured; the pill hung
"Transcribing…" until the widget's 15s fallback fired). Fix the class: a hard,
env-tunable budget (OMNIVOICE_REFINE_TIMEOUT_S, default 4s) via a new
maybe_refine_async — a slow/dead endpoint now falls back to the unrefined (but
polished) text within the budget and can NEVER delay the final beyond it. The
LLM HTTP call is bounded to the same budget so the orphaned worker unwinds
instead of holding a connection for the client's full 45s. Refinement is now
also fully best-effort in the legacy handler (it can't turn a good final into
an error frame).

P1 — REST /transcribe lacked polish parity. capture.py never applied
polish_text, so REST returned raw "…test" while the WS returned "…test."
Apply text_polish.polish_text to `text` and `refined_text` (segments stay raw),
so the widget POST fallback and MCP/CLI callers match the live socket.

P1 — The #888 "instant first dictation" preload was a no-op. The preload called
warmup() only `if hasattr`, but SherpaDictationBackend had none, and the WS
handlers built a FRESH backend per session so a warm singleton wasn't reused.
Add SherpaDictationBackend.warmup() (builds the recognizer) and share one warm
recognizer per model id across sessions (get_sherpa_dictation_backend, same
invalidation + a shared lock as the capture singleton); each session keeps its
own decode stream. First dictation no longer pays the 1.3–2.5s load.

P1 — llm_ready is a lie (feeds the P0). It only means "an endpoint is
configured", so a placeholder key reads as ready. The P0 timeout makes a dead
endpoint harmless; add last_refine_status so RefinementPanel flags a
configured-but-failing LLM and links to LLM Providers → Test.

Regression tests (fail-before/pass-after): slow-LLM WS final arrives < budget;
maybe_refine_async hard timeout + status; REST polish parity + refined_text
polish; warmup builds the recognizer and a second session reuses it; the panel
honesty note. Backend refinement/capture_ws/capture/sherpa suites, CJK + route
inventory gates, full vitest (733), lint (0 errors) and format all green.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:52:45 +05:30
16294fed44 feat(updates): data-safe updates — pre-migration DB backups, guarded venv heal, release notes + changelog reader (#909)
Backend:
- core/db_backup.py: WAL-safe SQLite snapshot to omnivoice.db.backup-<version>-<n>
  before pending alembic migrations run; keep newest 3, prune older; skip >500MB
  with a log line. Restore is never automatic.
- core/db.py: _run_alembic_upgrade now plans the run (up_to_date / pending /
  unknown_revision), snapshots first when migrations will execute, and raises
  MigrationError on a mid-flight failure — startup stops with the backup path
  named instead of continuing on a half-migrated DB. The #552/#547
  unknown-revision class stays non-fatal (warn + additive reconcile).
- core/changelog.py + GET /api/settings/changelog: parse the shipped
  CHANGELOG.md (single-line and wrapped bullet styles) into structured releases.
- GET /api/settings/db-backup: newest pre-migration backup for the panel.

Rust (bootstrap.rs):
- #314 heal guard: an exit-signature match alone can no longer delete the venv —
  venv_rebuild_justified requires a structural problem or a failed direct
  interpreter probe; a venv that probes healthy is kept and the real error
  surfaced. Drift/repair remains in-place `uv sync` (non-destructive).
- CHANGELOG.md now ships as a bundle resource and is copied/refreshed into the
  project dir so the changelog endpoint works in packaged installs.

Frontend (Settings → Updates):
- Available update shows its actual release notes (updater metadata body)
  through a safe markdown-lite renderer (text nodes only, refs stay plain).
- "Your data is backed up before every update" line with the latest backup
  timestamp from the new endpoint.
- "What's new" changelog reader (accordion, newest expanded) over the shipped
  CHANGELOG.md; GitHub releases list reuses the same renderer.
- One-time, non-blocking "What's new" footer pill after an update
  (persisted last-seen version; fresh installs baseline silently).
- All strings via t() with en keys (other locales fall back to English).

Tests: db backup/rotation/failure-path units, migration-safety units, changelog
parser (both bullet styles + real CHANGELOG.md), endpoint tests, route
inventory regenerated, Rust decision-logic + probe tests, vitest suites for
renderer/viewer/panel/pill logic.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:52:17 +05:30
e2c4ea93b0 fix(models-settings): surface async install errors, disk-space guard, cancel wiring, honest restart (#908)
Live-audit fixes for the Models settings surface — the P1s were cases where
the feature silently didn't work for the user.

P1-A — Async install errors were invisible. The `install_error` SSE event
carries excellent mirror-aware text (#890 core/failure.py), but the Model
Store auto-purged the errored row ~800ms later (same as a success) and the
first-run WizardLibrary DELETED the row without ever reading `ev.error`. The
SSE→rowState reduction is now a pure, tested reducer (downloadReducer.js /
reduceWizardDownloadEvent); only SUCCESS terminals auto-purge
(isAutoPurgeTerminal), an error persists on the row with inline text + Retry +
Dismiss (Model Store) / a Retry (wizard).

P1-B — No disk-space check on install. `POST /models/install` now compares the
FDL-05 plan's exact `to_download_bytes` (+ MIN_FREE_GB headroom) against
`shutil.disk_usage(cache).free` BEFORE downloading and emits an actionable
install_error naming the sizes (needs X, headroom Y, have Z) instead of failing
mid-download. `/models` also surfaces `disk_free_gb` in the header. MIN_FREE_GB
+ disk_free_bytes are single-sourced in setup/models.py (wizard delegates).

P2-A — Wired the orphaned cancel. `POST /models/install/cancel` (FDL-11) had
zero frontend refs; the in-progress row now shows a Cancel button that calls it
and transitions the row to install_cancelled.

P2-B — Honest restart_required. The HF-mirror PUT returned restart_required:true
unconditionally; it now returns true only when the persisted value actually
changed, with accurate copy (Model Store downloads use the new mirror
immediately — resolved per-call; only transformers model loads need a restart).

P3 — i18n the un-localized panels (HFMirrorPanel, ApiKeysPanel source
labels/help/status, MODEL_ROLE_LABEL) via new en.json keys; other locales fall
back to en.

Tests: new tests/test_install_disk_space.py (reject-when-over-budget incl. the
worker wiring; allow-when-fits; degrade on unknown size/unprobeable volume),
updated tests/test_hf_mirror_settings.py (change-only restart_required), and new
frontend reducer + column-render tests for install_error persistence, Retry,
Dismiss, and Cancel.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:40:15 +05:30
b3c18db33f feat(settings): Storage panel — real disk usage, category breakdown, and low-space warnings (#906)
Settings → Storage now opens with a Disk usage panel backed by a new
loopback-gated GET /api/settings/storage endpoint:

- Per-volume totals (grouped by st_dev) + du-style sizes for everything
  the app owns: the HF model cache (with its ~10 largest models), the
  app data dir broken into voices/outputs/dub_jobs/batch/preview/
  database/logs/other subtotals, engine venvs (backend/engines/*/.venv
  + the app venv), and omnivoice* entries in the OS temp dir.
- Bounded scanning: per-category 10 s deadline → partial totals with an
  "unreadable" warning instead of a hung request; results cached
  in-process for 5 minutes, ?refresh=1 forces a rescan; the walk runs
  in a worker thread so the event loop never blocks.
- Server-side warnings reuse the setup wizard's MIN_FREE_GB: free <
  min → critical, free < 2×min → low, volume holding the cache/data
  >90% full → volume_pressure, unreadable/timed-out paths → unreadable.

The panel renders severity-colored banners, a data-volume gauge,
proportion bars per category, Open-folder buttons (existing
/export/reveal pattern), a Model Store jump for reclaiming model
space, and the existing clear-logs action on the logs row. A critical
warning is also surfaced outside Settings via the app-wide toast —
once per session. All strings via i18n (en fallback).

Tests: tests/test_storage_report.py (sizes, thresholds, cache/refresh,
timeout partials, endpoint wiring) + StorageUsagePanel.test.jsx
(categories, banners, once-per-session toast, refresh=1, error state);
route added to tests/fixtures/api_routes.txt via the dump script.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:39:49 +05:30
e7fc37d438 fix(settings): retire legacy LLM endpoint panel, surface env overrides, fix Cloudflare account + fast-fail probes (#907)
Live-audit fixes for Settings → LLM Providers / Translation.

Retire the legacy LLMEndpointPanel from the UI (backend endpoint kept).
TranslationTab no longer embeds the inline endpoint panel — it now points to
Settings → LLM Providers (openSettingsTab('llm-providers')), which fully covers
it via the `custom` provider (a lone TRANSLATE_BASE_URL still resolves to
`custom`). Kills the panel's lying "reachable" badge, its hardcoded-English
strings, and one of three duplicate TRANSLATE_* surfaces. The third duplicate —
TranslationTab's "Provider keys" collapsible — drops the TRANSLATE_* trio
(now owned by LLM Providers) and keeps only the DeepL/Microsoft translator
keys; its toast no longer claims "saved for session" (these are in
PERSISTENT_KEYS, restored at startup). GET/PUT /api/settings/llm-endpoint is
untouched (DubTab gates Cinematic off it; tests + route inventory cover it).

Surface env overrides. describe() now reports base_url_from_env / model_from_env
/ active_from_env (mirroring key_from_env). The panel disables env-pinned
base_url/model/account fields with an explainer, and — when
LLM_DEFAULT_PROVIDER pins the active provider — disables make-active and shows a
banner, instead of silently reverting the user's edit / no-oping the button.

Fix the Cloudflare account-id flow (broken two ways): describe() now returns the
stored account_id (the field no longer resets to empty) and shows the RAW
base_url template ({account_id} kept literal) instead of the substituted value;
save_overrides drops a base_url override equal to the built-in default, so the
UI posting the shown value back can't freeze the URL — later account-id changes
take effect again (also self-heals if a default URL changes in a release).

Fast-fail the Test / Fetch-models probes. Pass max_retries=0 to the probe
OpenAI clients so a 429/timeout returns in seconds instead of ~34s on the SDK's
default retry ladder. /models now returns truncated:true when capped at 200 and
the UI hint reads "first 200 shown".

Tests: registry env-flag + Cloudflare round-trip/no-freeze regressions; router
truncation + max_retries=0 assertions; panel disabled+explained + banner;
new TranslationTab test (pointer wired, legacy panel gone, TRANSLATE_* dropped).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:36:50 +05:30
b825d99337 fix(engines): revive dead license dialog, refresh matrix on select, surface routing verdict (#905)
Six live-audit fixes for the Engines settings surface:

- P1-A: the Supertonic license dialog was dead since #101 — `useState`
  threw away the state value (`const [, setLicenseDialogFor]`) and the
  imported dialog was never mounted, so "Accept license" did nothing.
  Keep the value and render LICENSE_DIALOGS[selected] with open/onClose/
  onAccepted (accept → matrix reload).
- P1-B: the matrix went stale after "Use" — active badge, Use buttons and
  family-tab captions stayed old until a manual Refresh. Await onSelect,
  then reload() so the picked engine reflects immediately.
- P2-A: consume the /engines/select routing echo. A `cpu_fallback` pick now
  shows a warn-tone toast naming the reason ("running on CPU — …"); the
  plain success toast stays for accelerated/cpu_only. Shared helper used by
  both Settings→Engines and the first-run WizardLibrary.
- P2-B: a CPU-native engine (gpu_compat == ("cpu",)) has nothing to fall
  back FROM, yet on a GPU/MPS host it was mis-classed cpu_fallback (warn).
  New routing rule classifies ("cpu",) as cpu_only (neutral) on any
  accelerator host; multi-target engines that could accelerate elsewhere
  are untouched.
- P3-A: the routing reason was only a badge `title` (unreachable on
  keyboard/touch) — surface it as small visible text under the badge.
- P3-B: an in-process "Test engine" pass is an import/liveness check, not a
  synthesis test — label it "deps OK" instead of a misleading "0 ms"
  latency; subprocess rows keep their real ping latency.

Adds RTL + unit regression tests for all six and updates the routing unit
tests to the corrected cpu-native intent. i18n keys added to en.json.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:22:49 +05:30
b6a3eba1ac feat(launchpad): deck-of-cards redesign — fanned feature cards with waveform faces (#904)
The seven launchpad feature cards now render as an overlapping deck fanned
left-to-right: each card sits in a fan slot with a subtle tilt (±4°) and
vertical stagger (≤14px), peeking ~33% out from under its right neighbor.
Every card face carries its lucide icon + name on the always-visible peek
edge, a one-line description, and a decorative CSS-only animated waveform
strip in the card's accent color (stagger-delayed scaleY bars, aria-hidden,
static under prefers-reduced-motion).

Hovering OR keyboard-focusing any card brings it fully forward — it
straightens, scales up and takes the top of the stack while every other
card slides toward it and tucks underneath (dimmed, scaled down, overlap
increased). The raise/tuck classes are React-state-driven so pointer and
focus share one code path and tests can assert it. Fixed deck height —
zero layout jump.

Navigation targets, i18n keys, per-feature accent hues and profile/project
counts are unchanged; both renderings share a single feature list so they
can't drift. On shell-narrow/shell-mini (the app-container's own width
classes — not viewport @media, per the UI-scale rationale in App.jsx) the
deck degrades to the pre-existing flat ActionCard grid, tracked live via
MutationObserver (new useShellNarrow hook), keeping 900×600 usable.

New LaunchpadDeck.test.jsx covers: 7 cards in canonical order, every
navigation target (incl. clone/design → studio + defineMethod), raise/tuck
partitioning for hover and focus, waveform decorativeness, the narrow
fallback under both shell classes, and the runtime class flip.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 23:22:35 +05:30
bef688e9dd release: freeze v0.3.9 — version bump, lockfiles, changelog (#899)
* release: freeze v0.3.9 — version bump + lockfiles + changelog

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* release: unwrap the [0.3.9] section — release bodies hard-break single newlines

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:35:39 +05:30
958a79ef7f fix(generate): device-aware timeout guidance — stop telling CPU hosts to switch to CPU (#896) (#902)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:26:24 +05:30
72d137e1f3 fix(bootstrap): port cuDNN 8 (NVIDIA CUDA GPU) + VC++ redist to packaged installs (#869)
* fix(bootstrap): port cuDNN 8 (NVIDIA CUDA GPU) + VC++ redist install into ensure_venv_ready()

* fix(bootstrap): address #869 review — drop dead VC++ half, cache negative CUDA probe, gate on ROCm, sync docs

Per maintainer review on #869:

1. Drop the VC++ Redistributable half: LoadLibraryA("vcruntime140.dll")
   from the running Tauri exe is a tautology (the exe itself links the
   MSVC CRT, so the process wouldn't be running without it), and torch's
   real failure mode is msvcp140.dll inside the venv python process.
   Dead code removed; a comment records why for future readers.

2. Stop taxing every non-CUDA launch: a negative torch probe (CPU /
   Intel / AMD — most installs) is now cached in a
   .venv/.cudnn8_probe_negative marker, so the synchronous `import
   torch` runs at most once per venv lifetime. Invalidated on every
   path that can change the torch build (drift sync #307, repair sync,
   first-run sync, ROCm reinstall) and implicitly by a venv rebuild.
   A probe that fails to run cleanly is skipped WITHOUT caching so a
   transient error can't wedge a real CUDA machine.

3. Rewrite docs/install/troubleshooting.md §10 to the actual root
   cause: packaged installs never had the cudnn8_compat libs (so
   reinstalling never restored them); the bootstrap now installs them
   automatically on CUDA machines, with the manual uv pip command as
   the offline fallback and PyTorch Whisper as the sidestep.

4. Gate the ~700 MB nvidia-cudnn-cu12 download on the venv torch being
   a real CUDA build: the probe now reports 'hip' before checking
   cuda.is_available() (which HIP spoofs), so opt-in ROCm installs
   (#124) never fetch the CUDA wheel.

Also reflow the CHANGELOG entry to house style (bold one-line lead,
1-3 lines of why, (#827, #869) refs) and extend the bootstrap unit
tests: classify_cuda_probe verdict mapping and the marker
write/invalidate round-trip (6 cuDNN tests total, 43 lib tests green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 22:23:06 +05:30
e8fdf0e244 feat(footer): Logs icon + uniform icon sizes + value-moment donate popover (Clippy-style, strictly throttled) (#898)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:23:36 +05:30
83e71c5689 fix(asr): close the #730 residuals — chunked dub wedge shares the guarded reset; repeated timeouts recommend the crash-isolated engine (#895)
Residual A — the chunked dub-stream had a PARALLEL wedge mechanism (its own
ping-loop timeout, its own _reset_pool_on_wedge, a dead-end "Try restarting
the server" message). A wedged chunk now routes through the SAME
run_transcribe_guarded bound+reset as the whole-file paths (#731/#851): the
guard resets the poisoned pool once per wedged attempt (no double-reset on
retry) and the user sees the actionable ASRTimeoutError. The reset logic is
extracted to asr_backend.reset_pool_after_wedge — one shared mechanism, so
the semantics can't drift again. run_transcribe_guarded also gains a
timeout_env param so chunk errors name OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S
instead of the whole-file knob.

Residual B — the crash-isolated ASR sidecar (#393, faster-whisper-isolated)
is wired as an explicit ESCAPE HATCH, not a default:
- selectable end-to-end: Settings engine list gets an explanatory
  install_hint; honest gpu_compat ("cuda","cpu" — it wraps the same
  CTranslate2 engine as faster-whisper); get_active_asr_backend now hands
  back a process-wide singleton for subprocess-isolated backends (a fresh
  instance per request would leak atexit hooks and respawn the sidecar —
  reloading its model — on every transcribe).
- on the SECOND consecutive guarded timeout-with-reset in one session
  (resets aren't recovering the hang; the wedged thread keeps its VRAM),
  the error the user sees + the log recommend switching to the isolated
  engine in Settings → Engines. Never auto-switched (owner rule: no silent
  behavior divergence); a completed transcribe resets the streak.

Tests (fail-before/pass-after verified against origin/main): wedged-chunk
SSE integration (reset count + actionable error + recommendation surfaces),
consecutive-timeout streak (fires at 2, resets on success, suppressed when
already on the isolated engine), timeout_env parametrization, shared-reset
helper, isolated backend in list_backends with hint + honest availability,
singleton caching, gpu_compat matrix entry. Docs: troubleshooting §14 gains
the chunk knob + escape-hatch guidance.

Closes the residuals tracked on #730.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:34:28 +05:30
3bb401f4e5 test: make LLM-provider state leaks between tests impossible (#878) (#894)
Root cause: LLM provider selection reads three process-global surfaces —
env vars (LLM_DEFAULT_PROVIDER, per-provider *_API_KEY/*_BASE_URL,
TRANSLATE_*), the SQLite settings store (llm.active_provider & co.), and
prefs.json (llm_backend). Importing `main` (TestClient fixtures do)
dotenv-loads the developer's .env and ~/.config/omnivoice/env straight
into os.environ, and several tests/endpoints mutate these surfaces
without teardown — so whichever test imported the app first flipped what
later tests' active_backend_id()/active_provider_id() resolved to
(order-dependent failures in test_engines.py,
test_llm_endpoint_settings.py, test_llm_providers.py).

Fix the class, not the instances:
- tests/conftest.py: redirect OMNIVOICE_DATA_DIR to a per-session tmp dir
  and OMNIVOICE_ENV_FILE into it (before collection freezes
  core.config.DATA_DIR), so tests never read or write the developer's
  real app state and local runs behave like clean CI.
- tests/conftest.py: autouse `_isolate_llm_provider_state` fixture
  snapshots env (derived from llm_providers._PROVIDERS, so new providers
  are guarded automatically), llm.* / secret.llm_key.* settings rows, and
  the prefs llm_backend/env.TRANSLATE* keys before every test and
  restores them exactly afterwards.
- shared `clean_llm_env` fixture clears the FULL provider env surface;
  the four LLM test modules' hand-picked partial delenv lists (which left
  e.g. LLM_DEFAULT_PROVIDER / OPENROUTER_API_KEY standing) now use it.
- tests/test_llm_state_isolation.py: deterministic fail-before/pass-after
  regression pair — pollutes all three surfaces without cleanup, then
  asserts the guard restored them.

Verified: the issue's two-test repro passes; the five LLM-related test
files pass in order; full suite green (2046 passed, 20 skipped,
10 xfailed, 4 xpassed).

Fixes #878

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 19:06:14 +05:30
86f5213055 fix(splash): IPC-independent watchdog + recovery panel for dead Tauri IPC (#879) (#892)
After an unclean shutdown (Windows BSOD), the WebView2 profile cache
(%LOCALAPPDATA%\com.debpalash.omnivoice-studio\EBWebView) can corrupt:
Tauri's IPC custom protocol fails AND the postMessage fallback breaks,
so invoke() hangs forever. useBootstrapStage's poll loop rode entirely
on that IPC — a hung bootstrap_status call silently killed the loop and
the splash sat at "preparing" forever, even with a fully healthy
backend answering over plain HTTP.

Class fix, three parts:

- splashWatchdog.js: IPC-independent escape hatch. If no IPC signal
  arrives within 10s, poll GET /health over plain HTTP; healthy →
  proceed to the app as if 'ready' was received (console.warn
  breadcrumb so diagnostic bundles carry it). Any successful IPC
  response disarms it for good.
- Recovery panel (stage 'ipc_lost'): if neither IPC nor HTTP succeed
  within 45s, show an actionable panel instead of the infinite
  spinner — "Open logs" (with an inline path fallback when IPC is
  dead) and, Windows-only and only in this error state, "Repair and
  restart". Health polling continues behind the panel so a slow
  first-run install with broken IPC still reaches the app.
- clear_webview_cache_and_relaunch (Rust): writes a marker and
  relaunches; the fresh process deletes EBWebView at the top of run()
  before any webview exists (WebView2 holds locks while running),
  with a bounded retry while the old instance exits. Runtime cfg!
  guards keep the whole path compiling on every platform.

Tauri 2 exposes no reliable flag for the postMessage-fallback mode
(closure-local in its injected ipc.js), so the logged detector is the
observable combination: zero IPC signals + working plain HTTP.

Fail-before/pass-after regression tests: hung invoke + healthy HTTP →
ready; hung invoke + dead backend → recovery panel, then auto-continue;
working IPC → normal path untouched, zero HTTP polling. Plus watchdog
state-machine unit tests and recovery-panel render/interaction tests
(6/7 fail on the pre-fix component). Troubleshooting doc gains the
matching section (docs-sync).

Fixes #879

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:52:44 +05:30
6e600c48cb fix(generation): classify network/download failures — stop mislabeling every unknown error as OOM (#880) (#893)
A kittentts first-use HuggingFace download died with httpx's "Cannot send
a request, as the client has been closed", and the generation error
classifier's catch-all fallback told the user (CPU-only ~80 MB ONNX engine,
12 GB-VRAM box) they were OUT OF MEMORY and to press Flush — the wrong
remedy for a network failure.

Three-part class fix:

- generation.py: new #880 branch (before the OOM hint) classifies
  httpx/requests transport failures — matched over the whole exception
  chain (type names like ConnectError/ReadTimeout plus stringified
  signatures like "client has been closed") — as a download/network
  problem with a retry/check-connection remedy.
- generation.py (the real class bug): the OOM hint is no longer the
  catch-all. It now requires an actual OOM signature (typed
  OutOfMemoryError/MemoryError anywhere in the chain, or CUDA/MPS/CPU
  allocator wording); genuinely unknown errors surface as unrecognized
  with the underlying detail instead of a false "ran out of memory".
- tts_backend.py: KittenTTS's first-use load retries exactly once with a
  fresh HF Hub client (huggingface_hub.utils.close_session()) on the
  specific closed-client failure — hub ≥1.x shares one global httpx
  client, and a closed one is recoverable, so the download self-heals
  instead of failing the generation.

Fail-before/pass-after tests: classifier (closed-client message, wrapped
httpx type names, unknown error, real OOM signatures incl. typed
OutOfMemoryError, WinError 1455) + the retry helper (recovers once,
walks the chain, no retry on unrelated errors, single-shot).

Fixes #880

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:38:02 +05:30
14f1257d1f fix(errors): name the configured HF mirror when a model download fails (#874) (#890)
When a non-default HF_ENDPOINT (Settings → Models → Hugging Face mirror,
e.g. hf-mirror.com) is configured and a model load/download fails with a
connectivity error, the raw transformers message ("We couldn't connect to
'https://hf-mirror.com' to load the files…") leaked to the UI as a bare 500
with no next step.

Class fix — one shared classifier in core/failure.py covers every surface:

- classify()/build_failure(): new HF_MIRROR_UNREACHABLE class with a dynamic
  hint that names the configured mirror, says it may be down, points at
  Settings → Models → Hugging Face mirror, suggests the official endpoint
  when the model isn't cached, and notes the restart requirement (HF reads
  HF_ENDPOINT at backend start). Checked before the video-download network
  class so a model download's "timed out" no longer gets the "video server"
  hint. Feeds /model/status and every build_failure event (dub, tasks).
- main.py global 500 handler: appends the hint to the surfaced detail, so
  ALL routes that can leak a model-load error benefit (generate, dub,
  archetypes, …), not just TTS generate.
- setup/download.py install SSE: the install_error event gets the same hint.
- error_journal: "couldn't connect to" / "max retries exceeded" now classify
  as NETWORK_ERROR (was UNKNOWN) for auto-attached bug reports.
- model_manager (#886 family): the "cache incomplete and could not be
  auto-repaired" message now names WHY the auto-repair failed (mirror
  outage, offline mode, full disk no longer read identically), which also
  lets the mirror hint fire on that surface when applicable.

Fail-before/pass-after regression tests in tests/test_hf_mirror_error_class.py
(12 of 13 fail on main).

Fixes #874

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:23:47 +05:30
be1ec3ade0 fix(platform): declare Intel-Mac local backend unsupported — honest first-run gate + docs (#889); Windows portable-install docs (#766 follow-up) (#891)
torch >=2.3 ships no macOS x86_64 wheels (transformers 5.x needs torch >=2.6),
so `uv sync` can never resolve on an Intel Mac — per the platform-parity rule
the honest option is declaring the platform unsupported, not letting first
launch die in a raw resolver error:

- bootstrap.rs: pre-check on macOS x86_64 before any venv create / uv sync
  (first-run AND repair paths) fails fast with an actionable message
  (remote-backend escape hatch + docs link); healthy pre-torch-bump venvs are
  deliberately untouched. Unit test pins the message's load-bearing phrases.
- BootstrapSplash: routes the failure to a dedicated localized hint
  (bootstrap.hint_intel_mac, all 21 locales) and suppresses the useless
  Retry-oriented hints for it.
- README + docs/install/macos.md (+ troubleshooting #9): every Intel-Mac
  support claim now says UI-installs-but-backend-cannot-run, including the
  from-source path (also broken); remote backend documented as the only use.
- release.yml: #889 note on the macos-15-intel leg — artifact is UI-only;
  keep-or-drop is an owner call, deliberately not changed here.
- docs/install/windows.md: new "Portable install (Windows)" section promised
  in #766 — custom MSI wizard folder / msiexec INSTALLDIR=..., what lives in
  OmniVoiceStudio-Data next to the exe, and the Program-Files-greyed-out why.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 18:22:37 +05:30
d58010fe1b feat(dictation): rebuild to Wispr-Flow quality — live waveform, streaming commits, honest insertion, polished text (#888)
* feat(dictation): rebuild to instant-feedback quality — waveform, streaming commits, honest insertion, text polish

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): dictation rebuild entry

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(lint): Array.from over new Array(n) — oxlint no-array-constructor

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:57:56 +05:30
da9315815d feat(settings): LLM provider testing pass — latency + classified errors, model discovery, full i18n, router tests (#887)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:03:13 +05:30
bb492086c9 fix(desktop): enforce maximize() at startup — macOS can ignore the conf flag with Overlay title bar (#881 follow-up) (#884)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 15:16:34 +05:30
641e660677 fix(shell): LogsFooter becomes a real grid row — bottom buttons can't clip under it at small window sizes (#882)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:31:23 +05:30
62035435e8 fix(desktop): always open maximized (not fullscreen) — stop window-state restoring stale geometry (#881)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:03:23 +05:30
4eed552153 fix(engines): Confucius4-TTS validated E2E — clone sys.path import, 22.05 kHz, real install docs (#590) (#872)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 04:49:50 +05:30
ae516cae63 fix(asr): un-gate Parakeet TDT from CUDA-only — measured ~10× realtime on CPU (#871)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 04:49:42 +05:30
85fd9ca799 fix(asr): VRAM preflight before whisperx load — no more native OOM abort on 8 GB cards (#723) (#870)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 04:48:51 +05:30
Palash Debnathandmergetest 86c701bff9 fix(translate): bound the whole cinematic/autofit pass so a slow LLM can't hang "Translating…" (#868)
The per-segment call already has a 45s timeout and concurrency is capped, but a
slow or rate-limited provider on a large dub (hundreds of segments) can still keep
the "Translating…" spinner spinning for minutes as segments queue through the
bounded pool. There was no ceiling on the *whole* pass.

Add an overall wall-clock budget, OMNIVOICE_CINEMATIC_BUDGET_S (default 180s,
<=0 disables). Segments that finish in time keep their cinematic refine; any still
in-flight when the budget hits is cancelled and degrades to its literal (Fast)
translation with error="cinematic-budget", so the translate ALWAYS returns instead
of hanging. Order and length of the result are preserved. Abandoned executor
threads follow the same fire-and-forget pattern as the GPU-pool wedge guard (#730).

Regression tests: a 3s-per-segment refine under a 0.3s budget returns in <2s with
literal fallbacks; budget<=0 runs every segment to completion.

Co-authored-by: mergetest <test@local>
2026-07-02 00:39:36 +05:30
Palash Debnathandmergetest f4e318f9f2 fix(translate+dub): wire Cinematic/Autofit to the LLM Providers registry; retry a wedged transcribe chunk instead of dropping it (#867)
Two bugs from real reports:

1. LLM not wired — translator._llm_client()/_llm_model() read TRANSLATE_*/OPENAI_*
   directly, bypassing the LLM Providers registry (#854). So a provider set up
   in Settings → LLM Providers never powered Cinematic/Autofit. Now resolves the
   ACTIVE provider (base_url/key/model) via llm_providers; the 'custom' provider
   still maps TRANSLATE_* so legacy env setups keep working.

2. Transcription 'missing the beginning' — the chunked dub transcribe dropped a
   whole chunk's window on failure/timeout (returned empty segments, no retry).
   A transient wedge on the FIRST chunk (whisperx cold-loads its model there, the
   #730 hang) therefore lost the start and left only middle+end. Now retries a
   failed/timed-out chunk once on a fresh pool (OMNIVOICE_TRANSCRIBE_CHUNK_ATTEMPTS,
   default 2) so the recovered chunk fills the hole.

Imports + dub_transcribe/translator/llm_providers tests green.

Co-authored-by: mergetest <test@local>
2026-07-02 00:18:35 +05:30
287a6cb3a2 refactor(gallery): cleaner, elegant voice cards — tokens over hardcoded surfaces, borderless state (#866)
Redesign ArchetypeCard for a calmer visual hierarchy and design-token
surfaces, no behavior change (all props/handlers/loading states identical).

- Replace hardcoded surfaces with tokens: chips/wand/preview bg → tokens
  (bg-white/[0.05] → --color-bg-elev-2, bg-white/[0.03] hover → --chrome-hover-bg),
  text → --color-fg / --color-fg-muted / --color-fg-subtle, and the literal
  #1d2021 hover text on Use voice → --color-fg-inverse.
- Borderless by direction: drop the hover/state border classes on the action
  buttons and the card; convey hover via background tint + text color and the
  playing state via an accent box-shadow ring (no literal/token borders).
- Hierarchy: name is the focal point (semibold, --color-fg); metadata line is
  smaller/muted (--color-fg-muted) so it recedes.
- Chip row renders only when there are chips (no empty min-h reserve); the grid
  stretches rows so mt-auto still bottom-aligns actions.
- Accent used tastefully: tinted Use voice → solid accent on hover/focus with
  inverse text; favorite star stays subtle until hover/active. Focus-visible
  rings intact.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 23:47:52 +05:30
Palash Debnathandmergetest c29c276dc1 fix(dub): compact + responsive header — tighter stepper/title/actions, drop hardcoded borders (#865)
- Stepper (inline): smaller step gap/font (0.66rem), 19px icons, 10px connectors
  → the 6 stages take far less width so title + actions fit before wrapping.
- Title: lighter weight (medium/0.78rem), normal-case, min-w-0 truncation; meta
  0.68rem; project name truncates too. Tighter header padding + gaps.
- Removed the hardcoded header border + border-left divider (borderless) and the
  rgba bg → token --color-bg-elev-1.

Co-authored-by: mergetest <test@local>
2026-07-01 23:46:58 +05:30
3d0705fdb7 feat(engines): Confucius4-TTS — finalized (API-validated + unit-tested; opt-in, GPU run pending) (#590) (#637)
* feat(engines): Confucius4-TTS scaffold (opt-in, needs hardware validation) (#590)

Plumbing for netease-youdao's Confucius4-TTS — LLM-based 14-language
cross-lingual zero-shot voice cloning, Apache-2.0 — mirroring the opt-in
subprocess-venv pattern of dots.tts / MOSS-TTS-v1.5:

- engines/confucius4/__init__.py: Confucius4Backend(SubprocessBackend), CUDA-only
  (gpu_compat=("cuda",)), language passthrough, ref_audio→prompt_wav. is_available
  reports a clear reason and stays unavailable without a clone.
- bootstrap.py: dedicated Python 3.10 venv resolution (user clone-level venv →
  package venv → uv bootstrap), import-probed on `confuciustts`.
- main.py: sidecar speaking the same length-prefixed JSON-over-stdio protocol as
  the other engines, calling ConfuciusTTS(config_path, device).generate(text,
  lang, prompt_wav).
- Registered lazily in _LAZY_REGISTRY; docs/engines/confucius4-tts.md.

Gated behind OMNIVOICE_CONFUCIUS4_TTS_DIR — inert on every default install, never
imports the upstream package unless opted in. The sidecar's synthesis API is
derived from the upstream README and is NOT yet validated on a CUDA box; the
module, docs, and CHANGELOG all flag this. 4 tests pin registration +
inert-by-default. No version bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#590): register Confucius4 in install-hints + docs inventory (CI gates)

Registering the engine tripped two completeness gates: every backend needs an
install_hint (test_issue_fixes) and every registry engine must appear in the
tts_engines docs inventory + README (check-docs-drift). Add the install_hint,
the docs/features.yaml entry, and the README engine-table row (with the scaffold
caveat). Docs-drift clean; gates pass. No version bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(confucius4): finalize — validate API vs upstream, add 22 sidecar unit tests, document external deps (Amphion/w2v-bert/weights)

The synthesis API (ConfuciusTTS(config_path, device) → generate(text, lang,
prompt_wav) → tensor, model.sample_rate) is confirmed against the
netease-youdao/Confucius4-TTS repo. Added runnable unit tests for the sidecar's
pure logic (language norm, tensor→PCM mono/stereo/clip, config resolution, wire
framing, synthesize dispatch with the model mocked) — 22 cases, all green.
Docs now list the external deps (Amphion/MaskGCT codec, facebook/w2v-bert-2.0,
~2-4GB HF checkpoint) and CUDA 12.6. Softened the scaffold warnings to reflect
API-validated + unit-tested status; a one-time CUDA GPU run is still needed to
confirm live inference + true sample rate.

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 23:37:40 +05:30
8f7b242610 fix(theme): remove stray token-border frames + make accent family theme-track (#864)
Task 1 — physically remove the token-based structural border utilities that
kept rendering stray frames (history panels, cards, rows, settings) whenever a
`--*-border` token didn't resolve transparent (theme re-declare, or bare
`border` = currentColor under Tailwind v4). Converted every
`border[-trbl]-[var(--chrome-border…)]` / `[var(--color-border…)]` (83
occurrences across 32 components/pages) to `border-transparent` — keeps the 1px
box (no layout shift, matches the badge.tsx convention), drops the frame, and
active/selected state stays visible via the existing bg-tint/text cues. Also
converted button.tsx's `border-border`/`border-input` variants and Panel's
header divider. Kept: focus-visible rings, aria-invalid, dashed drop-zones, and
the waveform/segment editor. Strengthened tests/test_no_literal_borders.py with
`test_no_token_border_utilities_in_jsx` so a reintroduced token border fails CI
(allowlists the editor + shadcn form-control primitives).

Task 2 — aliased the accent family in the base :root to the themed brand token
(`--chrome-accent: var(--color-brand)`, `-bg`/`-border` via color-mix), so
donate/support/commercial CTAs, active tabs, .btn-primary, status pills and
GoalBar/Pip track the active theme instead of the fixed pink. Replaced the
hardcoded `#d3869b`/`#f3a5b6`/`rgba(243,165,182,…)` pinks and the DONATE_HUE
constant in SupportPage.jsx with `var(--color-brand)` tints.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 22:06:56 +05:30
Palash Debnathandmergetest d0aa2fbd52 fix(ui): remove the SECOND history aside's border (missed by #860's replace_all — different indentation) + filter-chip borders (#862)
Co-authored-by: mergetest <test@local>
2026-07-01 21:39:34 +05:30
df845f45a0 fix(theme): theme-aware native selects — color-scheme per theme + token-driven caret/options/focus ring (#861)
Native <select> chrome (option popups, scrollbars, form UA elements) rendered
in the OS light scheme on dark themes because `color-scheme` was never set as a
property (only a `prefers-color-scheme: light` media query existed, which is
not the same thing). The dropdown caret was also a hardcoded gray SVG that
ignored theme + accent. Owner report: gallery/install/language selects looked
wrong for accent + dark/light.

- Declare `color-scheme: dark` on :root (default Gruvbox Dark) and re-assert it
  on every [data-theme] block. All six shipped themes are dark (verified by
  their real --color-bg lightness: midnight #0f172a, nord #2e3440, solarized
  #002b36, rose-pine #191724, catppuccin #1e1e2e), so all get `dark`. The empty
  auto/light scaffold is left at dark (no light theme ships yet; a light value
  there would mismatch the still-dark surface) with a note for when one lands.
- Replace the hardcoded %23a1a1aa caret in select.input-base and .ui-select with
  a single --select-caret token, overridden per theme to that theme's muted
  foreground (a background-image SVG can't read a CSS var, so the color is baked
  per theme). Both selects consume the one token (DRY).
- Paint <option>/<optgroup> from --color-bg-elev-1 / --chrome-fg so Chromium
  (Windows/Linux) popups match; macOS WebKit popups follow color-scheme.
- Give selects a themed focus-visible ring (--color-ring → --color-brand) to
  match the buttons/checkboxes tokenized last phase, instead of the
  non-theme-tracking --chrome-accent.

Borderless guardrail and :focus-visible rings intact. Covers every named
native select (DubTab/AudiobookTab language, DubLeftColumn engine, gallery,
ui/Input.jsx Select, VoicePreview, StoriesEditor, ExportModal, DubSegmentRow)
via the shared input-base/ui-select rules — no per-call-site edits needed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 21:28:05 +05:30
Palash Debnathandmergetest d55976eff0 fix(ui): physically remove the panel-frame border utilities on history + active-voice panels (#860)
#857 zeroed the border TOKENS but left token-based border utilities
(border-t-[var(--chrome-border-strong,…)], border-b-[var(--chrome-border)]) in
the JSX — a fragile indirection that still renders a line if the token doesn't
resolve transparent (stale HMR / the pre-zero rgba base value). Per 'no borders
whatsoever', remove the utilities outright from the WorkspaceHistory (dub +
regular history) and WorkspaceVoices (active-voice) panel frames; the
active-voice card keeps its background tint as the selection cue.

Co-authored-by: mergetest <test@local>
2026-07-01 21:21:37 +05:30
b2e578b21d style(controls): unify buttons/inputs/selects/checkboxes/toggles onto design tokens (Phase 2) (#859)
* style(controls): Phase 2 — tokenize + unify buttons/inputs/checkboxes/toggles onto design tokens

Phase 2 of the borderless styling pass. Converts interactive controls to
design tokens with a cohesive, theme-tracking active/checked affordance,
building on Phase 1's borderless base. No behavior changes — visual/token only.

Shared primitives (highest leverage):
- ui/button.tsx: replace literal `hover:bg-white/[0.04]` (subtle/softGhost/
  chip/preset/iconBtn) with `hover:bg-[var(--chrome-hover-bg)]`.
- ui/toggle.tsx (seg): drop hardcoded `text-[#fff9ef]` active text and hover
  white literal for `text-fg` + `--chrome-hover-bg`.
- ui/Segmented.jsx: recessed track `bg-black/[0.28]` -> `bg-bg-elev-2`.
- index.css: native checkbox `accent-color` and range-input thumb/track/active
  moved off non-themed `--chrome-accent` / legacy `--text-primary`/`--primary`/
  raw rgba onto themed `--color-brand` / `--color-fg` / `--color-bg-elev-2` +
  radius/shadow/duration tokens. Checked state is now brand-tinted and recolors
  per [data-theme], matching sliders/segmented/primary buttons.
- SettingsToggle: on-state -> `--color-brand`, focus ring -> `--color-ring`,
  knob shadow -> `--shadow-sm`, radius -> `--radius-pill`.

Control call sites (exact-token swaps, remove hardcoded hex/rgba):
- Unified every checkbox `accent`/`accentColor` override onto `--color-brand`
  (DubbingDemo, DubRightColumn, DubLeftColumn, IdleSkeleton, DubFooter,
  DubSegmentRow, AppearancePanel range).
- FooterBtn blue/orange tones -> --color-info/--color-warn.
- MicButton danger tint/neutral fill -> tokens.
- DubLeftColumn install CTAs + engine chip -> brand tokens + --radius-pill.
- NetworkToggle: neutral fg/hover tokens; removed stray `#504945` fallback border.

Focus rings and the borderless guardrail (tests/test_no_literal_borders.py)
intact. build + format:check + lint (0 errors) + guardrail all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(dub): assert the tokenized brand-accent install button (bg-[var(--color-brand)]) after Phase 2

Phase 2 tokenized the highlighted Install CTA from the hardcoded #d3869b to
var(--color-brand); update the two assertions to match.

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 20:50:04 +05:30
Palash Debnathandmergetest f7b7e2c13a chore(version): pin main to 0.3.8 (revert the post-release auto-bump) (#858)
* Revert "chore(version): main -> 0.3.9 after v0.3.8 release"

This reverts commit 7489bef085.

* chore(release): gate the post-release version-bump behind AUTO_VERSION_BUMP (owner controls bumps)

Owner decision (2026-07-01): keep main pinned to the released version and bump
only on explicit request. The version-bump job now runs only when the repo
variable AUTO_VERSION_BUMP == 'true' (default off), so releasing no longer
auto-rolls main to +1. Documented the override in CLAUDE.md's versioning rule.

---------

Co-authored-by: mergetest <test@local>
2026-07-01 20:18:15 +05:30
1550ce2976 feat(ui): app-wide decorative border/divider removal (keep focus rings, bg cues) (#857)
Remove decorative borders, hairlines, dividers, and panel frames across the
frontend for a flat, frameless look. Selection/active state and input fields
stay perceivable via background/elevation cues; keyboard :focus-visible focus
rings are preserved.

index.css:
- Append a final `:root, [data-theme]` block zeroing every border token
  (--color-border[-strong|-warm], --chrome-border[-strong], --chrome-accent-
  border, --glass-border) → transparent. Kept last so it wins over the default
  root and all [data-theme] overrides. --color-ring / --focus-ring untouched.
- .glass-panel::before decorative top-highlight → display:none.
- Zero 22 neutral (white/black rgba) literal hairline borders (history divider,
  segment table, override toggles, etc.).
- Selection cue: .project-active border → transparent, stronger bg tint.
- Inputs: .input-base / textarea.input-base get a recessed --color-bg-elev-2
  fill (was --chrome-hover-bg / --chrome-bg which equalled the panel bg) so
  fields stay visible without a border; subtle elevation shift on focus.
- .history-kind--audio colored pill border → bg tint.

JSX/TSX:
- 79 literal-color border utilities (border-white/black, border-[#|rgba|
  color-mix]) → border-transparent (width kept: app ships without Preflight,
  so a bare button keeps a UA border).
- badge.tsx / button.tsx colored tone/active variants → border-transparent
  (bg fill + text color carry the tone); outline badge gains a bg.
- Bare `border` on shadcn card/dialog/select/dropdown content + ui/Tabs →
  border-transparent (no-Preflight currentColor line).
- Active/selected chips (StoriesEditor track, HfTokenCard, FirstRunSetup
  option, WorkspaceHistory/Sidebar/WorkspaceVoices kind pills) → background
  tint instead of accent border.
- 9 inline style borderColor → removed or swapped to a background tint
  (selection/error stay perceivable).

Kept intentionally: :focus-visible / border-ring focus rings, aria-invalid
error borders, the waveform segment editor (SegmentTrack) functional
boundaries/handles/selection, drag-active dropzone accent, and severity-token
state cues — these are functional affordances, not decorative chrome.

Guardrail: tests/test_no_literal_borders.py fails if the regression class
reappears (neutral literal borders in index.css, literal-color border
utilities / inline borderColor in jsx/tsx) and asserts focus tokens survive.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 20:02:18 +05:30
github-actions[bot] 7489bef085 chore(version): main -> 0.3.9 after v0.3.8 release 2026-07-01 13:57:41 +00:00
7c0b0c2572 fix(diagnostics): harden the bug-report scrubber (5 audited leak/correctness gaps) (#856)
* fix(diagnostics): harden the bug-report scrubber against 5 audited leak/correctness gaps

Audit of the (already-on-main) diagnostics/bug-report feature found the opt-in/
no-telemetry contract clean but 5 real gaps in the redaction + URL assembly.
Fixed in both scrub twins (backend/core/scrub.py + frontend utils/bugReport.js):

- Windows home paths with lowercase 'users' now redact (case-insensitive) — a
  spec-level PII leak: c:\users\john\… kept the username verbatim.
- Broadened credential shapes (JWT/Bearer, Google AIza, Slack xox, AWS AKIA) +
  a URL query-secret pass (?token=/?api_key=… → value redacted, name kept) so a
  secret propagated from a backend error into error.message/.stack can't reach a
  public issue. The webview has no env backstop, so these shapes are its only
  defense.
- Boundary-safe $HOME replace: a home of /Users/john no longer rewrites
  /Users/johnny to '~ny' (fragment leak + path mangling).
- Bug-report URL now bounds the URL-ENCODED body length (~7k), not the raw
  length — a dense 6k markdown body encoded to ~9k and blew past GitHub's ceiling
  (silent truncation / failed open). Message body is capped too.

- 9 new scrub regressions (backend) + 9 (frontend); all green. No API/behavior
  change beyond stricter redaction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): note the bug-report scrubber hardening in [0.3.8] (#856)

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 19:26:38 +05:30
5e2d314efe feat(dub): consolidate pipeline stepper and title/meta into one header row (#855)
Merge the two stacked dub-editor header rows into a single line to save
vertical space. The pipeline stepper (Upload → … → Export) is now inlined
onto the DubHeader row alongside the title, duration · N segs metadata, and
the primary action buttons (Generate Dub / QC / Export). All step
active/complete styling, data bindings, and button onClick/disabled/loading
props carry over unchanged.

- DubPipelineStepper gains an `inline` prop → `dub-stepper--inline` variant
  (drops the standalone border-bottom/padding, tighter connectors).
- DubHeader renders the inline stepper as the leftmost element; the row is
  flex-wrap so it wraps gracefully on narrow windows.
- DubTab only renders the standalone spine before the editor exists, so the
  stepper is never duplicated once the editor (and inline spine) is shown.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 19:02:26 +05:30
29269b9cf0 feat: LLM Providers page + Autofit translation quality (fit-to-segment-time) (#838) (#854)
* feat(llm): multi-provider LLM registry + encrypted key storage + settings API (v0.3.8, phase 1)

Foundation for the LLM Providers settings page and timing-aware (Autofit)
translation. Every provider in the shipped .env is OpenAI-compatible, so one
client drives all of them via a registry instead of a class-per-provider.

- llm_providers.py: registry of 16 providers (OpenAI, OpenRouter, Groq,
  Cerebras, Google AI, Mistral, Cohere, NVIDIA, GitHub Models, Cloudflare,
  HuggingFace, SambaNova, SiliconFlow, + local Ollama/LM Studio + Custom).
  Field resolution precedence env → encrypted store → default; active-provider
  selection (LLM_DEFAULT_PROVIDER → stored → first keyed remote; local requires
  explicit pick so we never assume a local server is up). Legacy TRANSLATE_*
  maps to the Custom provider (keyless-with-base_url preserved).
- settings_store.py: generic ENCRYPTED secrets (get/set/clear_secret,
  list_secret_names) reusing the HF-token Fernet path; get_text/set_text now
  refuse the secret namespace (no ciphertext leak).
- llm_backend.py: OpenAICompatBackend resolves the active provider's
  base_url/key/model from the registry. Backward-compatible.
- settings API: GET /llm-providers, PUT /llm-providers/{id} (encrypted key +
  overrides), POST /llm-providers/active, POST /llm-providers/{id}/test.
  Loopback-gated; never returns key material.
- 11 registry tests; existing llm-endpoint/openai-available tests still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(settings): LLM Providers page — configure any provider's key/URL/model + Test + set active (v0.3.8, phase 2)

New Settings → System → LLM Providers pane (Brain icon, searchable). Lists all
16 registry providers; pick one to configure its encrypted API key, base URL,
model (and Cloudflare account id), Test the connection with one round-trip, and
'Save & use for translation' to make it the active provider for Cinematic/
Autofit. Keys are write-only from the UI (masked placeholder, never echoed);
env-set keys show as read-only. Local providers (Ollama/LM Studio) need no key.

- LLMProvidersPanel.jsx: provider selector + per-provider config + Test/activate,
  following the LLMEndpointPanel pattern (apiJson/apiFetch/apiPost, SettingsSection
  primitives).
- settingsCategories.jsx: new 'llm-providers' category under System + Brain icon.
- Settings.jsx: route the category to the panel.
- en.json: settings.llm_providers label.
- Frontend build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(translate): Autofit quality style + one-click LLM setup from the dub menu (v0.3.8, phases 3-4)

Autofit = Cinematic + a strict 'never exceed the segment time' fit. The LLM
rewrites each translated line so its target-language reading time fits within
the slot, preserving the video timing without harsh audio time-stretch.

Backend:
- speech_rate.adjust_for_slot(strict=): strict caps the accepted upper ratio at
  1.0 (fit within slot) vs Cinematic's 1.08; best-effort, degrades gracefully
  with no LLM.
- dub_translate: quality='autofit' takes the LLM refine path and runs the fit
  pass with strict=True; reports quality_used accurately.
- TranslateRequest.quality doc note.

Frontend:
- 'autofit' added to the quality control (Settings Translation + dub menu) and
  the TranslateQuality type.
- Dub menu: picking Cinematic/Autofit with no LLM no longer dead-ends on a toast
  — it offers a one-click 'Set up' that routes to Settings → LLM Providers,
  with copy about fitting translations to segment time (#838).

- 4 strict-fit tests; frontend build green; i18n keys added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(translate): document Autofit quality + the LLM Providers page (v0.3.8, phase 5)

- CHANGELOG [0.3.8] Added: Autofit style + LLM Providers page.
- docs/dubbing/translation-engines.md: Fast/Autofit/Cinematic quality section
  and an LLM Providers setup section (16 providers, encrypted keys, offline
  Ollama/LM Studio, env overrides).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(api): add /api/settings/llm-providers routes to the route-inventory snapshot

Regenerated tests/fixtures/api_routes.txt for the 4 new LLM-providers endpoints
so test_route_inventory_matches_snapshot passes (keep-main-green).

* style(frontend): oxfmt the LLM Providers panel + dub quality control (format:check green)

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 18:55:59 +05:30
c5c57508b3 fix(device): fall back to CPU when the GPU arch is unsupported, not 500 every generate (#756) (#757)
* fix(settings): contain + tighten the whole Settings surface (measure cap, container-query stacking, wrap the shared rows)

Two systemic issues drove 'too spread out' + 'elements go out of view' across
many Settings pages:

1. Spread — .settings-content capped at 1280px, so on wide windows every
   label-left/control-right row left a huge void. Introduce a --settings-measure
   token (720px, macOS-like) + --settings-rail, and cap the content to it,
   left-aligned under the nav. One token now controls the reading width.

2. Overflow + bad responsiveness — the row stack break was a *viewport* media
   query (560px), but the 168px nav rail means a 760px-viewport window only has
   ~530px of content, so rows went side-by-side in a cramped box. Make
   .settings-content a container (container-type: inline-size) and stack on the
   CONTENT width via @container, keeping the viewport @media as a fallback for
   the .st-row instances used outside Settings (Splash/FirstRun/Dub/SetupWizard).

3. The shared .perfpanel__row (button/badge row reused by 6+ panels:
   RemoteBackend, HFMirror, LLMEndpoint, Pronunciation, MCPBindings, …) was an
   inline-flex with no wrap and no max-width, so it ran off the right edge —
   add flex-wrap + max-width:100% + min-width:0. Plus two rigid-width fixes that
   escaped the row cap: ApiKeys input min-width:220→0, Appearance scale floor.

Frontend builds clean; tokens, @container query, and the wrap all verified in the
emitted CSS bundle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(settings): center the settings block + tighten measure (kill the lopsided right void)

The capped content was left-aligned, so on a wide window everything jammed to the
left with a dead empty third on the right (screenshot). Center the whole settings
block (nav rail + content) as a unit via max-width + margin-inline:auto, and drop
the measure 720→660 so label→control rows read denser. The cap is computed from
the tokens (rail + gap + measure + page padding) so the content track lands
exactly at --settings-measure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(device): fall back to CPU when the GPU arch is unsupported, instead of 500-ing every generate (#756)

get_best_device() called check_device_compatibility() and, on an unsupported
compute capability, only LOGGED a warning then still returned 'cuda' — so the
model loaded on a GPU whose kernels can't launch and every generate 500'd with
'CUDA error: no kernel image is available for execution'. Both a too-old card
(Pascal sm_61, GTX 10-series) and a too-new one (Blackwell sm_120 on pre-cu128
wheels) hit this.

Now an unsupported arch falls back to CPU (works, just slower) with a clear
warning; OMNIVOICE_FORCE_CUDA=1 overrides. Belt-and-suspenders: _oom_friendly_reraise
classifies a raw 'no kernel image is available' as an unsupported-GPU error
(switch to CPU / install matching torch) rather than the OOM/Flush message.

Tests: get_best_device → cpu on incompatible, stays cuda on compatible, honors
the force override; reraise gives the actionable GPU message, not OOM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(device): patch detect_host_caps via string path so the #756 fallback test is full-suite robust

The first version aliased the import + inserted backend on sys.path, which patched
a module copy get_best_device's local 'from core.device_caps import detect_host_caps'
didn't resolve in the full suite (passed alone, failed in CI). Use the string-form
monkeypatch target; verified passing alongside the other device/model tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): fold #757 device-fallback entry into [0.3.8]; drop the merge's stale [Unreleased] dupe

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 17:59:49 +05:30
5e0d6826da docs(changelog): cut v0.3.8 — fold Unreleased into the 0.3.8 release section (2026-07-01) (#852)
Renames [Unreleased] to [0.3.8] — 2026-07-01 and merges the settings-hub
redesign, translation/network/factory-reset panes, the GPU-pool generate-hang
fix (#851), and the translation-banner fix into the release section so
release.yml extracts a complete, house-style body at tag time.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 17:53:17 +05:30
e347f99542 fix(tts): bound + reset the GPU pool on a hung generate so it can't brick the backend (#730 class) (#851)
* fix(tts): bound + reset the GPU pool on a hung generate so it can't brick the backend (#730 class)

A GPU job that wedges on some Windows+CUDA setups occupies its worker
forever — run_in_executor can't cancel the thread — so on the 1–2 worker
pools we ship, one stuck job starves every other request and the next
action surfaces as the misleading "Can't reach the local backend" even
though the process is alive.

ASR/dub/model-load already bound+reset the pool on hang (#730). The TTS
**generate** paths (generation.py, tts_stream.py) were the last unguarded
GPU dispatch — and the residual on-main reports (#850 #802 #755 #723 #721,
plus the 0.3.7 generate cohort) all fail on generate:start (audio).

- model_manager: add run_on_gpu_pool_guarded() + GpuJobTimeoutError, a
  generalized version of the ASR guard so every GPU dispatch shares one
  bound+reset recovery path. Env-tunable via OMNIVOICE_GENERATE_TIMEOUT_S
  (default 300s).
- generation.py: route both inference branches + the reference-clip
  transcribe through the guard; map a timeout to an actionable 503.
- tts_stream.py: same guard on the streaming path (timeout → error frame).
- test_generate_timeout_730: fail-before/pass-after regression (timeout
  resets pool + restores capacity, happy path, env override, no-reset exec).
- docs + CHANGELOG: extend troubleshooting §14 to cover generate; document
  the new env var.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(tts): extend the GPU-pool hang guard to batch/dub/archetype/openai-compat generate (#730 class)

The generate-hang class wasn't only in Studio + streaming: batch generate,
the dub per-segment + preview generate, archetype preview render, and the
OpenAI-compat /v1/audio/speech path all dispatched the TTS model to the GPU
pool with no wall-clock bound either. Any one of them wedging on a
Windows+CUDA hang starves the pool and bricks the backend the same way.

Route all of them through run_on_gpu_pool_guarded so the whole class is
closed — a hung generate anywhere resets the pool and returns an actionable
timeout instead of a dead backend. Batch/dub recover per-segment on a fresh
worker; drop the now-dead loop/_gpu_pool/asyncio locals ruff flagged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:44:02 +05:30
38b8c55e52 fix(theme): restore per-theme chrome recoloring — default :root was clobbering [data-theme] overrides (P5 consolidation regression) (#849)
The color themes (Midnight, Catppuccin, Nord, Solarized, Rose Pine) stopped
recoloring the app chrome — the Settings hub, header, footer and everything
else that reads var(--chrome-*) stayed the default dark on the real app.

Root cause: in the real app `data-theme` is set on <html>, and <html> IS
`:root` (documentElement === :root). The P5 tokens consolidation inlined the
default legacy/chrome `:root` block (--chrome-bg:#0f1011, …) AFTER all the
[data-theme] blocks. A plain `:root {…}` and a `[data-theme="x"] {…}` both
match that same element at EQUAL specificity (0,1,0), so source order is the
only tiebreaker — the later default `:root` won and clobbered every theme's
--chrome-*/--color-* overrides. The visual-regression suite kept passing
because its harness applies `data-theme` to a WRAPPER div (a closer ancestor
that wins by proximity, not source order), so it never exercised the <html>
path where the bug lives.

Fix (source order, not specificity): reorder index.css so every default
`:root` block precedes all `[data-theme]` blocks. The [data-theme] blocks
(+ the @media prefers-color-scheme:light theme block) now sit LAST, after the
default legacy/chrome `:root`. The `[data-theme="x"]` selectors are unchanged
(bumping to `:root[data-theme="x"]` would stop matching the wrapper-based
harness and break the 48 snapshots).

Crucially the Tailwind v4 region is left byte-for-byte intact: the @theme base
and the adjacent `@theme inline` shadcn bridge keep their exact positions.
Moving a `:root` between/across them changes the GENERATED CSS (`@theme inline`
stops inlining, so shadcn utilities lose their brand color) — so instead of
lifting the default :root above @theme, the [data-theme] blocks are lowered
below it. Verified: the compiled CSS is byte-identical to before (260686 B),
and every token value is preserved byte-for-byte (pure reordering).

Regression test: src/test/themeCascade.test.js replays the documentElement
cascade from index.css source order and asserts each theme's --chrome-bg/-fg
wins over the default :root. Fails-before / passes-after. Verified live in
Chromium too: getComputedStyle(documentElement)['--chrome-bg'] now resolves to
#0f1011 (default) / #1e293b (midnight) / #313244 (catppuccin) / #3b4252 (nord).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:43:32 +05:30
255ac1bad0 fix(settings): convert un-migrated panel controls to design-system primitives (theme-consistent inputs/buttons/selects) (#848)
Several Settings panels were re-hosted in the redesign without converting
their raw <input>/<select>/<button> to the design system, so they rendered
as native UA controls (white input fields, light-gray buttons, system
fonts) that ignored the theme tokens — jarring on the dark chrome. Convert
every native control in the affected panels to the shared primitives
(SettingsInput / ui Button / ui Select / ui Badge) so all of Settings
themes coherently in every palette.

Panels fixed:
- LLMEndpointPanel: Ollama/LM Studio/vLLM/OpenAI preset chips -> Button
  (preset); Base URL / Model / API key -> SettingsInput (mono); Save ->
  Button (subtle/sm, loading); reachable/not-configured status -> Badge
  (success/warn, dot).
- HFMirrorPanel: mirror preset chips -> Button (preset); Save -> Button
  (subtle/sm, loading). (HF_ENDPOINT was already SettingsInput.)
- PronunciationPanel: add-entry term/replacement/language + test inputs ->
  SettingsInput; type selector -> ui Select; per-row enable checkbox ->
  SettingsToggle; type/scope pills -> Badge; Add + per-row delete ->
  Button (subtle/sm, danger/sm).
- RemoteBackendPanel: Test connection + Save & reload -> Button
  (subtle/sm, loading); probe result -> Badge (success/danger, dot).
- MCPBindingsPanel: client-id input -> SettingsInput; voice select ->
  ui Select; Bind -> Button (subtle/sm); per-binding profile pill ->
  Badge; delete -> Button (danger/sm).

Also dropped the perfpanel__row / perfpanel__badge / perfpanel__checkbox
class usages from these panels (replaced by primitives + token flex
utilities). The perfpanel CSS block lives in src/index.css (owned by an
in-flight theme-cascade change), so it was left in place; the remaining
perfpanel__error / perfpanel__help references are token-based themed
banners, not native controls.

Behavior, handlers, state, endpoints, and all data-testids are preserved.
No new user-facing strings (styling-only). Gates: vite build, oxlint (0 on
touched files), oxfmt --check clean, vitest 645 pass, 48 visual snapshots
unchanged, bun install --frozen-lockfile clean. Live eyeball across all 5
categories in default + catppuccin themes confirms no white fields / no
native buttons.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:34:58 +05:30
522bbddccf feat(translate): highlighted Install affordance for uninstalled engines + dismissable/auto-clearing error banner (#847)
Two related Dub-tab translation-flow fixes, one PR.

TASK 1 — proactive, highlighted Install affordance in the translate engine
selector (replaces "find out only via a translate-time 400"):

- FROM-SOURCE lane (activeEngineUnavailable && !enginesSandboxed): the muted
  install chip is promoted to a HIGHLIGHTED brand-accent Install button, still
  wired to handleInstallEngine(translateProvider) with the installing/disabled
  state. Selecting any uninstalled engine surfaces it immediately.
- FROZEN lane (enginesSandboxed): pip install is impossible in the read-only,
  signed packaged env, so the disabled "needs dev install" span becomes an
  equally highlighted button opening a popover with (1) the exact install
  command + copy-to-clipboard, (2) one-click "Switch to Argos (bundled,
  offline)" — the guaranteed importable escape hatch, and (3) a Docs link via
  the existing Tauri shell.open path. Gated on the existing `sandboxed` flag,
  not platform.
- Single-source install command: new translation_engines.install_command()
  is the one source of truth; list_engines() stamps `install_command` per
  engine and BOTH the argos + deep_translator translate-time 400 messages build
  their command from it, so the proactive button and the 400 can't drift.
  engines.ts gains `install_command: string | null`.

TASK 2 — the translation error banner now dismisses and clears (class fix):

- Root cause: handleTranslateAll never cleared dubError, so a stale 400
  survived even a successful retry. It now clears at the start of every
  attempt.
- Corrective-action clears (whole class): changing the engine and installing
  the package both clear dubError (wrapped setTranslateProvider +
  handleInstallEngine in DubTab).
- DubFooter's banner gains a × dismiss and a guarded auto-timeout (skipped
  while generating so live per-segment errors persist).

i18n: 8 new dub.* keys translated across all 21 locales. Docs: new
docs/dubbing/translation-engines.md (from-source vs packaged build) linked from
the popover Docs button + a troubleshooting cross-reference. Tests: FE
regression for both lanes + never-installs-when-sandboxed + banner
dismiss/auto-clear; BE regression that list_engines() install_command is
embedded verbatim in the dub_translate 400s.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:16:13 +05:30
66ad03948b fix(dub): show transcribing/progress view instead of idle dropzone while the pipeline runs (#846)
The Dub stepper could show Upload ✓ → Prepare ✓ → Transcribe (active) while the
main content pane still rendered the IDLE upload dropzone ("Drop video or audio
here" + paste-URL input + "Pull YouTube captions"). Contradictory: if the
pipeline is transcribing, the pane must reflect that stage, not the landing.

Root cause (frontend/src/components/dub/IdleSkeleton.jsx): the main-view branch
keys off the non-serialisable local File `dubVideoFile`. That File is only set
on the drag/drop + file-input path — never on the URL-ingest path (and not on a
restored job). The `dubVideoFile ?` branch correctly renders both the prepare
(PrepOverlay) and transcribe (TranscribeOverlay) overlays via the WaveformTimeline,
but the no-file branch only handled `dubStep === 'uploading'` (PrepOverlay large)
and otherwise fell straight through to the idle dropzone. So a URL-ingested job
in `dubStep === 'transcribing'` (no File) rendered the dropzone — the exact
desync in the screenshot.

Not a #818 regression: the no-file branch never handled `transcribing`. It was
identical before #818 (verified against 9d79bb8) — a pre-existing gap that only
bites the URL-ingest / restored-job paths.

Fix (whole class, recurrence-proof):
- Add a `dubStep === 'transcribing'` case to the no-file path that renders
  TranscribeOverlay, symmetric to the existing `uploading` → PrepOverlay case.
  This covers URL-ingest AND restored/resumed jobs that lack a local File.
- Gate the idle dropzone on `dubStep === 'idle'` so it can render ONLY when
  genuinely idle; any other non-idle no-file step (e.g. `stopping`) shows a
  neutral working indicator instead of falling back to the dropzone. This makes
  it structurally impossible to show the dropzone during an active pipeline.

All existing behavior/handlers preserved (failure banner + retry still show in
the idle-after-failure state, since that sets dubStep back to 'idle').

Regression test: frontend/src/test/DubIdleSkeleton.test.jsx — asserts the
dropzone renders only when truly idle, is hidden (and the transcribe overlay
shown) while transcribing a URL-ingested job, is hidden while preparing, and
never falls back to the dropzone for a non-idle no-file step. Fails before /
passes after.

Verified live (Playwright, real backend): before → transcribe stage shows the
dropzone (transcribingHasDrop=1, overlay=0); after → shows the transcribe
overlay (transcribingHasDrop=0, overlay=1), idle still shows the dropzone,
reset returns to idle.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 10:47:05 +05:30
18de6d26d3 fix(settings): right-anchored controls fill leftward on the full-width hub (#845)
The full-width Settings hub (#843) left every right-aligned SettingRow
control capped at `max-w-[60%]`, so wide fields (text/URL/key inputs,
selects, textareas) sat cramped against the right edge with a big empty
gap to the label. Read-only mono values ("0.3.8", version strings) also
wrapped character-by-character because `[overflow-wrap:anywhere]` collapsed
the auto grid cell to a 1-char min-content, and removing the content
measure spread rows edge-to-edge on wide/ultrawide screens.

SettingRow.jsx:
- Widen the control grid track to `minmax(0,1fr) minmax(0,1.9fr)` only
  when the row contains a real field (`has-[input:not(checkbox/radio/range)]`,
  `has-[select]`, `has-[textarea]`), gated to `@min-[601px]/settings` so the
  narrow-container stacking is untouched. Toggles (checkbox), Segmented /
  Slider (Radix), and buttons don't match, so short controls keep the `auto`
  track and stay compact, right-pinned.
- Lift the `max-w-[60%]` cap to `max-w-[85%]`; make the control cell `w-full`
  (has-gated) so wide fields fill the widened track leftward to a clean right
  edge. Existing `w-full` fields fill; short controls unaffected.
- Fix mono/read-only wrapping: `[overflow-wrap:anywhere]` -> `break-word` and
  the percentage `max-w-[75%]` -> length-based `max-w-[42ch]`, so short values
  render on one line (the percentage cap forced the auto track to min-content)
  while long paths still wrap on boundaries.

Settings.jsx:
- Re-introduce a generous, centered content measure (`w-full max-w-[1100px]
  mx-auto`) on the content column so rows fill from the middle instead of
  spreading to the screen edges on wide/ultrawide displays; the rail stays
  fixed. Wider than the old cramped 660px measure, capped for readability.

Verified visually with Playwright at 1400px (General, Translation, Network,
Credentials, Appearance, Dictation, About) and 700px (stacking intact). All
gates pass: vite build, oxlint, oxfmt, vitest (641), visual (48, no baseline
change needed), frozen lockfile.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 10:14:41 +05:30
424ab000dd fix(settings): sidebar items show UA button-gray in dark themes (#844)
The sidebar nav items are native <button>s and the non-active state set no
background, so with Tailwind preflight disabled they fell back to the browser's
default `ButtonFace` (light gray) — washed-out pills in the dark themes, and the
active item paradoxically looked darker (it got the subtle --chrome-hover-bg
overlay while inactive items showed UA gray). Add explicit `bg-transparent` +
`appearance-none` so items are theme-adaptive; active/hover keep the overlay.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 09:53:56 +05:30
28aeacbc39 feat(settings): make the Settings hub full-width (#843)
Dropped the root max-width cap + mx-auto centering and the content pane's
reading-measure cap so Settings spans the full content area (rail + fluid
content) instead of sitting in a centered column with side gutters. The
`container-name:settings` inline-size container is preserved, so SettingRow's
narrow-width stacking still fires on the real content width.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 09:41:42 +05:30
48ccb1da30 docs(contributing): reconcile file-size/co-location rules with the one-stylesheet end-state (#842)
The CSS consolidation (#837) collapsed all component CSS into src/index.css, so
the "hard 500 lines per .css" cap and "co-locate Foo.css" rule no longer apply.
index.css is the single intentional styling foundation (exempt from the cap);
styling is utilities + shadcn, not per-component files.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:17:28 +05:30
2345888c4e docs(contributing): CSS guidance for the one-stylesheet end-state (#841)
The CSS consolidation (#837) folded every per-component stylesheet into
src/index.css — the note still implied component-level .css files exist for
keyframes/glass/hooks. Now: all styling lives in src/index.css; don't create
new component .css files.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 06:14:13 +05:30
df194e2a93 refactor(ui): consolidate component CSS into index.css — collapse to ~one stylesheet (#837)
Fold every remaining per-component stylesheet into src/index.css so the frontend
ships essentially ONE CSS file. index.css keeps its Tailwind v4 token foundation
(@layer order + @theme + [data-theme] + shadcn bridge) and now also carries, in a
clearly-commented "CONSOLIDATED COMPONENT STYLES" section, the former residual.css
plus all 28 component .css files — verbatim, unlayered, appended AFTER index.css's
own rules so the previous cross-file load order (index.css → residual.css →
component css) is preserved exactly. @keyframes move by name (all globally unique);
glass/backdrop-filter, cascade-override hooks, and library-DOM hooks (WaveSurfer
wfm-*, virtualized rows) keep winning over @layer utilities because they stay
unlayered. Zero visual/behaviour change — proven by the 48-snapshot visual suite
passing with no PNG diffs.

CSS files: 31 → 2 (src/index.css + src/test/visual/harness.css, test-only).

Deleted (29), each import removed from its component:
  styles/residual.css, components/{Misc,firstrun,Sidebar,LogsFooter,CaptureWidget,
  CompareModal,FloatingPill,DubSegmentRow,DubSegmentTable,SegmentTrack,VoicePreview,
  WaveformErrorBoundary,WorkspaceHistory,WorkspaceVoices}.css,
  components/dub/dub.css, components/donate/{DonateGoal,Postcard}.css,
  components/settings/{AppearancePanel,PerformancePanel,VoicePanel}.css,
  pages/{AudiobookTab,BatchQueue,Settings,VoiceGallery}.css,
  ui/{Dialog,Menu,Table,Tooltip}.css

Kept: src/test/visual/harness.css (test-only harness chrome; not shipped).

Guard update: test/workspaceHistoryReflow.test.js now slices the WorkspaceHistory
block out of index.css by its provenance markers, so the #476 CTA-clipping
regression guard (no @media max-width, shell-class reflow, sticky action bar) still
holds on the relocated rules.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 05:56:39 +05:30
c7b5d99133 feat(settings): rebuild Settings as a sidebar-nav hub with full app-level IA (#835)
* wip(settings): partial sidebar-hub redesign (recovered from killed agent)

Shell (sidebar/search/categories/restart-badge) + new panes (Network/Translation/Storage/PerformanceDevice) + partial panel rewiring. Not yet verified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(settings): finish + verify sidebar-hub Settings rebuild; changelog

Completes the partial sidebar-nav Settings redesign: confirmed all 16
categories are wired in Settings.jsx's renderCategory and render their real
panels with every store/pref/API binding preserved (theme→Appearance,
review-mode→General, proxy/ffmpeg→Network, provider keys→Translation — all
relocated, none dropped or duplicated). Verified search filtering, restart
badges, factory-reset dialog, narrow-width dropdown, and i18n key coverage.

Gates: vite build, oxlint (0), oxfmt --check, vitest (641 pass),
bun install --frozen-lockfile — all green. Adds the user-facing CHANGELOG
[Unreleased] entry required by the changelog hard rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(visual): refresh GeneralTab/AppearancePanel/StoragePanel baselines for the Settings redesign

The sidebar-hub rebuild changed three snapshotted panels: GeneralTab (lost
proxy/ffmpeg + theme, gained review mode), AppearancePanel (gained the
header-live-stats toggle), and StoragePanel (gained a RestartBadge header). The
recovery commit shipped stale baselines; regenerate all three across the default/
midnight/catppuccin themes so `bun run test:visual` is green against the new UI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n: backfill all 20 locales for the Settings redesign (and pre-existing drift)

The Settings rebuild added ~42 new keys to en.json; ran scripts/translate_all.py
to translate them into all 20 non-English locales (masking {{vars}}/<n> tags),
which also caught up pre-existing key drift — every locale is now at full parity
with en.json (0 missing keys). Satisfies the all-21-locales hard rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 05:26:13 +05:30
e97c297113 docs(contributing): update CSS guidance for the shadcn/Tailwind end-state (#834)
The CSS→Tailwind/shadcn migration is largely complete: every screen is on
shadcn/ui primitives + Tailwind utilities, and the design tokens were
consolidated into a single foundation file (`tokens.css`/`themes.css` folded
into `src/index.css`'s @theme/[data-theme]). The old note still pointed at the
deleted `src/ui/tokens.css` and said "migration in progress".

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 04:00:15 +05:30
c9843479db feat(ui): rewrite demo/transcription/queue components on clean shadcn, delete their CSS (fast mode) (#833)
FAST-mode shadcn migration of the tail components — the demos, the
transcriptions history, the batch queue, and the audiobook tab — onto the
shared src/ui primitives (Button/Panel/Badge/Tabs) + Tailwind token utilities
(bg-card/text-fg/border-border + standard spacing), dropping each component's
stylesheet where the residual rules reduce cleanly to utilities.

Fully deleted (residuals inlined as utilities):
  - DictationDemo.css   — status pills, scripts grid, result boxes (gruvbox
                          hues preserved as arbitrary utilities; em → not-italic;
                          .dictation-demo/.__scripts class hooks kept for tests)
  - DubbingDemo.css     — container/loading shell, 720px collapse → max-[720px]:,
                          checkbox accent, pane-label/video, active chip
  - Transcriptions.css  — search input (placeholder:/focus:), item hover/active,
                          seg-title h4 → div (escapes the unlayered global h1-h4
                          rule); list scrollbar dropped as redundant with the
                          global ::-webkit-scrollbar

Trimmed to genuinely-irreducible only (import kept):
  - BatchQueue.css      — only the progress-fill gradient + ::after shimmer +
                          @keyframes remain; the bar heading (h1 → div role=
                          heading) and per-status card borders are now utilities
  - AudiobookTab.css    — only the <textarea> override (beats the unlayered
                          textarea.input-base + custom 900px floor) remains;
                          title (h2 → div role=heading), field labels (utility
                          const), body/side collapse (max-[900px]:), and the
                          redundant select width are now utilities

Behavior preserved: test class hooks intact, headings keep heading semantics
via role/aria-level. Verified: vite build, oxlint (0), oxfmt clean, vitest
641/641, visual 48/48, bun install --frozen-lockfile clean. Eyeballed all three
pages + states in the dev app.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:50:51 +05:30
7264c321f8 feat(ui): rewrite workspace/voice tail components on clean shadcn, trim their CSS (fast mode) (#832)
FAST-mode shadcn/Tailwind migration of the workspace + voice "tail"
components: the cleanly JSX-controlled chrome moves onto the JSX as
Tailwind v4 utilities (token utilities + arbitrary var()/px to preserve
exact pixels/colors), with irreducible CSS kept co-located.

- WaveformPlayer: all three render branches (player, native fallback,
  missing notice) converted to Tailwind; WaveformPlayer.css deleted
  (-87). The `wf-player__btn` class is retained as the focus-visible
  hook for the shared a11y ring in index.css; the dead `wf-player__spin`
  rule + `wf-spin` keyframe + its reduced-motion block were removed.

- VoicePreview: popover container/header/title/close/body/foot/hint
  converted to Tailwind; VoicePreview.css trimmed 87->23 lines. Kept the
  `voice-preview-in` entrance @keyframes (referenced via animate-[…]) and
  the `.voice-preview__select`/`__text` rules — they layer on top of the
  *unlayered* shared `.input-base`, and Tailwind utilities (in
  @layer utilities) would lose that cascade, so they stay unlayered.

- WorkspaceHistory: finished the voice variant, which #781 left on the
  now-deleted `.wh`/`.wh__head`/`.wh__title`/`.wh__scroll`/`.wh__empty`
  classes (rendering unstyled). Converted them to the same Tailwind
  utilities the dub variant already uses. Kept the studio-with-history/
  studio-right/shell-narrow layout + the `.studio-action-bar` sticky
  override (#476, guarded by workspaceHistoryReflow.test.js).

- WorkspaceVoices: already fully converted by #781; its `.wv*` chrome is
  shared with the out-of-scope WorkspaceProjects.jsx, so the CSS stays.

Verified: vite build OK, oxlint exit 0, oxfmt --check clean, 641 vitest
pass (incl. workspaceHistoryReflow + waveform), 48 visual pass, bun
install --frozen-lockfile clean. Eyeballed the Voice workspace (history
rows + waveform players) and the VoicePreview popover in a live dev run.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:49:28 +05:30
8541eb75c5 feat(ui): rewrite dialog/panel tail components on clean shadcn, delete their CSS (fast mode) (#831)
Migrate the tail dialog/panel components onto the shadcn-backed `src/ui`
primitives + Tailwind utilities, removing their bespoke stylesheets.

- BatchAddDialog: rebuilt on the `Dialog` primitive (header/body/footer +
  Radix overlay/animation/focus-trap replace the hand-rolled overlay/card),
  drop zone + toggle + select moved to Tailwind / the `Select` primitive.
  BatchAddDialog.css deleted.
- KeyboardCheatsheet: rebuilt on the `Dialog` primitive; kbd pills, section
  grid, rows and footer are now Tailwind utilities. KeyboardCheatsheet.css
  deleted.
- CompareModal: kept as the deliberate non-modal bottom drawer (preserves the
  "app stays interactive behind" behavior — a shadcn modal Dialog would
  regress it). Inner content already rode the shadcn primitives; migrated the
  two remaining CSS-class deps (`.compare-textarea--noresize` -> `resize-none`,
  `.ui-compare__grid` base -> Tailwind `grid grid-cols-2`). CompareModal.css
  slimmed to just the irreducible drawer chrome + slide-up keyframe; the
  responsive one-column collapse stays owned by index.css via the retained
  `ui-compare__grid` class hook.
- GlossaryPanel: table styling moved to Tailwind (`[&_th]`/`[&_td]`
  descendant utilities); the `.glossary-panel .ui-panel__body` max-height
  override replaced by a `max-h-[35vh] overflow-y-auto` wrapper.
  GlossaryPanel.css deleted.
- Misc.css: removed only the CompareModal-owned `.compare-textarea--noresize`
  rule; the rest is shared by out-of-scope components (CheckpointBanner,
  DirectionDialog, App startup/wizard, AudioTrimmer) and is kept intact.

Behavior preserved exactly (batch add flow, cheatsheet overlay, compare
A/B, glossary add/edit). Verified: vite build, oxlint (0), oxfmt --check
clean, vitest (641 pass), visual (48 pass), bun install --frozen-lockfile.
Eyeballed all four via a temporary Playwright harness.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:48:30 +05:30
ff7d8d08dc feat(ui): migrate Settings models/reco/engines styling to shadcn, trim Settings.css (fast mode) (#830)
Migrate the last reducible CSS chunk in Settings.css — the recommendation
banner, the models/engines toolbar chrome, and the role-tab/search controls —
to Tailwind utilities (chrome tokens kept) at the JSX, following the established
shadcn fast-mode convention. Behavior and palette unchanged.

What moved to Tailwind:
- RecoBanner (.reco-banner* → utilities on models/RecoBanner.jsx)
- Models/Engines toolbar (.models-toolbar* → ModelStoreTab.jsx + EnginesTab.jsx),
  including the previously-unstyled HF-token inline chrome
- Role tabs + search (.models-controls/.models-search/.models-roletabs)

What was deleted as dead CSS (zero consumers, grep-verified):
- the entire .engines-* block (EnginesTab already on shadcn; no consumer)
- the .models-table__body > .models-row override (selector no longer matches
  the body > virtual > row DOM the table renders)

What was KEPT as irreducible styling hooks (cannot be utilities):
- .models-table* + .models-row* — the virtualized table geometry. Rows are
  absolutely positioned with an inline translateY from the virtualizer; the
  table body/virtual spacer and per-cell hooks must stay class-based.

Settings.css: 386 → 225 lines (−161). Not deleted (virtualized hooks remain).

Verified: vite build ✓, oxlint 0, oxfmt clean, vitest 641/641, visual 48/48,
bun install --frozen-lockfile ✓. Live-eyeballed Settings → Models (store +
17-row virtualized table + reco banner) and Engines (matrix + toolbar) against
the live backend; rows render correctly and chrome is coherent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:23:40 +05:30
ce74ac8777 refactor(ui): consolidate tokens.css + themes.css into index.css (P5, single foundation file) (#829)
Fold src/ui/tokens.css (134 lines) and src/ui/themes.css (195 lines) into
src/index.css so the design-token foundation lives in ONE file, then delete
the two source files and repoint every import. Pure consolidation — zero
behavior/visual change.

Cascade is preserved EXACTLY. The previous cross-file load order was
tokens.css -> themes.css -> index.css (ui/index.js imported the first two,
main-app.jsx imported index.css after). The inlined content reproduces that
order inside index.css: the token :root first, then the [data-theme] blocks,
then index.css's @theme bridge + its own legacy/chrome :root + rules. The
[data-theme] blocks intentionally sit AFTER the token :root but BEFORE the
legacy/chrome :root so the --chrome-* tokens (declared in both a plain :root
and the [data-theme] blocks at equal specificity) keep resolving by source
order exactly as before.

Imports updated:
- src/ui/index.js: the two token-CSS side-effect imports -> import '../index.css'
  (preserves "import a primitive, get the full token scale" for every consumer).
- src/test/visual/harness.jsx: drop the tokens/themes imports, keep index.css.
- src/test/tokenParity.test.js: read the token :root from index.css (located by
  its --color-muted-mono signature) instead of the deleted ui/tokens.css.

Verified: vite build OK; oxlint 0; oxfmt --check clean; vitest 641 pass
(incl. tokenParity); visual suite 48 pass with NO baseline changes (default/
midnight/catppuccin render pixel-identical); bun install --frozen-lockfile
clean. Live full-app check (data-theme on <html>) confirms semantic tokens
recolor per theme while chrome tokens hold the :root value — identical to
pre-consolidation behavior.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:22:54 +05:30
94799b4a63 feat(ui): migrate settings primitives + panels to shadcn, delete primitives/Settings CSS (fast mode) (#828)
FAST-mode shadcn migration of the shared Settings primitives and their ~8
consuming panels onto Tailwind utilities layered on the OmniVoice
`--chrome-*` / `--space-*` token bridge — palette and behavior preserved
exactly (every migrated snapshot is pixel-identical to its old-CSS baseline).

Primitives migrated off the `.st-*` CSS class family (all in JSX now):
- SettingsSection → token-bridge Card surface (exported SETTINGS_SECTION_SURFACE
  + `data-slot="settings-section"` so the raw EnginesTab / ModelStoreTab sections
  and the Settings.css table hooks stay coupled without `.st-section`).
- SettingRow → Tailwind grid; new `stack` prop replaces the `st-row--stack`
  className; control slot carries `data-slot="setting-row-control"`. Row-stacking
  reproduced with the Tailwind v4 named-container variant `@max-[600px]/settings:`
  plus the legacy `max-[560px]:` viewport fallback.
- SettingsToggle, SettingsInput, InfoHint, Collapsible → Tailwind utilities.

Consumers updated to the new API:
- GeneralTab, StoragePanel, CredentialsTab, AppearancePanel, HFMirrorPanel,
  RemoteBackendPanel: `st-row--stack` → `stack` prop; raw `.st-input` inputs →
  SettingsInput; raw `.st-section` (EnginesTab, ModelStoreTab) → token surface +
  data-slot.
- AppearancePanel.css / VoicePanel.css `.st-row__control` hooks →
  `[data-slot=setting-row-control]`; Settings.css `.st-section` hooks →
  `[data-slot=settings-section]`; `.models-search.st-input` → `.models-search`.

Deleted primitives.css (368 lines) and removed its imports (primitives barrel +
visual harness). The `.models-*` / `.reco-*` / `.engines-*` table families in
Settings.css are intentionally LEFT intact (out of `.st-*` scope).

Verified: vite build ✓, oxlint 0, oxfmt clean, vitest 641 ✓, visual 48 ✓
(baselines pixel-identical — only the harness CSS import changed),
bun install --frozen-lockfile ✓, and a live Playwright eyeball of Settings →
General / Appearance / Models / Engines (incl. embedded Storage / Performance /
HF-mirror panels) confirms every tab is coherent on-palette.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 03:03:26 +05:30
b3690e8d31 feat(ui): rewrite misc components on clean shadcn, delete their CSS (fast mode) (#826)
Migrate a batch of MISC components to clean shadcn primitives + Tailwind
token utilities, deleting per-component CSS where the styling is fully
expressible as utilities. Palette and behavior are preserved exactly.

Fully migrated (CSS deleted):
- VoiceProfile (+ ProfileHeader / ProfileDetails / ProfileActivity): all
  voice-profile__* layout/spacing classes → token utilities; the hero panel
  body becomes an explicit flex wrapper inside <Panel> (drops the external
  .ui-panel__body override). Deletes VoiceProfile.css (217 lines).
- Projects (OmniDrive): title / search input / view-toggle / filter rail /
  card grid+list variants → utilities (list/grid driven by a `view` prop
  instead of descendant-combinator CSS; per-card --card-accent kept via inline
  style + arbitrary utilities for border-left and the color-mix hover).
  Deletes Projects.css (181 lines).
- NotificationPanel: the .notif-* dropdown rules were already dead (the JSX
  migrated to utilities in a prior wave; the dropdown now lives in LogsFooter).
  Drops the dead import + deletes NotificationPanel.css (201 lines).

Trimmed (irreducible CSS kept):
- CaptureWidget: content / label / timer / dismiss / spinner moved to
  utilities (spinner uses motion-safe:animate-spin). Kept the irreducible
  glass always-on-top window shell, state borders, slide-in/dot-pulse
  keyframes, reduced-motion, and the `body:has(.capture-pill)` standalone-
  window transparency rule.

Kept as-is (with reason):
- FloatingPill: its remaining CSS is all irreducible — fixed+glass shell,
  enter/exit + dot-pulse + indeterminate-sweep keyframes, and unlayered
  --done/--error border/label overrides that must out-rank @layer utilities
  (the file's own comments document this). Content/meta/progress/dismiss were
  already utilities.
- PerformancePanel: already built on the shared SettingsSection/SettingRow
  primitives; its CSS (.perfpanel__error/__row/__badge/__help) is a SHARED
  stylesheet consumed by 7+ settings panels (MCPBindings, RemoteBackend,
  Refinement, LLMEndpoint, Pronunciation, HFMirror, …), so it can't be deleted
  without migrating out-of-scope panels.

Verify: vite build ✓, oxlint (0), oxfmt --check clean, vitest 641 pass,
visual 48 pass, bun install --frozen-lockfile ✓. Eyeballed Projects +
VoiceProfile + header bell via Playwright (real backend proxied through route
interception) — coherent, zero console errors.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:32:32 +05:30
7ccb2da19f feat(ui): rewrite modals + segment/matrix components on clean shadcn, delete their CSS (fast mode) (#825)
Migrate the independent modal + segment/matrix components onto the
shadcn-backed `src/ui` primitive surface + Tailwind utilities, deleting
their bespoke component CSS. Palette kept, behaviour intact.

- SupertonicLicenseDialog: rebuilt on the shadcn Dialog primitive
  (Radix focus-trap / scroll-lock / ESC); non-dismissable while the
  license POST is in flight. Accept/Cancel via shadcn Button. Deletes
  SupertonicLicenseDialog.css.
- ExportModal: kept as the non-blocking bottom drawer (background stays
  interactive — Radix Dialog would break that), but folded the track
  chips / tab strip / toggles / drawer shell into Tailwind utilities and
  swapped the slide-up keyframe for tw-animate-css. Deletes
  ExportModal.css.
- ErrorBoundary (WaveformErrorBoundary.css): fallback UI rebuilt on
  Tailwind + shadcn Button. Removed the `errbnd-*` block from the shared
  CSS; the `wfm-*` WaveformTimeline rules stay (file still imported by
  WaveformTimeline).
- EngineCompatibilityMatrix: folded the GPU-chip color system,
  `is-effective` highlight, `Why unavailable?` disclosure triangle, and
  the horizontal-scroll table min-width into Tailwind. Kept the
  `is-effective` marker class (matrix test asserts it), roles, testids,
  and aria-labels. Deletes EngineCompatibilityMatrix.css.

DubSegmentRow / SegmentTrack were already migrated in a prior wave and
already use the shadcn-backed Button/Badge/Menu; their remaining CSS is
the deliberate irreducible remainder (cascade-fighting `!important` state
rules that must stay unlayered to beat index.css, `font:inherit` focus
rings, `input-base`/range overrides), so it stays co-located. The shared
`segment-*` contract in index.css is left untouched.

Verified: vite build, oxlint (0), oxfmt --check clean, full vitest
(641 pass incl. ExportModal/SegmentTrack/EngineCompatibilityMatrix/
ErrorBoundary), visual suite (48 pass), and a real-browser eyeball of all
four rewritten components via the visual harness.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:31:44 +05:30
0f014129ad feat(ui): migrate app-container shell + Sidebar to utilities, trim index.css (fast mode) (#824)
Shell (app-container grid) — KEPT as-is, by design. The outer `.app-container`
grid family is the canonical cross-cutting positioning hook and is deliberately
left in index.css:
- `appShellScale.test.js` parses the literal `.app-container { … }` block and
  asserts the `zoom`/`calc(100vw/--ui-scale)` scale pattern + the
  `[data-zoom-layout=off]` 100vw/100vh fallback — migrating the base rule away
  would break that regression guard.
- `LogsFooter.css` hooks `.app-container .logs-footer` and
  `.app-container.rail-right .logs-footer` (+ a ≤600px media query) via ancestor
  combinators that Tailwind utilities can't express.
- Child placement (nav-rail / history-panel / main-content) comes from
  `.app-container > .child` descendant combinators that reflow `grid-column`
  across six dynamic state classes (sidebar-collapsed / sidebar-hidden /
  rail-right / shell-narrow / shell-mini); reproducing that as utilities would
  require editing out-of-scope child components. Net index.css delta: 0.

Sidebar.css — safe, contained migrations + dead-rule removal:
- Moved the two collapsed combinators whose base is already utilities to
  conditional utilities in Sidebar.jsx: `.sidebar.is-collapsed .sidebar__tabs`
  and `.sidebar__scroll.is-collapsed` (mutually-exclusive conditional classes,
  so no Tailwind same-property ordering trap).
- Removed dead/redundant rules: `.sidebar.is-collapsed .sidebar__tab svg`
  (icon size already set by the JSX `size` prop) and the
  `.sidebar.is-collapsed .sidebar__subtitle` / `__search` hides (both blocks are
  already gated out of the JSX when collapsed).

Kept (reported): `.sidebar__tab` base + :hover/.is-active/:focus-visible
(is-active must beat :hover via source order — not reproducible cleanly in
layered utilities), `.sidebar.is-collapsed .sidebar__tab` (its base is still
unlayered CSS, so the override must stay unlayered too), `.sidebar__search-input`
(overrides the unlayered `.input-base` primitive), `.sidebar__search-clear`
(!important Button overrides), `.sidebar__save-btn*` (consumed by out-of-scope
WorkspaceProjects.jsx), `.sidebar__section-title:hover` + `.sidebar__icon-tile`
states (prior-wave unlayered-by-design), `.sidebar.is-collapsed .sidebar__empty`
(shared EmptyState has no collapsed prop), and all `history-*` rules (consumed by
the out-of-scope Workspace* feature panels).

Verified: vite build, oxlint (0), oxfmt --check clean, vitest 641/641 (incl.
appShellScale guard), visual 48/48, bun install --frozen-lockfile. Eyeballed
Launchpad + responsive widths (1280/1000/560/1366) + a forced-render of the
collapsed Sidebar: rail/header/main/footer placement intact, footer reclaims
full width at ≤600px, 0 console errors.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:30:53 +05:30
60ac321a1b feat(ui): rewrite marketing/donate pages on clean shadcn, delete their CSS (fast mode) (#823)
Rewrites the static marketing/info surfaces on shadcn primitives (Card / Button /
Badge) + Tailwind token utilities, dropping the three legacy page stylesheets.
FAST mode: clean shadcn + Tailwind defaults, palette kept (via the existing
color-mix + var(--chrome-*) arbitrary utilities), behavior intact, not
pixel-perfect.

- SupportPage.jsx (SupportView=donate + LicenseView=enterprise): hero, segmented
  Support/License toggle, Fund-Claude-Max goal Card, amount picker, Ko-fi/PayPal
  link cards, benefit Cards, and the per-deployment quote panel — all on
  Card/Button/Badge + Tailwind. All i18n keys, URLs, openExternal, amount state,
  and view toggling preserved.
- ContactPage.jsx: hero + Discord/Email/Issues/Website channel cards rebuilt as
  hue-tinted Tailwind link rows.
- Deleted DonatePage.css (282), EnterprisePage.css (233), SupportPage.css (140)
  = 655 lines removed; no JS imports them anymore.

Kept (shared, untouched): index.css `.lp-aurora*` + `.lp-hero__sweep` (also used
by Launchpad). Left the donate widgets (GoalBar/Pip/Postcard) and their
DonateGoal.css/Postcard.css in place — already Tailwind-based with genuinely
irreducible keyframes (goal-fill grow, Pip bob/wave, postcard stamp/perforation),
the sanctioned "small co-located keyframe CSS" exception. The dead no-op
`lp-glow-card` class (never defined in CSS) was dropped.

Verified: vite build ✓, oxlint 0, oxfmt clean, vitest 641 pass, visual 48 pass,
bun install --frozen-lockfile ✓. Eyeballed Donate/Enterprise/Support + Contact in
chromium against a stubbed backend — all coherent and on-palette.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:28:38 +05:30
c931d8f4f8 feat(ui): rewrite clone/design on clean shadcn, delete CloneDesign CSS + studio shell (fast mode) (#819)
Migrate the Clone / Voice-Design feature area to clean shadcn primitives +
Tailwind utilities, deleting the 338-line CloneDesignTab.css and trimming the
`studio-*` shell from index.css. Fast mode: palette kept, behavior intact, no
pixel-perfect reproduction.

Components rewritten on utilities (token utilities bg/border/text + standard
spacing), behavior preserved exactly:
- MicButton: mic-btn idle/recording/cleaning → utilities; pulse/spin reuse the
  global keyframes via `animate-[…]`.
- ScriptPanel: studio-column/studio-panel, the ⊕ Insert button + popover,
  coachmark close, and the script textarea → utilities.
- AudioMethodPanel: drop zone (clone-drop-zone padding override folded in),
  design-seed input, save-as-profile row → utilities.
- DesignMethodPanel: describe textarea, Starting-points scroll lane (mask edge
  fade), identity recipe line, category chip/select grid → utilities.
- ActionBar: production-override sliders row, language/steps controls, overrides
  disclosure, footer CTA → utilities.
- CloneDesignTab: clone-split-grid + voice column/panel → utilities; CSS import
  removed.

studio-* shell decision (grep-verified cross-file):
- `.studio-panel` — KEPT (dub: DubLeftColumn/RightColumn/Footer, IdleSkeleton).
  Clone usages migrated to inline utilities.
- `.studio-action-bar` — KEPT, relocated from the deleted CSS into index.css.
  WorkspaceHistory.css adds its `position: sticky` narrow-shell override (#476
  CTA-clip fix, guarded by workspaceHistoryReflow.test.js), so it stays a class.
  Its __row/__lang/__steps/__overrides children migrated to utilities.
- `.studio-column` — DELETED (only consumers were clone; now utilities).

Removed `.identity-line`/`.clone-insert-btn`/`.studio-action-bar__overrides`
from the shared focus-visible rule; the accent ring is now inline on each.

Verify: vite build ✓, oxlint 0, oxfmt clean, vitest 641 pass, bun
--frozen-lockfile ✓. Eyeballed both From-audio and By-design sub-views (incl.
production overrides) in chromium — coherent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:02:58 +05:30
b940eb460f feat(ui): rewrite Gallery/Stories/Logs on clean shadcn, delete their CSS (fast mode) (#822)
FAST-mode shadcn migration of three independent areas — Voice Gallery,
Stories editor, and the logs/status footer — onto shadcn primitives
(src/ui barrel over src/components/ui/*) + Tailwind token utilities. Palette
kept; behavior preserved; ~1000 lines of bespoke CSS removed.

Voice Gallery (VoiceGallery.jsx + gallery/{ArchetypeCard,ArchetypesZone,
CommunityZone,ImportsZone}.jsx):
- Zone toggle → <Segmented>; category chips → Button variant="chip"; facet
  dropdowns → <Select>; grid/list view toggle → <Segmented>; cards/chips/
  buttons/empty/loading → Tailwind token utilities.
- VoiceGallery.css 427 → 70 lines: kept only the now-playing equalizer
  @keyframes, the .arch-avatar/.accent-flag/.flag-globe classes rendered by
  the out-of-scope utils/archetypeIcons.jsx, and the app-wide .spin helper
  (it lived here, NOT in index.css — kept to avoid breaking ~30 consumers).

Stories editor (StoriesEditor.jsx):
- Track grid, chapter bar, cast/projects/split panels, tone/speed drawer,
  and native textarea/select/range chrome → Tailwind utilities; reusable
  class-string consts hoisted. Drag-reorder, preview chain, generate/stems,
  global speed, refs, i18n keys and aria-labels all unchanged.
- StoriesEditor.css 349 → 0 lines (file deleted; import removed). The dead
  .stories-track__voice-dot[data-char] palette (no data-char ever set) and
  cosmetic webkit scrollbars were dropped.
- Native <select>s get [color-scheme:dark] so the cast/voice pickers render
  on dark chrome across WebKit/WebView2/WebKitGTK (matches the old
  .facet-select intent; the original cast select was unstyled/light).

Logs/status footer (LogsFooter.jsx):
- Icon buttons, source pills + severity badges, version badge + pulse dot,
  discord/contact/donate, log lines and notification severity → Tailwind
  utilities. Spinner → motion-safe:animate-spin; reduced-motion via
  motion-reduce: variants.
- LogsFooter.css 376 → 78 lines: kept the position:fixed shell + the
  .app-container/.rail-right/≤600px ancestor-combinator insets (can't be
  element-local), the body ::-webkit-scrollbar, the collapsed/open heights,
  and the version-dot-pulse/heart-glow @keyframes.
- The shared --chrome-* token vars and index.css are untouched; Header is
  unaffected (no logs-footer__* class is referenced outside this component).

Verified: vite build, oxlint (0), oxfmt --check (clean), vitest (641
passed), bun run test:visual (48 passed), bun install --frozen-lockfile.
Eyeballed all three areas in a stubbed dev build — coherent and on-palette.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 01:56:36 +05:30
8a9c2f1ce4 feat(ui): move Settings page chrome to Tailwind, drop page-specific CSS (fast mode) (#820)
FAST-mode shadcn pass over the Settings *page chrome*. The settings tab
components already render on shadcn — the live `src/ui/*` primitives
(Button/Badge/Tabs/Segmented/Slider/Input) are thin wrappers over the
`src/components/ui/*` shadcn primitives via the index.css token bridge — so
the only non-shadcn layer left here that is *safe to migrate* is the page
layout itself.

What changed:
- Settings.jsx: `.settings-page` / `.settings-content` are now Tailwind on the
  token bridge — the centered, scrollable column that becomes a
  [rail | content] grid at ≥760px, and the content column that establishes the
  `settings` container query primitives.css relies on. No behavior change: tab
  nav, deep-link tab, and every panel render exactly as before.
- Settings.css: removed the page-chrome rules now living in Tailwind
  (`.settings-page` + grid, `.settings-content`) and the dead ones
  (`.settings-row__mono`, `.settings-section__head-*`). Kept what can't migrate:
  the tab-rail look (must stay UNLAYERED to win over the shared shadcn Tabs
  primitive), `.settings-prose strong`, and the Models/Engines/recommendation
  rules consumed by their sub-components.
- index.css: removed the duplicate base `.settings-page` block.

Deliberately NOT deleted (verified by cross-file grep, per "delete once
unused"): primitives.css + the `.st-*` class contract (out-of-scope StoragePanel
passes `st-row--stack`; AppearancePanel.css/VoicePanel.css/PronunciationPanel
test reach into `.st-row__control`), and Settings.css's `.models-*`/`.engines-*`/
`.reco-*` (consumed by out-of-scope ModelsTable / RecoBanner /
EngineCompatibilityMatrix). Deleting either would break out-of-scope code and
main CI.

Net: 3 files, ~51 fewer CSS lines. Verified: vite build, oxlint (0), oxfmt,
vitest (641), visual (48, no baseline change — snapshotted components untouched),
bun install --frozen-lockfile. Eyeballed Settings (General + Logs) via the visual
harness: rail + content grid + centered max-width + active-tab accent + tab
switching all coherent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 01:51:53 +05:30
b033b8e9da feat(ui): rewrite dub studio on clean shadcn, delete dub CSS (fast mode) (#818)
FAST-mode shadcn migration of the Dub Studio feature area. The dub
components now style with Tailwind utilities on the OmniVoice palette
tokens (bg/text/border via chrome-* + space/text vars) plus the src/ui
shadcn primitives (Button/Badge/Progress/Segmented/Table), and the
800-line page stylesheet is gone.

What changed
- Deleted frontend/src/pages/DubTab.css (800 lines). The irreducible
  pieces that can't be utilities — keyframe motion (stepper spin,
  idle-drop pulse, skeleton shimmer), ::before stepper connectors, and a
  handful of rules that must override other *global* design-system
  classes (.studio-panel / .label-row / .override-toggle / .segment-del)
  — moved to a small co-located frontend/src/components/dub/dub.css.
- Converted the dub-* presentational classes to inline utilities across
  DubFooter (footer panel, export-track chips, compression warn),
  DubLeftColumn (generating overlay, cast strip, the whole translation
  settings bar + fields), DubRightColumn (output-options rows, transcript
  body, glossary chip, bulk-select row), IdleSkeleton (speakers input,
  ingest opt-in, landing advanced, ghost footer + buttons), and
  TranscribeOverlay (stats row).
- Rewrote FooterBtn off the global .btn-primary / .dub-footer-btn
  subsystem onto a Tailwind tone map (idle/danger/green/pink/amber/…),
  preserving the flat tinted-outline look.
- Removed the dub-* fragments from src/index.css (tabular-nums group,
  focus-visible group, and the dub-split-grid / dub-settings-bar /
  dub-footer-btns responsive media queries — now inline max-[…] utils).
  Kept .btn-primary base (still used by ErrorBoundary) and all shared
  design-system classes.

Left intact (reported): the segment-* subsystem (DubSegmentTable.jsx/css,
DubSegmentRow.jsx/css, segment-* in index.css). It's the lowest-risk
option for the core, most test-covered segment table; ModelsTable and
EngineCompatibilityMatrix were verified NOT to consume segment-* (they
use models-*/engine-matrix-*), so nothing else breaks.

Behavior preserved exactly — every onClick/state/prop/hook untouched.
Verified: vite build ✓, oxlint 0 ✓, oxfmt --check clean ✓,
vitest 641/641 ✓, bun install --frozen-lockfile ✓. Eyeballed the idle
dropzone and the loaded skeleton (stepper, settings bar, skeleton
segment table, footer) in Chromium — palette + layout coherent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 01:24:30 +05:30
bafe9677d9 feat(ui): rewrite first-run/setup on clean shadcn, delete frs CSS (fast mode) (#817)
Rebuild the first-run / setup feature area on standard shadcn primitives
(Button/Input/Select/Progress/Badge from src/ui) + Tailwind utility classes
themed by the OmniVoice palette tokens, replacing the 954-line bespoke
"studio console" stylesheet wholesale (FAST mode: clean shadcn look, not a
per-pixel reproduction of the old design).

Components rewritten:
- FirstRunSetup.jsx (install-plan screen: mode/storage/compute/channel,
  live disk gate, mirrors, Start)
- BootstrapSplash.jsx (install progress, steps, activity log, failure
  hints + retry, awaiting_setup → FirstRunSetup handoff)
- WizardLibrary.jsx (unified model/engine list + SSE download progress)
- HfTokenCard.jsx (inline HF token bar)
- SetupWizard.jsx (preflight + models + dictation acts, stepper nav)

All behavior preserved: every onClick/state/prop, the radio-group keyboard
nav, the disk-space blocker logic, the SSE progress aggregation, retry /
clean-retry, the launch flow, and all exported pure helpers (kept the
unit-tested fmtBytes/fmtRate/isPlatformPick/aggregate/progressFromAgg/
radioGroupNav exports).

CSS deleted: FirstRunSetup.css (954) + SetupWizard.css (184) + the dead
swiz-check* block in Misc.css (~21). The only bespoke CSS kept is a new
63-line firstrun.css holding the three irreducible keyframes (breathing
waveform, rise-in stagger, active-step LED pulse) that Tailwind utilities
can't express — net ~1075 lines of bespoke CSS removed. index.css had no
frs-* rules (0 line delta there).

Verified: vite build, oxlint (0), oxfmt clean, vitest (641 pass),
bun install --frozen-lockfile. Live-eyeballed all four screens
(FirstRunSetup, install splash, failed state, wizard) via Playwright —
palette correct, layout coherent, no UA button-chrome leaks.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 00:31:16 +05:30
c44bae0d0d feat(ui): migrate waveform-* to utilities, trim index.css (P4) (#816)
Move the waveform-* global class family off index.css onto Tailwind v4
utilities on WaveformTimeline.jsx, then delete the now-dead rules.

Migrated to utilities (rules deleted): waveform-timeline (mb), waveform-controls
+ -left/-right (flex/items/justify/gap), waveform-btn + :hover/:disabled and
waveform-btn-play + :hover (shared WF_BTN/WF_BTN_PLAY consts; UA <button>
padding/font preserved since the app ships no preflight), waveform-time
(text/border/bg/mono/tabular-nums), waveform-zoom-slider (important w/h/mt).
States -> hover:/disabled: variants; no-preflight borders -> explicit
[border:1px_solid_...]; exact px via arbitrary values.

Deleted as dead (zero usages anywhere): waveform-video-preview,
waveform-track-bg (+ nth-child + the 800px media-query track rows).

Kept (irreducible): .waveform-container and its
.waveform-container [data-id^="wavesurfer-region"] descendant rules (+ the
800px container/region media query) — those style WaveSurfer-generated DOM
we don't render in JSX, so they can't be utilities. The class stays as a hook.

index.css net -55 lines (+7/-62).

Cascade-correctness verified live (Playwright getComputedStyle, both
stylesheets loaded): new utilities reproduce the pre-migration computed styles
exactly. Caught two subtleties: (1) controls margin-top is 3px (unlayered
wfm-controls already wins over the old 4px), so no mt utility is added;
(2) referencing var(--chrome-font-mono) in a class string tripped the global
[class*="chrome-font-mono"] selector (adds slashed-zero + ss02) — switched the
time font to var(--font-mono) (identical stack) to avoid the substring match.
Screenshot pixel-diff old vs new = 0 (AE). Updated record_promo.js's fallback
selector (.waveform-controls -> [aria-label="Playback controls"]).

Gates: oxlint 0, oxfmt clean, vite build, vitest 641 pass, test:visual 48 pass,
bun install --frozen-lockfile no change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 00:20:33 +05:30
093e85e47a feat(ui): migrate ss-*/file-drag to utilities, trim index.css (P4) (#815)
Move the searchable-select (`ss-*`) and file-dropzone (`file-drag`) global
class families out of `src/index.css` into inline Tailwind utilities, then
delete the now-dead rules (-131 lines net in index.css).

- SearchableSelect.jsx: trigger/label/chevron/popup/search/list/group-label/
  option (incl. the highlight + selected + selected-highlighted cascade)/
  kind-icon/check/empty/more all rendered with token utilities + arbitrary
  var()/px values; no-preflight borders made explicit; `:focus`/`:hover` and
  the `::-webkit-scrollbar` pseudo-elements moved to Tailwind variants. The
  `.ss-sm/.ss-md .ss-trigger` descendant rules collapse to a size-conditional
  class on the trigger. `ss-wrap` keeps its class *name* only (its style is now
  utilities) because residual.css targets `.voice-selector > .ss-wrap` via a
  cross-file child combinator — deleting the name would break VoiceSelector
  layout.
- AudioMethodPanel.jsx: `.file-drag` (+ `:hover`/`.is-dragging`/`p`) → utilities;
  `is-dragging` stays a JS-toggled marker matched via `[&.is-dragging]:`. The
  out-of-scope, unlayered `.clone-drop-zone` padding override still wins.
- index.css: removed the `.ss-*` block, the dead `.ss-popover/.ss-menu/
  .ss-dropdown/.ss-item/.ss-highlighted` rules (zero JSX usages), and both
  `.file-drag` blocks, leaving migration breadcrumbs.

Verified live (Vite + Playwright/chromium): the Clone screen's dropzone and an
open SearchableSelect popup (search box, POPULAR group label, highlighted
option) render coherently. Gates: oxlint 0, oxfmt clean, vite build OK,
vitest 641 pass, visual suite 48 pass, `bun install --frozen-lockfile` no-op.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 00:17:30 +05:30
630912df55 feat(ui): migrate history-* to utilities, trim index.css (P4) (#814)
P4 of the shadcn/Tailwind migration for the `history-*` global class
family. The family is a shared, cross-file-composed component system
used across WorkspaceHistory, Sidebar, WorkspaceProjects and
WorkspaceVoices, so most of it is irreducible to per-usage utilities.

Migrated the one cleanly-isolable class:
- `.history-row-head` -> `flex items-center justify-between gap-2 min-w-0`
  (pure flex layout; no variants, pseudo-elements, descendant selectors,
  or cross-file/selector coupling). Converted all 9 usages, deleted the
  index.css rule (now zero usages). Verified in the running app that the
  utilities compute byte-for-byte identically to the old rule
  (display:flex / center / space-between / gap 8px / min-width 0).

Kept (composed cross-file / irreducible) and documented for later:
- `.history-item` (::before accent bar, descendant hover-reveal,
  `.project-active` compound, `--row-accent` set inline + `--dub`
  variant in Sidebar.css, duplicate !important defs)
- `.history-panel` (selector target of out-of-scope
  `.app-container > .history-panel` / `.glass-panel.history-panel`)
- `.history-kind` / `.history-meta` / `.history-title` / `.history-subtitle`
  (each has `--audio` / `--locked` / `--clamp`/`--expanded` / `--italic`/`--seed`
  variants defined in Sidebar.css)
- `.history-actions` (revealed via `.history-item:hover/:focus-within`
  descendant selector)
- `.history-action-btn` / `.history-action-icon` (compound `.accent`/`.danger`
  hover modifiers; ~30 usages; kept whole as a cohesive subsystem)

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 00:16:41 +05:30
3d17104228 feat(ui): migrate chip/preset/tag classes to Button variants/utilities, trim index.css (P4) (#813)
P4 shadcn/Tailwind migration of the chip/preset/tag global class families out
of src/index.css and onto their components as Tailwind utilities.

- personality-chip (+ __icon, + .active): -> token utilities inline in
  clone/DesignMethodPanel.jsx (PCHIP_* consts). Active stays chrome-accent
  (pink); icon span -> inline-flex items-center. The cross-file
  `.starting-points__strip .personality-chip { flex:0 0 auto }` in
  CloneDesignTab.css moved onto the chip as the `flex-none` utility and the
  dead rule was removed.
- chip-group .chip (+ :hover/.active) and the chip-group container: chips ->
  token utilities (CHIP_* consts) in DesignMethodPanel.jsx; the container's
  flex layout -> `flex flex-wrap gap-1` utilities. The `chip-group` class name
  is KEPT on the container purely as a JS hook (CloneDesignTab's roving-tabindex
  keyboard nav does `closest('.chip-group')`).
- tag-btn (Insert-menu token chips): -> token utilities in clone/ScriptPanel.jsx
  (TAG_BTN const), preserving the mono face. Removing tag-btn's `!important`
  un-masks the intended `.clone-auto-extract-btn` green on the [CMU] button
  (author intent restored; palette-coherent).
- preset-btn: had ZERO usages -> both rule blocks deleted.
- The shared 10x a11y focus ring is reproduced on the migrated chips via a
  `focus-visible:[outline:2px_solid_var(--chrome-accent)]` utility, on top of
  the app's global `:focus-visible` ring.

Kept (irreducible): the shared `.personality-chip:focus-visible, .chip:focus-visible,
...` a11y rule (groups out-of-scope selectors); `.chip-auto`, `.preset-grid`,
`.tags-container`, `.personality-strip` (out of scope, still used).

index.css: +13 / -126 (net -113). Verified live (Clone "By design": personality
chips, identity chip-groups, Insert tag popover) before/after — pixel-coherent.
Gates: oxlint 0, oxfmt clean, vite build, vitest 641 pass, test:visual 48 pass,
bun --frozen-lockfile no change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:48:19 +05:30
992eb67143 feat(ui): migrate hq-* classes to utilities, trim index.css (P4) (#812)
Move the header "quick" chrome (hq-*) global class families out of
src/index.css onto Tailwind utilities on their sole consumer, Header.jsx,
then delete the now-dead rules. No visual change (verified live below).

Migrated families: hq-col-* (layout columns), hq-logo-*, hq-breadcrumb-sep,
hq-view-* (breadcrumb title/dot/kicker/label/project + icon), hq-stats*
(readout + status badge override), hq-flush-btn/reload-btn, hq-flush-dropdown*
(portalled memory dropdown), hq-wave/hq-wave-bar (mini waveform). The three
@keyframes (flush-slide, hqPulse, hqBounce) are kept in index.css and driven
via [animation:...] arbitrary utilities.

- no-preflight: borders set explicitly with [border:...] arbitrary props.
- Badge override (hq-stats__status-badge) uses important modifiers (foo!) to
  beat the primitive's own utilities.
- @media responsive rules become max-[Npx]: variants on the elements. Tailwind
  v4's max-[N] compiles to `not all and (width>=N)` = strictly `< N`, whereas
  the original `@media (max-width: N)` is `<= N`; bumped each breakpoint +1px
  (e.g. 820 -> max-[821px]) so the boundary pixel matches exactly.
- The dead `.hq-scale` rule (zero usages) is dropped; the surviving non-hq
  @media rules (.header-area reload/wordmark hide) stay in index.css.

index.css: 224 lines removed, 2 added (net -220).

Verified live (vite :3922, Playwright chromium, backend :3900 stubbed):
header at 1600/1000/820px + flush dropdown open, before vs after pixel-diff —
820px identical (0px); residual sub-1% diffs at other widths are purely the
live pulse-dot / wave-bar animation phase (the only red regions in the diff).
Gates green: oxlint 0, oxfmt clean, vite build, vitest 641 passed,
bun install --frozen-lockfile no change, test:visual 48 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:44:37 +05:30
b14ad0cc1a feat(ui): migrate nav-rail/rail-btn to utilities, trim index.css (P4) (#811)
Move the `nav-rail` + `rail-btn` global class families out of
src/index.css into Tailwind utilities on NavRail.jsx, deleting the
entire 120-line nav-rail CSS block.

- `.rail-btn` / `:hover` / `.active` (+ accent `::before` indicator bar)
  → utilities on the shared RailBtn button; active state and the
  edge-indicator side are driven by props (`active`, `side`) instead of
  the `.nav-rail.rail-right` descendant selectors.
- `.rail-label` tooltip → group-hover utilities; flips edge by `side`.
- `.rail-flip` and `.donate-pill` (+ `donate-pill__heart`, reduced-motion)
  → utilities, incl. `motion-reduce:` for the heart.
- `.nav-rail .rail-top` / `.rail-bottom` → flex utilities.

The `nav-rail` CLASS is retained on the <aside> purely as the layout
hook the out-of-scope `.app-container > .nav-rail` grid rules position
by (those selectors are unlayered, so they still win over the layered
utilities); only its visual rules are deleted.

No-preflight safe: borders use explicit per-side `[border-*:1px_solid_…]`
shorthands (the flip button uses four independent side shorthands so the
top hairline can't be reset by a `border` shorthand override).

Verified: live before/after pixel-diff of the rail on Launchpad +
Gallery is pixel-identical (AE=0). oxlint/oxfmt/vite build clean,
vitest 641 passed, frozen lockfile unchanged.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:43:42 +05:30
c7220a45ab feat(ui): migrate Launchpad lp-* classes to utilities, trim index.css (P4) (#810)
Part 4 of the shadcn/Tailwind migration. Moves the Launchpad's static
layout/typography lp-* global classes from src/index.css onto the
component as Tailwind utilities (token-referencing arbitrary var()
values to preserve exact spacing/colour, explicit border shorthand for
the no-preflight setup, max-[900px]/max-[640px] variants for the former
@media rules), then deletes the now-unused rules from index.css.

Migrated + deleted: lp-hero (+__row/__col/__kicker-row/__wave-group),
lp-kicker, lp-hero__title (+em), lp-hero p / .lp-pill, the dead
.lp-underline rule, lp-actions (grid container), lp-section,
lp-section-title (+::after divider via after:), lp-section__grid,
lp-col, lp-proj-icon--* tints, lp-proj-meta--italic, lp-files__head/
__grid + lp-view-all + lp-file-card, lp-locked-badge, lp-empty (+__inner/
__bars/__hint), lp-dub-thumb, lp-demo-callout (+__icon/__btn),
lp-project-card (+ .proj-icon/info/name/meta/action), lp-ab-compare, and
the unused lp-action-card__emoji.

Kept (reported, not forced):
- Cross-file shared, reused by ContactPage/SupportPage/DonatePage.css/
  EnterprisePage.css: .lp-aurora, .lp-aurora__blob(+--pink/green/amber),
  .lp-hero__sweep (+ their @keyframes).
- ::pseudo / structural-selector / cursor-tracking component that can't be
  flat utilities: the .lp-action-card family + .lp-glow-layer
  (::before spotlight, ::after breath ring, nth-child stagger), .lp-animate.
- @keyframes-driven: .lp-wave-bar, .lp-hero__halo, and all @keyframes
  (lpDrift1-3, lpHeroHalo, lpHeroSweep, lpBreath, lpFadeUp, lpWaveBeat) +
  the prefers-reduced-motion block.

The bare `h1,h2,h3,h4` rule is unlayered, so the hero title's serif
font-family + letter-spacing utilities use `!` to win the cascade over it.

Verified: Launchpad landing screenshot is pixel-identical before/after
(Playwright chromium, animations disabled). oxlint 0, oxfmt clean, vite
build, vitest 641 passed, test:visual 48 passed, frozen lockfile unchanged.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 23:42:59 +05:30
4e03e363ed feat(ui): migrate settings/form/row global classes to utilities, trim index.css (P4) (#808)
P4 of the shadcn/Tailwind migration. Targets the settings/form/row LAYOUT
globals in src/index.css:

- .settings-log: converted its sole usage (LogsTab.jsx) to Tailwind utilities
  (bg/border/rounded/padding/max-h/overflow/font-mono/whitespace), then deleted
  the rule. --chrome-font-mono is an alias of --font-mono, so `font-mono` is
  exact parity; no visual change (live-verified on the Logs tab).
- .settings-section + .settings-section h2 and .settings-row(.label/.value/
  :last-child): zero remaining usages — superseded by the st-section primitive
  (components/settings/primitives/SettingsSection.jsx) in an earlier wave.
  Deleted as dead code.

Left BLOCKED (cross-file/cross-wave contracts, not forced):
- .settings-page / .settings-page h1 / .settings-page .settings-subtitle —
  extended by pages/Settings.css via descendant selectors and a media-query
  grid override that depend on the class living in the DOM.
- .label-row / .label-icon — owned by the clone/dub workspaces (out of scope),
  extended in CloneDesignTab.css and DubTab.css.

Verified: oxlint (0), oxfmt clean, vite build, vitest (641 passed),
bun install --frozen-lockfile (no change), test:visual (48 passed), and live
Settings screenshots (General/Appearance/Models/Engines/Credentials/Logs)
before-vs-after coherent.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:38:35 +05:30
40a97daa9a feat(ui): migrate misc global helper classes to utilities, trim index.css (P4) (#807)
P4 of the shadcn/Tailwind migration — eliminate small, self-contained MISC
global helper classes from src/index.css by converting their raw-className
usages to Tailwind utilities, then deleting the dead rules.

Migrated + deleted:
- .grid-2  (1 usage, AudioMethodPanel.jsx) → grid grid-cols-2 gap-[6px]
  max-[700px]:grid-cols-1, preserving the responsive single-column collapse.
- .grid-4  (1 usage, clone/ActionBar.jsx) → grid + arbitrary
  [grid-template-columns:repeat(auto-fit,minmax(120px,1fr))] gap-[6px]
  max-[500px]:grid-cols-2, preserving the responsive collapse.
- .val-bubble (7 usages, clone/ActionBar.jsx) → text-[0.65rem] bg-black/35
  px-[5px] py-px rounded-[3px] explicit border (preflight is disabled) +
  [font-variant-numeric:tabular-nums].
- .grid-3 was already dead (no base rule, no usages — only stray media-query
  overrides) and is dropped alongside the grid-2/grid-4 collapse block.

index.css net -10 lines. The other class families in this file are
component-scoped (hq-*, lp-*, ss-*, segment-*, waveform-*, settings-*, etc.)
or owned by other waves/agents, so they were left untouched.

Verified: Clone tab (base + Production Overrides expanded) screenshots are
pixel-identical before/after. Gates: oxlint 0, oxfmt clean, vite build,
vitest 641 pass, visual suite 48 pass, bun install --frozen-lockfile no-change.

Part of a HELD batch — do not merge standalone.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:37:37 +05:30
12f125c77e feat(ui): migrate global .ui-btn-* classes to shadcn Button, delete ui/Button.css (P4) (#805)
P4 of the shadcn migration. Removes the global `.ui-btn*` button
design-system family (the last raw-className button class set still
applied directly in JSX) by routing every consumer through the
shadcn-backed Button component / `buttonVariants()` helper, then deletes
the now-dead stylesheet.

Migrated — AudiobookTab.jsx (9 raw `.ui-btn*` sites):
- `<button>` actions (Preview plan / Create / cover-remove / lex-remove /
  Add word / chapter-preview) → `<Button variant={subtle|primary|icon}>`.
- non-<button> elements that can't be the component (file-picker `<label>`s,
  the download `<a>`) → shadcn `buttonVariants({ variant:'subtle' })`
  className, preserving label/anchor semantics + href/download/file input.
- onClick / disabled / aria-label / inline style all preserved verbatim.

Deleted:
- `src/ui/Button.css` (177 lines) — the entire `.ui-btn*` family; it had no
  remaining consumers (the Button component stopped emitting these classes
  in the earlier shadcn wrap). Dropped its import from `ui/Button.jsx` and
  refreshed the stale comment in `index.css` that referenced it.

Left for a later wave (blocked — see step 4):
- `.btn-primary` (index.css) — composed/extended by DubTab.css
  (`.dub-footer-btn` tone family, `.dub-change-row__cta`, `.dub-skel-gen-btn`
  all "sit on .btn-primary") + index.css media queries; deleting needs a
  refactor of the whole dub footer button subsystem. Risky, left intact.
- `.frs-btn` (FirstRunSetup.css) — custom LED indicator (`.frs-btn__led`) +
  `.is-armed` animated state with no Button-variant equivalent, spanning the
  entire first-run/setup flow (the project's Core Value). Left intact.

Part of a held batch — do not merge standalone.

Verified: live screenshots of Launchpad + AudiobookTab before/after (buttons
render on-palette — brand-pink primary, bordered subtle pills, correct
sizes); oxlint 0; oxfmt clean; vite build ok; vitest 641 pass; test:visual
48 pass; bun install --frozen-lockfile no change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:32:49 +05:30
e6067dfaec feat(ui): back Dialog/Tooltip/Tabs/Menu/Panel with shadcn (prop APIs preserved) (#803)
Migrate the five overlay/nav primitives in src/ui to compose the shadcn/ui
layer in src/components/ui, while keeping their existing prop surfaces and
exports byte-for-byte so no call site changes.

shadcn wraps the SAME @radix-ui primitives these already used (dialog, tooltip,
tabs, dropdown-menu) plus Card for Panel, so the swap is structural, not a
behavior change. No new dependencies — every required @radix package was
already pinned; package.json and bun.lock are unchanged.

- Added src/components/ui/{dialog,tooltip,tabs,dropdown-menu,card}.tsx
  (new-york style, themed through the existing index.css token bridge;
  DialogContent gains showCloseButton, TooltipContent gains showArrow, Card
  gains asChild so the wrappers can preserve their exact look/markup).
- Wrappers now delegate positioning + open/close animation to shadcn
  (Radix data-[state]/data-[side] + tw-animate-css animate-in/out). The GLASS
  look that utilities can't express in this Tailwind v4 build (backdrop-filter +
  layered gradients) stays in CSS, now keyed off shadcn data-slots / passed via
  the .ui-* classes — unlayered, so it wins over shadcn's bg-popover/bg-card.
- Dialog.css/Menu.css/Tooltip.css trimmed to surface-only (obsolete position +
  @keyframes removed); residual.css .ui-panel--glass unchanged.
- Tabs active/inactive state moved to data-[state] variants so it has the right
  specificity to override shadcn's TabsTrigger defaults; .ui-tabs/.is-active and
  all other cross-file hooks preserved.

Verified: oxlint (0), oxfmt clean, vite build, vitest (641 pass), bun
install --frozen-lockfile clean, and the visual suite (48 pass) — Panel + Tabs
render pixel-identical to existing baselines, so no baseline updates were
needed. Dialog/Menu/Tooltip are Radix-portal and not snapshot-harness-coverable;
verified via build + vitest + review.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:36:38 +05:30
86b30ece98 feat(ui): back Button/Badge/Progress/Segmented with shadcn (prop APIs preserved) (#799)
Migrate four OmniVoice UI primitives onto shadcn/ui foundations while keeping
their exact legacy prop APIs, so no call site changes.

- Button: thin wrapper over src/components/ui/button.tsx. Extends the shadcn
  CVA with the OmniVoice variants (primary/subtle/softGhost/danger/chip[+Active]/
  preset[+Active]/iconBtn[+Active]) + sizes (omniSm/omniMd/chip/preset/iconSm/
  iconMd), styled via palette token utilities. Maps variant/size/iconSize/active/
  loading/leading/trailing/block/ref. Each variant sets an explicit border
  (transparent where needed) since the app ships Tailwind without Preflight.
- Badge: new src/components/ui/badge.tsx; CVA carries the tones (neutral/brand/
  success/warn/danger/info/violet) + xs/sm sizes. Wrapper maps tone->variant and
  keeps the ui-badge / ui-badge__dot hooks so the Header --pulse animation works.
- Progress: new src/components/ui/progress.tsx (on @radix-ui/react-progress) with
  indicatorClassName + indeterminate support. Wrapper keeps per-tone gradients,
  sizes, shimmer overlay, and the ui-progress / has-shimmer / is-indeterminate
  hooks (residual.css keyframes unchanged).
- Segmented: new toggle.tsx + toggle-group.tsx (adds @radix-ui/react-toggle). The
  `seg` toggle variant reproduces the segmented look; wrapper preserves the
  items/value/onChange/size API.

residual.css: drop the obsolete .ui-seg__opt:focus-visible rule (focus now falls
through to the global ring). Badge pulse + Progress shimmer/indeterminate rules
kept (still needed).

Visual baselines: Badge/Progress/Segmented render byte-identical to before;
only Button baselines updated (shadcn markup differs in padding/radius, palette-
coherent across default/midnight/catppuccin). All gates pass: oxlint, oxfmt, tsc,
vite build, vitest (641), test:visual (48), bun install --frozen-lockfile.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:31:49 +05:30
836d69178c chore(dev): bun install before dev/desktop so pulled deps are present (#800)
`bun desktop`/`bun dev` assumed node_modules was current, so after pulling a
branch that adds a frontend dep (e.g. the shadcn migration's tw-animate-css /
@radix-* packages) vite failed with "Can't resolve '<pkg>'" until the user
manually ran bun install. CI never caught it (CI does a frozen install).

predev/predesktop now run `bun install` first (a no-op ~25ms when up-to-date),
so a fresh pull just works.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:30:33 +05:30
cb70c2b1af feat(ui): back Input/Select/Textarea/Slider with shadcn (prop APIs preserved) (#798)
P1 of the shadcn/ui primitive migration (docs/shadcn-migration.md): route the
OmniVoice form/data primitives through the shadcn components in
src/components/ui/* while keeping their exact exports and prop APIs, so no call
site changes.

- input.tsx: export `inputBaseClass` (the shell) with no behaviour change —
  ShadcnInput baseline stays byte-identical.
- New shadcn components: textarea.tsx, select.tsx (+@radix-ui/react-select),
  slider.tsx, table.tsx.
- src/ui/Input.jsx (Input/Textarea/Select/Field): Input/Textarea now render the
  shadcn components; a small `fieldSizeVariants` cva (named palette utilities,
  tailwind-merge-clean) restores the OmniVoice padding-based sm/md/lg scale +
  filled bg-bg-elev-2 over the shell. Select stays a NATIVE <select> wearing the
  same shell — DubSegmentTable/CompareModal/GeneralTab depend on
  onChange={(e) => …e.target.value}, which Radix's value-only Select would break;
  the Radix select.tsx is added for new call sites only.
- src/ui/Slider.jsx: wraps the shadcn Slider, keeping the number-based onChange +
  label/value-bubble chrome; track/thumb sized via the data-slot selectors.
- Table deliberately NOT rerouted: ui/Table.jsx is a flex-<div> chrome wrapper
  whose .ui-table*/.segment-table global classes are a SHARED CONTRACT used
  directly by ModelsTable/DubSegmentTable/EngineCompatibilityMatrix (virtualised
  lists needing the div/flex layout, not a semantic <table>). table.tsx is
  provided for new tabular data; Table.jsx + its globals are untouched. Its
  toolbar inherits the shadcn-backed Input/Button for free.

Verification: only the 3 Input-* visual baselines moved (palette-coherent across
default/midnight/catppuccin); Slider/Table stayed within tolerance. vitest 641
green, oxlint 0 errors, oxfmt --check clean, vite build green, root bun.lock
regenerated and bun install --frozen-lockfile in sync (Docker).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:28:47 +05:30
200a559183 feat(ui): shadcn/ui foundation + OmniVoice palette token bridge (Button/Input proof) (#797)
Lay the foundation for migrating OmniVoice's UI to clean Tailwind v4 + shadcn/ui
WITHOUT changing the look: shadcn primitives inherit the existing OmniVoice
palette (Gruvbox-pink default + every [data-theme] variant) through a semantic
token bridge. Foundation only — no existing component is replaced.

What landed:
- shadcn init for Tailwind v4 + Vite + React 19: frontend/components.json
  (new-york, rsc:false, tsx:true), src/lib/utils.ts (cn = clsx + tailwind-merge),
  and a @/* -> src/* alias in vite.config.js + tsconfig.json so future
  `npx shadcn add` resolves.
- Token bridge in src/index.css: a single `@theme inline` block maps shadcn's
  semantic vocab (--color-background/-foreground/-card/-popover/-primary/
  -secondary/-muted/-muted-foreground/-accent-foreground/-destructive/-input/
  -ring + --radius) onto the existing OmniVoice --color-* tokens. Because those
  tokens are re-declared per theme in ui/themes.css, theme switching recolors
  shadcn components automatically — no per-theme shadcn block. Existing
  --color-accent/--color-border and the --radius-* scale are left intact.
- Two proof components: src/components/ui/button.tsx + input.tsx (verbatim
  shadcn new-york), rendered across default/midnight/catppuccin in the visual
  harness with committed baselines (brand-pink / purple / lavender confirmed).
- New deps: class-variance-authority, clsx, tailwind-merge, tw-animate-css,
  @radix-ui/react-slot. Root bun.lock regenerated; `bun install
  --frozen-lockfile` verified in sync (Docker-green).
- Migration plan at docs/shadcn-migration.md (bridge table, primitive->shadcn
  mapping, prop-compat wrapper strategy, staged waves, honest risk/effort).

Verified: vite build, typecheck:ci, oxlint (0 errors), oxfmt --check, vitest
(641 pass), test:visual (48 pass incl. 6 new baselines), frozen lockfile in sync.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:04:41 +05:30
30b3886f9f refactor(css): consolidate ~15 residual stub stylesheets into src/styles/residual.css (#795)
After the Tailwind v4 migration, ~15 component .css files were reduced to tiny
stubs holding only the few irreducible rules that can't be layered utilities
(@keyframes animations, focus-visible rings, a glass surface, a <select> caret,
::before/::after pseudos, attribute-selector overrides). Each still lived as its
own file + its own per-component import. They are all plain GLOBAL class
selectors, so the file boundary bought nothing.

This collapses them into one shared, intentionally-UNLAYERED stylesheet
(src/styles/residual.css), loaded once at app root (main-app.jsx, right after
index.css to preserve cascade order) and once in the visual harness
(harness.jsx, which previously got these rules transitively via the component
imports). Rules are moved verbatim — byte-identical selectors/keyframes/values —
with a "from <Component>" provenance header above each block. No @layer wrapping,
so they keep beating Tailwind's @layer utilities exactly as before. Zero visual
change: all 42 visual-regression snapshots pass unchanged.

Net -14 .css files (68 -> 54): 15 stubs removed, 1 consolidated file added.

Deleted stub stylesheets (import removed from each component .jsx):
- ui/Badge.css            (.ui-badge--pulse dot animation)
- ui/Input.css            (.ui-select native caret)
- ui/Panel.css            (.ui-panel--glass backdrop surface + ::before)
- ui/Progress.css         (shimmer ::after + indeterminate keyframes)
- ui/Segmented.css        (.ui-seg__opt:focus-visible ring)
- components/AudioTrimmer.css         (.audio-trimmer layout)
- components/DemoPresetGrid.css       ([aria-pressed] active preview)
- components/MultiLangPicker.css      (.multi-lang__drop + mlp-in keyframes)
- components/ReadinessChecklist.css   (glass panel + rc-spin keyframes)
- components/TranscriptionPicker.css  (row hover/focus-visible combinators)
- components/UpdatesPanel.css         (updates panel chrome)
- components/VoiceSelector.css        (combinators + spin keyframes)
- components/settings/ApiKeysPanel.css(.apikeys-row/badge test contract)
- pages/ToolsPage.css                 (h1 + code/pre typography overrides)
- components/BootstrapSplash.css      (comment-only, no rules; import dropped)

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:22:59 +05:30
9d79bb8e36 feat(ui): convert more DubTab CSS to Tailwind (wave 2, live-screenshot-verified) (#794)
Second-wave CSS->Tailwind conversion of DubTab, building on wave 1 (#788).
Removes 119 more lines from DubTab.css (919 -> 800) by moving the
stateless/standalone idle-skeleton rules into utilities in IdleSkeleton.jsx.

Every conversion was proven pixel-identical against the LIVE app (real Dub
screen on a dev server, not the isolated component harness). A throwaway
Playwright spec captured baselines of three reachable Dub states, the rules
were converted, and the same states were re-shot and pixel-diffed with
maxDiffPixels:0 (exact). States verified:
  - idle drop-zone (drop-zone leaves, URL ingest row, landing options)
  - idle + Advanced expanded (landing-adv field row)
  - file-loaded skeleton via setInputFiles, no backend upload (skel settings,
    skel table cells/headers/hint, cast strip, stepper)

Converted (base/standalone rules -> utilities): dub-idle-drop__lines/__title/
__sub, dub-ingest-row + __input, dub-idle-upload-label, dub-hidden-file,
dub-landing-opts + __label, dub-landing-opts__lang base, dub-landing-adv +
__field base, dub-cast base + __row + __kicker/__label base + --muted__chip,
dub-skel-settings, dub-skel-field/--sm, dub-skel-translate-btn,
dub-skel-transcript-toggle, dub-inline-icon, dub-skel-cell-*/header-* cells,
dub-skel-hint, dub-skel-gen-row.

Deliberately LEFT as CSS (would regress, per the diff oracle / wave-1 doctrine):
anything with @keyframes/animation (dub-skel-bar shimmer, dub-idle-drop pulse),
:hover/state interplay (dub-ingest-row__cta.is-ready, dub-landing-opts__adv,
dub-cast__pair), and cross-file unlayered overrides that a layered utility
would lose to (dub-skel-table on .segment-table, dub-skel-row on .segment-row,
dub-skel-gen-btn / dub-change-row__cta on .btn-primary,
dub-skel-transcript-toggle__inner on .override-toggle,
dub-skel-cell-acts__icon on .segment-del, dub-speakers-input on .input-base,
dub-ghost-footer on .studio-panel). Class hooks were kept on elements whose
.dub-cast--muted / --grow / select descendant rules still need them.

Gates: oxlint (0), oxfmt --check (clean), vite build, vitest (641 pass),
bun install --frozen-lockfile (no change), bun run test:visual (42 pass).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 19:42:33 +05:30
ab87ef734d test(visual): extend harness to render panels/pages with mocked store/query/i18n (#793)
The visual-regression harness could only snapshot pure leaf components.
Pages and settings panels couldn't render because they depend on the
Zustand store, react-i18next, react-query, and direct api/* fetches — so
the CSS→Tailwind migration had no pixel safety net for them.

Add an OPT-IN provider wrapper (providers.jsx): a spec declaring a
`providers` block gets a seeded Zustand store, forced-English i18n, a
snapshot-tuned QueryClient pre-filled via setQueryData, and an optional
window.fetch stub for components that call api/* directly. Nothing runs
for pure leaf specs, so existing leaf baselines are byte-for-byte
unaffected (verified: 0 leaf PNGs changed on regenerate).

Prove it on three CSS-heavy settings panels, each x3 themes:
- AppearancePanel — store + i18n only
- GeneralTab — store + i18n + seeded useSystemInfo query
- StoragePanel — fetch-stubbed GET on mount

ModelStoreTab is documented as not-harness-able yet (live EventSource SSE
+ virtualized react-table + required props). No new deps. Suite stays
local-only (bun run test:visual), not a CI gate.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:53:09 +05:30
3d88779eff feat(ui): convert Settings + misc page CSS to Tailwind utilities (partial) (#791)
Mechanical, conservative CSS→Tailwind v4 migration of the safe layout/spacing/
typography 80% across the Settings page and several smaller pages. No intended
visual change. Kept in CSS (per the migration plan's "hard 20%"): @keyframes,
::before/::after, :has()/child/sibling combinators, glass/backdrop-filter,
!important, media/container queries, state-modifier specificity interplay, and
any rule that fights an unlayered global element rule (h1..h4 font/letter-spacing,
code/pre, a) which would beat @layer utilities.

Conventions followed: BEM class names retained alongside utilities so external
selectors and removal stay safe; only @theme tokens use named utilities
(text-fg, bg-bg-elev-2, rounded-lg, font-mono); --chrome-*/--space-*/--text-*/
--frs-* and exact pixels use arbitrary var()/px values; no-preflight borders via
[border:1px_solid_...]; transitions via arbitrary [transition:...].

Files (rules removed → utilities; rules kept = the hard 20%):
- ToolsPage: page/card layout → utils; kept h1, code/pre descendants.
- BatchQueue: page/cards/progress/meta/outputs → utils; kept h1, card status
  modifiers, progress-fill shimmer pseudo + keyframes.
- Transcriptions: header/list/detail/segments → utils; kept search input
  (+placeholder), list scrollbar, item hover/active interplay, h4 seg-title.
- Projects: page/header/toolbar/search/rail/body/content/empty → utils; kept
  title h1, search input, view-toggle + rail-item + card clusters, list-view
  descendants, content view modifiers.
- AudiobookTab: page/head/body/script/side/field/duo → utils; kept title h2,
  scoped .field-label, textarea/select descendants, @media collapse.
- Donate/Support/Enterprise (shared across SupportPage + ContactPage): page,
  content, hero subtitle, footer, social-proof, amounts, topbar, spacer,
  methods, chips, contact value, ent kicker/subtitle/why-grid/label/desc →
  utils; kept all animation/pseudo/state/color-mix/custom-prop chrome.
- SetupWizard: standalone swiz-slide/note/checks/loading + frs-embed → utils;
  left frs-coupled lib/hfbar rows in CSS (first-run, extends frs primitives).
- Settings: settings-muted/prose(base)/log-meta/log__empty/link-row/actions-row
  + models-row__progressline → utils in the settings/* tab components; kept the
  page grid/container-query shell, tab-rail rules, tables, rows, reco-banner.

Verified: vite build clean, oxlint exit 0 (only pre-existing warnings), oxfmt
--check clean, vitest 641/641 pass, bun install --frozen-lockfile no change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:23:45 +05:30
d88e57675d feat(ui): convert VoiceGallery + CloneDesign page CSS to Tailwind utilities (partial) (#792)
Mechanically convert the safe, low-risk page CSS of the Voice Gallery and
Clone/Design pages to Tailwind v4 utilities, removing each converted rule from
the page CSS so there is a single source of truth.

Scope (conservative, partial — complex rules left as CSS):
- VoiceGallery.css: pure flex/grid containers + text/ellipsis spans converted
  (voice-gallery, gallery-header, header-top, gallery-sub, gallery-search,
  search-row base, search-results-panel, panel-header, results-list, result-*,
  content-header, content-title base, count-badge, voice-list base, voice-info/
  name/meta/actions base, arch-head/title, archetype-name/sub/chips, arch-foot,
  archetype-section, load-more, import-explainer, community-explainer,
  submit-actions).
- CloneDesignTab.css: studio-def-col, clone-script-wrap, clone-insert-backdrop,
  clone-prod-col/check, clone-hear-demo-chip, clone-drop-row, clone-drop-filename,
  grid-2--indent, describe-voice-block margin / hint / feedback, starting-points
  (+__label), clone-sliders-col, clone-slider-kicker, identity-line__kicker/recipe,
  design-seed(+__row/__keep), clone-coachmark(+__icon/__msg), clone-profile-banner
  (+__label), clone-save-profile(+__row base).

Rules followed:
- No reliance on Tailwind preflight: borders use arbitrary [border:...]; only
  @theme tokens map to named utilities (bg-bg-elev-2, text-fg, text-success,
  rounded-lg/md), everything else (--chrome-*/--space-*/--text-* + literal px)
  stays exact via arbitrary var()/px.
- Left in CSS: keyframes, ::before/::after, :has/combinators, masks, scrollbar
  pseudo, !important, media queries, hover/border-heavy buttons & chips, and any
  rule overriding an unlayered base (.file-drag/.input-base/.studio-panel) that a
  @layer utility can't beat.
- Retained class names that anchor kept descendant selectors
  (search-row, clone-save-profile__row).

Verified: oxlint 0, oxfmt --check clean, vite build OK, 641 vitest pass,
bun install --frozen-lockfile unchanged.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:15:14 +05:30
3c17d19b7e feat(ui): convert misc component CSS (Sidebar/EngineMatrix/donate/…) to Tailwind utilities (#790)
Move the mechanical, self-contained layout/spacing/typography/simple-color CSS
of six leaf/misc components to Tailwind v4 utilities in their JSX, deleting the
now-redundant rules from each component .css. No intended visual change.

Conversion rules followed (matching the prior ui/ migration PRs):
- No preflight reliance: borders use `[border:1px_solid_…]`, button resets are
  replicated (border/background/padding) rather than assuming a base.
- Only @theme tokens become named utilities (font-sans/serif/mono, rounded-lg,
  text-fg…); --chrome-*/--space-*/--text-*/shadows use arbitrary `var()`/exact px.
- Component .css is unlayered and outranks @layer utilities, so a class is only
  converted when its rule is removed; classes still governed by an unlayered
  global rule (.input-base) or a remaining state rule keep their CSS.
- Kept in CSS: @keyframes, ::before/::after, :has()/child/sibling combinators,
  :hover/:focus-visible/.is-active states, gradients/box-shadow/glass, animation,
  !important, and @media. Class names are retained on the elements so those
  rules (and the test selectors) keep matching.
- Shared/other-owned classes left alone: Sidebar's history-*/save-btn (rendered
  by Workspace*), EngineMatrix's chip block (tested `.is-effective`, color-mix
  variants) and __table (Table primitive), all Pip animation classes.

Files (rules removed vs kept):
- Sidebar: tabs/badge/search/empty/section-title/icon-tile/subtitle/scroll/tile
  bases → utilities; kept .sidebar__tab (interactive), search-input (.input-base
  override), search-clear (!important), save-btn (shared), is-collapsed
  combinators, hovers. 286→182.
- EngineCompatibilityMatrix: matrix/head/title/body/row/cells/name/id/reason/
  hint/last-error/why/why-body/chips/result/tabs/empty → utilities; kept table,
  why-summary pseudo triangle, chip color system + tested .is-effective. 289→104.
- donate/Postcard: close/body/title/lead/goal-link/actions/cta/later/minor/star/
  optout bases → utilities; kept the animated card, ::before perforation, grain,
  stamp, hovers, keyframes, reduced-motion @media. 229→134.
- donate/DonateGoal (GoalBar): goal root/head/title/pct/track/caption/remaining/
  caption-met → utilities; kept fill/shimmer/pip animations, --met/--mini
  overrides, amounts-strong combinator, all Pip classes. 179→128.
- VoiceSelector: container/adornments/btn base → utilities; kept > .ss-wrap
  combinator, btn hover/disabled, spin animation. 45→23.
- TranscriptionPicker: search/list/row/text/meta/empty base → utilities; kept
  search>* and meta-span combinators, row hover/focus-visible. 29→10.

Verified: oxlint (0 errors), oxfmt --check clean, vite build, vitest (641
passed), bun install --frozen-lockfile (no change).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:14:32 +05:30
0b16481633 feat(ui): convert StoriesEditor + LogsFooter CSS to Tailwind utilities (partial) (#789)
Migrate the safe, mechanical layout/spacing/typography CSS of two components to
Tailwind v4 utilities, leaving the hard-to-express rules in their .css files.
Conservative + partial by design (per the migration plan §8): no preflight is
assumed, so borders/transitions/chrome tokens stay as arbitrary properties
referencing the exact original vars (`[border:1px_solid_var(--color-border)]`,
`[color:var(--chrome-fg-muted)]`), @theme tokens use named utilities
(text-fg, bg-bg-elev-2, rounded-sm/md, text-accent/brand, bg-border), and every
non-@theme value (--chrome-*, --space-*, --text-*) is exact px or `var()`.

StoriesEditor.css 525 -> 349 (-176): converted the editor shell, header,
subtitle, toolbar groups/divider, stats/footer, empty state, cast/split panels,
the panel title, voice/cast dot, and the tone/drawer containers. KEPT: the h2
title (global `h1..h4` element rule is unlayered and would beat a `font-serif`
utility), the `.stories-track` grid + its hover/active/drag combinators, all
native controls (textarea/select/range + their focus states), every button
(UA reset + hover/disabled/`--on`/`--delete` states), the chapter bar (hover
combinators), the `::-webkit-scrollbar` pseudos, and the `[data-char]` color
palette attribute selectors.

LogsFooter.css 507 -> 376 (-131): converted the resize handle, top bar,
left/right clusters, the LOGS title, the count-badge base, the log-line base +
icon + line-text base, and the notification panel (body/item/icon/content/msg/
action). KEPT: the `.logs-footer` fixed shell (anchor for the
`.app-container .logs-footer` inset combinators + the <=600px media query),
every button (toggle/pill/version/discord/donate/icon-btn with hover/disabled/
animations), the severity color modifiers + their descendant overrides
(`.logs-footer__line--error .logs-footer__line-text`, badge/item variants,
clickable hover), the body scrollbar pseudos, the `notif-content strong` rule,
and all @keyframes + the reduced-motion block.

Classes that remain referenced by kept CSS (combinators/pseudos/attrs) keep
their BEM class in the JSX alongside the new utilities; fully-removed rules drop
the class entirely. No class used elsewhere in the tree was removed (grep-checked
across frontend/src).

Verified: npx oxlint (0 errors), oxfmt --write src + --check . (clean),
vite build (ok), vitest run (641 passed), bun install --frozen-lockfile
(no changes).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:03:49 +05:30
ecd2545fcf feat(ui): convert DubTab page layout CSS to Tailwind utilities (partial; complex/stateful CSS kept) (#788)
Mechanically convert the low-risk layout/spacing/typography/simple-color rules
on the Dub section components to Tailwind v4 utilities, removing each converted
rule from DubTab.css so the unlayered page CSS can't shadow the utilities.

Converted (rule removed from CSS + utilities applied in JSX):
- DubHeader/IdleSkeleton: .dub-head strip, __filename/__meta/__project/__actions/__primary
- PrepOverlay: .dub-prep-overlay base, .dub-prep-chips base, __title/__note/__detail
- TranscribeOverlay: .dub-trans-overlay base, __head/__title/__bar
- DubFailureNotice: .dub-failure-notice + __hint/__actions
- DubFooter/IdleSkeleton: .dub-footer-banner, __badge-gap
- DubRightColumn: .dub-bulk-row__label-brand, .dub-lazy-fallback
- DubTab/IdleSkeleton: .dub-col, .dub-split-1, .dub-split-2
- IdleSkeleton: .dub-change-row, .dub-speakers-hint

Kept in CSS (left as-is, by the project's gotchas):
- .dub-head__title (coexists with the unlayered global .label-row it overrides)
- .dub-panel-col / .dub-ghost-footer (sit on .studio-panel, override its overflow/padding)
- .dub-change-row__cta (sits on .btn-primary, overrides its margin-top)
- .dub-trans-overlay__stats (targeted by the global tabular-nums rule)
- .dub-prep-bar/__fill, .dub-prep-chip, --large/--lg modifiers (combinators/state/animation)
- .dub-hidden-file (used outside the converted files)
- all keyframes/animations, ::before, :has/combinators, !important, media queries,
  chrome-token surfaces, the stepper, skeleton bars, footer-btn family, etc.

Tokens preserved exactly: spacing/text/chrome → arbitrary var()/px; @theme colors
+ radius + weight → named utilities. Borders use the [border:...] arbitrary form
since the app ships Tailwind v4 without preflight.

DubTab.css: 989 → 919 lines (92 CSS lines removed, 22 explanatory notes added).
Verified: oxlint 0, oxfmt clean, vite build, vitest 641 pass, bun --frozen-lockfile no-op.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:02:15 +05:30
930b403799 feat(ui): convert FirstRunSetup layout CSS to Tailwind utilities (partial; animations/states kept) (#787)
Converts only the clearly-mechanical, low-risk layout rules of the shared
"studio console" sheet (FirstRunSetup.css) to Tailwind v4 utilities in the
JSX consumers (FirstRunSetup, BootstrapSplash, SetupWizard). Most of the
1020-line sheet stays in CSS by design.

Converted (15 static layout-only rules, full property sets):
- containers: .frs__deck, .frs__col, .frs-panel, .frs__grid
- masthead: .frs__mast, .frs__mast-row, .frs__mast-meta, .frs__mast-selects, .frs-wsteps
- misc layout: .frs-opt__head, .frs__hw, .frs-row__gauge, .frs__foot-row,
  .frs-log__bar, .frs-banner__actions

Approach honoring the no-preflight setup (only theme.css + utilities.css
are imported): exact rem/px preserved via arbitrary values
(gap-[1.1rem], grid-cols-[minmax(0,7fr)_minmax(0,5fr)], etc.); each base
rule is removed from CSS (component CSS is unlayered and would otherwise
beat @layer utilities) and replaced with a one-line breadcrumb. Every
remaining override stays in CSS and still wins because it is unlayered:
responsive media queries (.frs__grid/.frs__mast-row/.frs__foot-row/
.frs-row__gauge), modifier classes (.frs__deck--focus,
.frs-banner__actions--end, .frs-wsteps--journey), and descendant rules
(.frs-row__gauge .frs-meter).

Kept in CSS (unchanged): all @keyframes/animations (rise, breathe, alarm,
hw-pulse, meter), ::before/::after, glass/masks, color-mix backgrounds,
hover/focus/state (.is-active/.is-armed/etc.), typography, and media
queries. Cross-file/combinator-bound classes (.frs-wnav, .frs-embed,
.frs-row*, .frs-check*) left as CSS.

Verification: oxlint 0, oxfmt --check clean, vite build OK, vitest 641
passing, bun install --frozen-lockfile no change.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:01:35 +05:30
4a6da3bf84 feat(ui): convert modal/dialog CSS to Tailwind utilities (#783)
Move the mechanical layout/typography rules of five modal/dialog/panel
components from their .css files onto JSX utilities (Tailwind v4). Exact
pixels preserved via arbitrary utilities referencing the same tokens/px;
no preflight, so UA resets (bg/border/padding) are replicated explicitly.
Overlays, positioning, open/close animations, state-class (.is-*) and
descendant selectors, gradients-as-state, media queries, and any
cross-file class are intentionally left in CSS.

- ExportModal: converted drawer head/handle/close, body, presets,
  preset-chip, kicker, tracks, section-head, track-row, track-label,
  tabs container, grid, field/field-head/-label/-hint, note, mt6,
  pkg-grid/-card(+ghost)/-head/-body, summary(+left/-name/-right),
  license-notice/-link. Kept: overlay, sheet+keyframes, track-quick
  (button descendant), track (input descendant + .is-on/.is-dub.is-on),
  tab (.is-active + hover), toggle (input descendant + --indent).
- CompareModal: converted drawer head/handle/title/close, body, foot,
  desc/head/audio/audio-empty. Kept: overlay, sheet+keyframes, and
  .ui-compare__grid (its responsive collapse is driven by a media query
  here AND in index.css — cross-file, STOP rule).
- BatchAddDialog: converted head/title/close, body, drop-hint,
  file-input, files/kicker/file-row/-name/-size/-x, settings, field,
  foot/estimate. Kept: overlay, card+keyframes, drop (.is-over state),
  select (overrides global .input-base), toggle (input descendant).
- SupertonicLicenseDialog: converted title, intro, sections, link,
  footer, actions. Kept: overlay, card + section (color-mix + unclassed
  h3/p/code descendants), buttons (color-mix :not(:disabled) states).
- NotificationPanel: converted the bell trigger + count badge (made the
  color/bg conditional in JSX to avoid Tailwind utility-ordering ties).
  Unused .notif-panel/.notif-item/.notif-hf-input blocks left as-is
  (pre-existing dead code; removal is out of scope for this refactor).

Verified: oxlint exit 0 (only pre-existing warnings), oxfmt --check
clean, vite build OK, 641 vitest tests pass, bun install
--frozen-lockfile unchanged.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:41:13 +05:30
16a8995a9a feat(ui): convert dub/demo component CSS to Tailwind utilities (#784)
Move mechanical layout/typography from five component CSS files onto JSX as
Tailwind v4 utilities. Conservative: rules that are shared across files, use
!important/color-mix/compound or descendant selectors, focus rings, font:inherit,
@media, or that would lose to unlayered index.css rules in the cascade are kept in
CSS. Verified visual equivalence, oxlint (0), oxfmt, vite build, and 641 vitest
tests; bun.lock unchanged.

DemoPresetGrid: fully converted grid/cards/buttons; CSS trimmed to only the
  .demo-preset-card__preview[aria-pressed="true"] state (attribute selector kept
  unlayered so it wins over the button's hover utilities).
DubbingDemo: converted head/title/dismiss/pane/caption/picker/chip/cta; kept the
  shared container base (reused by the loading state), the max-width:720px media
  query, and the input/pane-label-span/pane-video descendant + chip.is-active
  compound rules.
DictationDemo: converted head/title/lede/card/lang/script/actions/result-base;
  kept .dictation-demo and .dictation-demo__scripts (queried by
  DictationDemo.test.jsx), plus status/result variants with their descendants.
DubSegmentRow: converted the local cell badges/labels/time-spans/restore-button/
  checkbox; kept the shared .segment-* row/state classes (used by
  DubSegmentTable.css, index.css, IdleSkeleton.jsx), the text inputs (font:inherit
  + focus), and the select/range/actions cells whose unlayered input-base /
  input[type=range] siblings would otherwise beat utilities.
SegmentTrack: converted container/onsets/viewport-base/lane/label/handle-base/
  actions/action-btn/playhead; kept the box and its JS-toggled state variants,
  handle edges with hover/selected compounds, the self-scroll viewport modifier,
  the disabled compound, and the visually-hidden announce region.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:40:32 +05:30
5673e2e772 feat(ui): convert settings panel CSS to Tailwind utilities (#782)
Converts the clearly-mechanical CSS (layout/sizing/typography/simple
color+border+radius, simple hover/focus/disabled) in the settings panels
to Tailwind v4 utility classes on the JSX, mapping to @theme token
utilities + arbitrary var()/px values for exact-pixel parity. The app
ships without preflight, so borders use the `[border:1px_solid_…]`
arbitrary-property form (matching ui/Badge.jsx) to keep border-style.
No behavior change; verified by strict 1:1 mapping + build + full tests.

StoragePanel: fully converted → StoragePanel.css DELETED (import removed).
  field/input/buttons/restart/error all utilities; placeholder + focus-ring
  via placeholder:/focus-visible: variants.

SharingPanel: fully converted → SharingPanel.css DELETED (import removed).
  section/row/addr/btn(+ghost)/iconbtn/tailscale-*/qr/note/envname/portinput.

AppearancePanel: converted scale slider+readout, theme/font containers, and
  the range input (accent-color). KEPT in CSS: `.appearance-panel__row--fonts
  .st-row__control` (reaches into the SettingRow primitive), and the
  theme-dot + font-tile rules (stateful transitions, multi-layer box-shadow
  rings, is-active state) — not 1:1 utility-safe.

ApiKeysPanel: converted error/rows/head/name/meta/set/unset/whoami/masked/
  actions/input/buttons/clear-dialog/checkbox. KEPT in CSS:
  `.apikeys-row`, `.apikeys-row--active`, `.apikeys-badge`,
  `.apikeys-badge--active` — ApiKeysPanel.test.jsx selects these by class
  name (cross-file contract; STOP rule).

VoicePanel: converted the warn banner + most of the speech-model dropdown
  (dropdown/trigger/name/list/item/itembtn/check/body/itemtop/itemname/
  size/itemdesc/progresstext/action/iconbtn). KEPT in CSS:
  `.voicepanel__row--model .st-row__control` (primitive descendant),
  `.voicepanel__dd-chev`/`.is-open` (transform transition — Tailwind
  `rotate-*` targets the `rotate` property, not `transform`, so it wouldn't
  animate), `.voicepanel__dd-progress` + `> :first-child` (child combinator),
  and `.voicepanel__spin` + `@keyframes` (animation).

PerformancePanel: UNTOUCHED. PerformancePanel.css is a de-facto shared
  stylesheet — `.perfpanel`, `.perfpanel__error`, `.perfpanel__row`,
  `.perfpanel__badge`, `.perfpanel__help` are used by 6 other panels
  (MCPBindings, LLMEndpoint, Refinement, RemoteBackend, HFMirror,
  Pronunciation), so the STOP rule leaves the whole file as-is.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:39:15 +05:30
349a40a261 feat(ui): convert standalone widget CSS to Tailwind utilities (#781)
Move the clearly-mechanical CSS (flex/grid, spacing, sizing, typography,
simple colors/borders/radii, and simple hover/disabled states) for seven
standalone widgets onto their JSX as Tailwind v4 utilities. Animation
(@keyframes), glass/backdrop-filter, pseudo-element/compound/sibling
selectors, !important, media queries, and any class referenced from
another file are left in CSS verbatim. Exact pixels/colors are preserved
via @theme token utilities plus arbitrary var()/px values; transitions use
arbitrary-property syntax so the timing function stays identical (Tailwind's
transition utilities inject a different default ease). No preflight is loaded,
so every converted border pairs an explicit border-solid/border-dashed + color.

- NetworkToggle: fully converted; NetworkToggle.css deleted and its import
  removed (all classes were local).
- FloatingPill: converted the static content/label/meta/timer/error/progress
  track + dismiss button; kept the pill base (animation+glass+fixed pos), dot,
  progress-fill (base + indeterminate !important/animation), keyframes, the
  prefers-reduced-motion block, and the --done/--error descendant overrides.
- AudioTrimmer: converted all audio-trimmer__* parts + trim-field*; kept the
  .audio-trimmer base rule (also targeted by unlayered overrides in index.css).
- ReadinessChecklist: converted title/list/item/status-layout/label/detail/
  fix/all-pass; kept the glass base, the rc-spin keyframe, and the dynamic
  status--pass/warn/fail/loading color+animation modifiers.
- MultiLangPicker: converted chips/add/summary/search/list/section/option;
  kept the .multi-lang__drop dropdown (animation + shadow) and mlp-in keyframe.
- WorkspaceVoices: converted only the local wv__active*/wv__empty-cta active-
  voice card; kept wv/wv__head/wv__title/wv__search*/wv__scroll/wv__empty/
  wv--collapsed/wv__rename-input (shared with WorkspaceProjects.jsx).
- WorkspaceHistory: converted the local wh/wh__* panel chrome (active chip
  state expressed as a mutually-exclusive ternary since utilities are equal
  specificity); kept the studio-with-history/studio-right/shell-narrow/
  shell-mini layout rules (referenced by App.jsx, index.css, and tests).

Verified: oxlint exit 0, oxfmt --check clean, vite build OK, 641 vitest pass,
bun install --frozen-lockfile clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:35:53 +05:30
5b6f49ee0c feat(ui): convert Button/Panel/Input/Menu to Tailwind utilities (visual-verified) (#780)
Migrate three UI leaf primitives from component .css to Tailwind v4 utilities,
each verified pixel-identical against the visual-regression harness across all
three baselined themes (default / midnight / catppuccin).

Because the app ships Tailwind v4 WITHOUT Preflight and themes override the
design tokens, colors/shadows/borders/transitions are expressed as arbitrary
*properties* (`[prop:value]`) referencing the exact original CSS variables
(avoiding `--tw-*` composition and color/length type ambiguity), while
@theme-mapped tokens use named utilities (text-fg, bg-bg-elev-2, rounded-lg,
text-danger…) which resolve to the same `var(--…)` and track themes. The
harness renders resting state, so hover/focus/active are converted faithfully
but not pixel-gated.

- Button: component is now fully utility-driven and no longer emits `.ui-btn*`
  classes. Button.css is RETAINED unchanged because AudiobookTab.jsx consumes
  `.ui-btn--{subtle,primary,icon}` as raw classNames (out of scope to refactor);
  keeping the component class-free avoids double-application.
- Panel: layout/border/radius/padding/header/title/actions + solid & flat
  variants → utilities. Panel.css trimmed to the glass variant only
  (backdrop-filter + layered gradient surface + ::before highlight, which
  utilities can't express). The header+body top-padding sibling rule is
  reproduced via a conditional `pt-` when a header is present.
- Input: shared input/textarea/select shell, sizes, states, and the Field
  wrapper → utilities. The `:has(.ui-field__icon)` padding rule is reproduced by
  cloning the control with `pl-` when an icon is present. Input.css trimmed to
  the native <select> caret (SVG data-URI background) only. Added Input to the
  visual harness with a representative spread; baselines committed.
- Menu: left as CSS. It is a Radix dropdown whose content renders through a
  Portal to document.body (outside the harness snapshot root #visual-root) and
  only renders when open + collision-positioned, so it cannot be captured in
  isolation here; its surface is also dominated by keep-as-CSS features
  (backdrop-filter glass, gradient, @keyframes pop-in, box-shadow token).

Verified: bun run test:visual (18 passed), npx oxlint (0 errors),
oxfmt --check (clean), vite build (ok), vitest (641 passed),
bun install --frozen-lockfile (no changes).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:13:39 +05:30
740a690644 feat(ui): convert Dialog/Slider/Table/Tabs to Tailwind utilities (visual-verified) (#779)
Continue the component CSS -> Tailwind v4 utility migration for UI group 3.
Added Slider, Table, and Tabs to the visual-regression harness (specs.jsx +
manifest.ts) and committed machine-local baselines, then converted each
component, proving the result pixel-identical with `bun run test:visual`.

- Slider: fully converted; Slider.css deleted (no keyframes / complex
  selectors). Token + arbitrary-value utilities preserve exact pixels; thumb
  hover/active/focus-visible and the multi-easing transition are kept faithful
  via arbitrary-property utilities. Visual-verified across all 3 themes.

- Tabs: fully converted; Tabs.css deleted. pill/underline variants, size,
  active and hover:not(active) states mapped to conditional utility sets. The
  `ui-tabs* / is-active / ui-tabs__icon` class names are retained as inert
  hooks so Settings.css's unlayered overrides (`.ui-tabs.settings-tabs-ui …`)
  keep winning over the layered utilities — Settings tab rail unchanged.
  Visual-verified across all 3 themes.

- Dialog: partial conversion. Header / title / body / footer box-model +
  typography and per-size max-width converted to utilities; the glass
  gradient surface, backdrop-filter, fixed centering, and open/close
  @keyframes remain in Dialog.css (cannot be reduced to utilities). NOT
  visually verified: Radix Portal + position:fixed render the dialog outside
  the harness's #visual-root, so it can't be snapshotted in isolation;
  verified instead by 1:1 token equivalence + build + unit tests.

- Table: LEFT AS CSS (STOP rule). Its classes are a shared CSS contract, not
  a private leaf — ModelsTable.jsx renders `ui-table-header`/`ui-table-header__cell`
  directly without the component, and DubSegmentTable.css,
  EngineCompatibilityMatrix.css, Settings.css, and index.css all hook those
  global classes. Removing Table.css would break them, so converting yields no
  safe net benefit. Added to the harness with a baseline for a future pass.

Verified: oxlint (0 errors), oxfmt --check (clean), vite build, vitest
(641 passed), bun install --frozen-lockfile (no changes), test:visual (24
passed).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:50:02 +05:30
9bb3cc2664 feat(ui): convert Badge/Segmented/Progress to Tailwind utilities (visual-verified) (#778)
Migrate three UI leaf primitives from component .css to Tailwind v4 utility
classes, mapping colors/radii/fonts to the @theme token utilities and using
arbitrary values (px / var() / color-mix / gradients) to preserve exact pixels.
Each conversion is proven pixel-identical to its pre-conversion baseline by the
Playwright visual-regression harness across all three themes.

- Badge: base, sizes, tones, and dot moved to utilities. Kept the
  `.ui-badge--pulse .ui-badge__dot` rule in CSS — it is driven by an
  externally-applied parent class (Header status badge) + global `pulse`
  keyframes, which a utility on the component can't express.
- Segmented: container, options, sizes, hover (Radix data-state=off) and
  active (data-state=on) moved to utilities. Kept `.ui-seg__opt:focus-visible`
  in CSS: the global `:focus-visible` rule is unlayered and would otherwise win
  over a layered utility, so the component override must stay unlayered too.
- Progress: track, sizes, fill, and per-tone gradient fills moved to utilities.
  Kept the shimmer `::after` + indeterminate descendant rule + both `@keyframes`
  in CSS (pseudo-elements / keyframes are not expressible as utilities).
- Tooltip: left as CSS. Its content renders through a Radix Portal into
  document.body, outside `#visual-root` (the only element the harness snapshots),
  so a conversion can't be visually verified — left untouched per the rule to
  not force an unverifiable change.

Added Segmented + Progress to the visual harness (specs.jsx + manifest.ts) with
representative variants/states and committed their baselines. Badge was already
in the suite; its baseline is unchanged (byte-identical).

Verified: oxlint (0 errors), oxfmt --check (clean), vite build, vitest
(641 passed), bun install --frozen-lockfile (no change), test:visual (21 green).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:35:51 +05:30
a0a4bcc903 test(visual): add Playwright component visual-regression baseline for CSS migration (#776)
Gating prerequisite for the CSS -> Tailwind v4 migration: a pixel-for-pixel
safety net so each utility conversion can be verified against a known-good
baseline. There were previously no visual tests.

Approach: a lightweight Vite-served harness (NOT @playwright/experimental-ct-react)
that renders one presentational leaf component in isolation, with no Python
backend. Chosen because it adds zero new deps (root bun.lock untouched -> no
Docker frozen-lockfile risk), reuses the existing @playwright/test + bundled
chromium, and renders through the project's real Vite 8 + Tailwind v4 + token
pipeline so snapshots reflect the actual build output. CT's experimental React
runner on Vite 8 + React 19 was an avoidable compatibility risk.

- harness.html / harness.jsx: isolated render target driven by ?component=&theme=
  URL params; applies themes via [data-theme] (default = bare :root Gruvbox),
  loads the same fonts + token layers as the app, signals font-ready for stable
  shots.
- specs.jsx: registry of pure variant spreads for Badge, Button, Panel,
  SettingRow, SettingsToggle.
- manifest.ts: COMPONENTS x THEMES (default, midnight, catppuccin) the spec
  iterates -> 15 committed baselines in __screenshots__/.
- playwright.visual.config.ts: dedicated config (separate from e2e), own Vite
  server on port 3902, animations disabled, caret hidden.
- scripts: test:visual / test:visual:update.
- README: how to add a component, how to update baselines after an intentional
  change, and why this stays local/manual (font/anti-alias differences across
  OSes) rather than a blocking CI gate for now.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:09:23 +05:30
2b76518c22 fix(css): make @theme the single source for design tokens (dedup drift) + parity test (#777)
The Tailwind v4 `@theme` block in src/index.css and the unlayered `:root`
in src/ui/tokens.css both declared the same `--color-*`, `--radius-*`, and
`--font-*` tokens. Because `@theme` lands in `@layer theme` (low priority)
while tokens.css's `:root` is unlayered, the tokens.css copy silently won —
the `@theme` literals were dead, losing duplicates. The two copies had
already drifted: the font stacks in `@theme` were the short variants while
tokens.css carried the full stacks (with 'Söhne', 'Cascadia Code', etc.),
so the resolved font-family came from tokens.css.

Make `@theme` the single home for the overlapping color/radius/font tokens
and delete the duplicates from tokens.css. To keep every resolved value
byte-identical (this is a pure de-dup, not a restyle), `@theme` adopts the
full font stacks that were actually winning at runtime. Tokens unique to
tokens.css (--color-muted-mono, --radius-pill, --font-display, --font-ui,
spacing, shadows, motion, z-index, etc.) are left untouched.

Theme switching is preserved: themes.css's `[data-theme=...]` overrides are
unlayered, so they still beat the now-@theme-sourced base (unlayered always
wins over @layer theme, regardless of source order).

Verification: a before/after `vite build` shows all 31 effective
`--color/--radius/--font` values identical; full vitest suite (641 tests)
green. Adds src/test/tokenParity.test.js, which fails if any
color/radius/font token is ever re-declared in both @theme and tokens.css
(catching the drift before it can recur).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:09:18 +05:30
7cad4633dc docs(contributing): switch the frontend CSS guidance to utilities-first (#775)
The "Vanilla CSS … no Tailwind" rule contradicted the (already-wired) Tailwind
v4 setup and the CSS→Tailwind migration plan (#772). Replace it with the
utilities-first standard: Tailwind utilities (bridged to the design tokens via
index.css @theme) for layout/spacing/typography; keep .css files only for the
hard parts (glass, keyframes, pseudo-elements, :has(), theme rules).

Required by the docs-sync rule as P0 of the migration.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:02:04 +05:30
5c9b7ca313 chore(format): adopt oxfmt for JS/TS/JSX + CI format gate (#774)
Adds oxfmt (Rust formatter, Prettier-conformant) — the repo had no formatter, so
this is a one-time normalization of the JS/TS/JSX code (257 files; purely
cosmetic — full suite stays 638/638).

Scope is deliberately narrowed in .oxfmtrc.json to JS/TS/JSX only:
- singleQuote:true + jsxSingleQuote:false — preserve the project's existing
  style (single-quoted JS, double-quoted JSX attrs), not oxfmt's double-quote
  default. (Flipping quotes globally also broke a source-string-parsing test;
  preserving them keeps featureCoverage green.)
- Excludes **/*.css (the CSS→Tailwind migration will rewrite those — formatting
  them now is wasted churn), **/*.json (avoids reformatting 20 i18n locale
  files + config), **/*.toml, and src-tauri/** (Rust/Tauri config — out of scope
  for a frontend JS formatter; oxfmt was reformatting Cargo.toml/tauri.conf.json).

Tooling:
- `bun run format` (write) / `bun run format:check` (verify).
- ci.yml: new "Frontend format check (oxfmt)" gate after the oxlint gate.

Verified: format:check clean; oxlint 0 errors; vite build; full suite 638/638;
bun install --frozen-lockfile in sync.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 15:18:01 +05:30
f293f10e7e chore(deps): add taze for manual dependency freshness checks (#773)
Adds taze (root devDep) + `bun run deps:check` = `taze -r --maturity-period 7`:
recurses the bun workspace (root + frontend), lists available updates, and is
READ-ONLY (never writes package.json without -w). The 7-day maturity window
skips just-published versions as a supply-chain precaution.

Manual tool by design — no auto-update, no Renovate infra, nothing added to CI.
Run `bun run deps:check` when you want a refresh overview; `taze major` for major
bumps; add `-w` to apply.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 14:34:21 +05:30
721cb34a9d docs: add the CSS → Tailwind v4 migration plan (#772)
Phased, bounded migration plan (not a big-bang): convert the mechanical ~80%
(flex/grid/gap/padding/typography/simple color) to Tailwind v4 utilities,
deliberately keep ~15-25% as CSS (glass/backdrop-filter, @keyframes,
::before/::after, :has(), !important). Realistic end state ~10-12k of 16.6k CSS
lines removed across ~5-7 weeks of small PRs.

Key gates the plan establishes before any conversion starts (P0):
- A Playwright screenshot baseline (default + dark + light) — the className-diff
  trick used for the page refactors is useless here since class names change.
- Fix the @theme ↔ tokens.css token drift (single source + a parity test).
- Rewrite the CONTRIBUTING.md "no Tailwind" line (docs-sync rule).

Companion to docs/maintenance-pages-modularization.md.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 14:21:01 +05:30
7b4033bc35 chore: adopt knip + remove dead files, deps, exports, and types (#771)
Add knip (dead-code/dep finder) for the bun workspace, then act on what it
found. Complements the oxlint gate: oxlint flags per-file unused symbols; knip
finds whole dead files/exports/deps across the project.

Tooling:
- frontend/knip.json + `bun run knip` script. Ignores the legitimate false
  positives: public/aec-worklet.js (loaded via a dynamic AudioWorklet URL),
  /@react-refresh (Vite dev inject), and tailwindcss + the Rust-side
  @tauri-apps/plugin-updater / plugin-window-state JS packages (used by the
  native plugin, not imported in JS).

Removed (all verified — build + tests + tsc + oxlint green):
- Dead files: CastingView.{jsx,css}, UpdateStatusChip.{jsx,css} (no refs; the
  latter only survived in a stale comment, now reworded), and ui/motion.js.
- Unused deps: @radix-ui/react-popover, @radix-ui/react-select, @eslint/js,
  eslint-plugin-react-refresh (the last two orphaned when eslint.config.js was
  stripped for the oxlint adoption).
- 44 dead exports + 56 dead exported types across api/*, store/*, ui/*, utils/*:
  deleted where used nowhere; dropped just the `export` keyword where still
  referenced in-file.

Kept (justified): Slider primitive (keeps @radix-ui/react-slider meaningful),
AppMode export (a test string-parses its source), tailwindcss (CSS @import +
vite plugin).

Verified: oxlint 0 errors; tsc clean; vite build; full suite 638/638;
bun install --frozen-lockfile in sync (Docker rule).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 14:20:55 +05:30
68d456bb58 refactor(tauri): use tauri-plugin-positioner for the dictation pill (replaces hand-rolled monitor math) (#770)
The floating dictation pill (the "widget" window) was positioned bottom-center
by three near-duplicate blocks in lib.rs that each read primary_monitor(),
divided size by scale_factor, and called set_position(LogicalPosition...), with
a win.center() fallback. Replace all three with the official
tauri-plugin-positioner: window.move_window(Position::BottomCenter), preserving
the center() fallback on error.

- Add tauri-plugin-positioner = { version = "2", features = ["tray-icon"] }
  (tray-icon enabled because the app ships a system tray).
- Register .plugin(tauri_plugin_positioner::init()) after single-instance.
- Collapse the global-shortcut, tray "dictate", and pill-mode pre-position
  blocks to the plugin API. Behavior-preserving: same window, same trigger
  points, still bottom-center.

Verified with cargo check (passes; the one warning is pre-existing in setup.rs).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 13:59:34 +05:30
87f884d7a1 refactor(tauri): remove dead pill-autostart code (#764)
The enable/disable/is_pill_autostart commands (and pill_autostart_path) were
defined in commands.rs and registered in lib.rs but NEVER invoked — no JS
caller, no internal Rust call, and no Settings toggle. ~155 lines of unwired,
hand-rolled cross-platform code (macOS plist / Windows registry / Linux
.desktop) maintained for a feature that was never shipped.

Investigated adopting tauri-plugin-autostart instead, but since nothing exposes
the feature, replacing dead code with a plugin (+ a new toggle) would be
building an unrequested feature. Removing the scaffolding is the honest cleanup;
if the "launch dictation pill at login" feature is ever wanted, wire it then via
tauri-plugin-autostart (init(LaunchAgent, Some(vec!["--pill"]))).

Kept: dirs-next (still used by config.rs/setup.rs — comment updated) and the
launch_as_widget config commands (those ARE used). cargo check passes.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 05:34:50 +05:30
294a5db4c1 fix(api): route backend fetches through apiFetch so they carry LAN-share auth (#765)
Under LAN-share / remote-backend (a PIN/API key is set), ~28 raw fetch() calls
to the backend 401'd because they skipped the X-OmniVoice-Pin / Authorization
headers that apiFetch injects. Route them through apiFetch — fixing the auth
gap and adding the same transport-retry robustness (backend-restart windows
become invisible) the rest of the app already has.

Since apiFetch throws ApiError on !ok (and fires ov:pin-required on 401), the
now-dead `if (!res.ok) {…}` blocks were removed; surrounding try/catch handles
the ApiError. Streaming (.body.getReader), FormData, cache, and signal opts are
all preserved (apiFetch passes opts through; apiUrl is idempotent for absolute
URLs).

Deliberately left as raw fetch (documented): the auth-exempt /health liveness
probe (custom timeout/backoff), the RemoteBackendPanel pre-save connectivity
test (uses a user-typed target+key), WaveformTimeline (branches on 404 + may be
a blob: URL), VoiceGallery playUrl (also serves external community-CDN URLs),
and bugReport's fetchJsonWithTimeout (hard 2.5s bound, no retry by design).

Updated the #532 in-app-playback regression test to assert via apiFetch.

Verified: oxlint 0 errors; vite build passes; full suite 638/638.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 05:34:44 +05:30
8765f766e1 chore(lint): adopt oxlint as the linter + CI gate; fix the bugs it surfaced (#761)
ESLint was misconfigured (only globals.browser → 47 false no-undef) and run
NOWHERE in CI, so 259 errors had accumulated unnoticed. Replace it with oxlint
(Rust, ~50-100x faster) as the primary linter AND a real CI gate so lint debt
can't silently pile up again.

Tooling:
- frontend/.oxlintrc.json — correctness=error; no-unused-vars with the existing
  ^[A-Z_] convention; node/vitest env overrides + AudioWorklet/__APP_VERSION__
  globals (kills the false no-undef class); max-lines:500 (warn).
- package.json: `lint` → oxlint, `lint:fix`, `lint:hooks` (advisory eslint).
- eslint.config.js stripped to ONLY the React-Compiler rule family oxlint can't
  do yet (set-state-in-effect etc.), run via `lint:hooks`, NOT gated. Drop once
  oxlint's JS-plugin support leaves alpha.
- ci.yml: new "Frontend lint (oxlint)" step in the Tests job — the gate.

Real bugs oxlint caught (were buried in ESLint's noise):
- GlossaryPanel: <X/> close-icon used but never imported → the edit-row cancel
  button threw ReferenceError at render. Imported X.
- Two use*-named NON-hooks (useEngine action, useArchetypeAsProfile API call)
  tripped rules-of-hooks; suppressed with documented disables (renaming these
  misleading names is a worthwhile follow-up).

Cleanup to reach a 0-error gate: removed 52 genuinely-dead vars/imports across
18 files (heavy in App.jsx — stale useState left over from prior refactors) and
4 behavior-preserving autofixes (no-useless-fallback-in-spread / no-useless-escape).

Verified: oxlint 0 errors; bun install --frozen-lockfile in sync (Docker rule);
vite build passes; full suite 638/638.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 05:08:52 +05:30
b9707e0d8f refactor(pages): modularize Clone/Gallery/Profile pages (all files <500) (#760)
Phase 3 — same standard as #758/#759, applied to the last three over-cap pages.
Pure-mechanical, no behavior change.

- VoiceGallery.jsx 768 → 205: relocate the already-separate zone components
  (ArchetypesZone, ArchetypeCard, CommunityZone, ImportsZone) + shared helpers
  into components/gallery/.
- CloneDesignTab.jsx 837 → 395: split the ~540-line JSX return into section
  components (ScriptPanel, AudioMethodPanel, DesignMethodPanel, ActionBar) +
  MicButton, under components/clone/. State stays in the page.
- VoiceProfile.jsx 515 → 287: split the main return into ProfileHeader /
  ProfileDetails / ProfileActivity under components/profile/.

Safety contract for the JSX splits (no render tests): explicit NAMED props on
every section so eslint no-undef verifies completeness on both ends; JSX moved
verbatim. Verified: 0 no-undef across all changed files; every original
className preserved (diffed main vs new set); every file <500 lines.

Verified: vite build passes; FULL frontend suite 638/638 pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 03:36:31 +05:30
a7f813b7a9 refactor(dub): modularize DubTab page (1593→380 lines, all files under 500) (#759)
* refactor(dub): extract DubTab sibling sub-components into components/dub (1593→1361)

Phase 2 (partial). Move the 5 self-contained presentational sub-components out
of the oversized DubTab.jsx into a new components/dub/ folder, matching the
components/settings/ pattern. Pure-mechanical, logic byte-for-byte identical.

Extracted (each with its own private helpers/constants):
- DubFailureNotice, DubPipelineStepper (+DUB_PIPELINE/DUB_PHASE_BY_STEP),
  PrepOverlay (+PREP_FULL/PREP_CACHED/fmtBytesRate/fmtEta), TranscribeOverlay,
  FooterBtn. fmtDur stays — it's used by the main component.

Pruned imports orphaned by the moves (copyText, errorDocsMap, a few icons).

Verified: vite build passes; dub tests (dubExpiredJobError + DubbingDemo)
11/11 pass; no new lint errors.

NOTE: DubTab.jsx is still 1361 lines — the main component is one ~1000-line
stateful JSX return over 28 hooks. Getting it under the 500 cap needs that JSX
split into section components, a higher-risk change deferred for a careful,
test-backed pass (see docs/maintenance-pages-modularization.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(dub): split DubTab JSX into section components (1361→380, all files <500)

Completes Phase 2. The DubTab component was one ~1000-line stateful JSX return.
Split that markup into five section components under components/dub/, keeping
ALL state/hooks/handlers/effects inside DubTab — only the JSX moved (verbatim,
by line-slicing).

Safety contract (this is behavior-critical and has no render test):
- Explicit NAMED props on every section (no bag/context object), so eslint
  no-undef verifies prop completeness on BOTH ends — a dropped value becomes a
  build error, not a silent runtime undefined. Verified: 0 no-undef across all files.
- All 137 classNames from the original are preserved (diffed main vs new set).

New sections: IdleSkeleton (368), DubLeftColumn (336), DubRightColumn (172),
DubFooter (78), DubHeader (63). DubTab.jsx is now a thin composition (380).

Verified: vite build passes; FULL frontend suite 638/638 pass; every settings &
dub file now under the 500-line cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 03:36:27 +05:30
f33bdc731d refactor(settings): modularize Settings page (1969→399 lines, all files under 500) (#758)
* refactor(settings): extract Settings.jsx tabs into components/settings (1969→602 lines)

Settings.jsx had grown to 1969 lines — every edit reloaded the whole file
into context and risked unrelated breakage. This finishes the migration the
existing components/settings/*Panel.jsx pattern started: the page is now a
thin orchestrator and each heavy tab lives in its own file.

Extracted (logic byte-for-byte identical; only import paths adjusted + the
shared isTauri/askConfirm moved to components/settings/native.js):
- GeneralTab, ModelStoreTab, EnginesTab, HotkeyTab, CredentialsTab
- native.js — shared isTauri() wrapper + askConfirm() Tauri-dialog helper

Also establishes the standard so files can't silently regrow:
- CONTRIBUTING.md: frontend file-structure & size limits (soft 300 / hard 500)
- eslint.config.js: warn-only max-lines:500 guardrail (CI stays green)
- docs/maintenance-pages-modularization.md: the phased refactor plan

Verified: vite build passes (all imports resolve); 18/18 settings tests pass;
no new lint errors introduced (the pruned imports were the only regressions).

Follow-ups (tracked in the plan doc): ModelStoreTab.jsx is 836 lines and
Settings.jsx 602 — both still over the 500 cap (warn-only); split next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(settings): split ModelStoreTab + Settings.jsx under the 500-line cap

Follow-up to the tab extraction: bring the two remaining over-cap files into
compliance with the new standard. Pure-mechanical, no behavior change.

Settings.jsx 602 → 399:
- Extract AboutTab, PrivacyTab, LogsTab into components/settings/
- Move the shared Row helper to components/settings/Row.jsx
- LogsTab keeps its state in Settings() (lower-risk); About/Privacy take props

ModelStoreTab.jsx 836 → 439, split into components/settings/models/:
- format.js (fmtBytes/orgColor), runtime.js (computeRowRuntime)
- columns.jsx exposes makeModelColumns(...) — a factory so the TanStack cell
  closures keep working; called with the same useMemo dep array as before
- ModelsTable.jsx (virtualized table view), RecoBanner.jsx

Every settings file is now under 500 lines. Verified: vite build passes;
18/18 settings tests pass; no new lint errors (the 4 remaining in Settings.jsx
are pre-existing — refreshInfo no-op, a catch(e), two set-state-in-effect).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 03:34:51 +05:30
e2f02327c3 fix(settings): contain + tighten the whole Settings surface (design-system pass) (#750)
* fix(settings): contain + tighten the whole Settings surface (measure cap, container-query stacking, wrap the shared rows)

Two systemic issues drove 'too spread out' + 'elements go out of view' across
many Settings pages:

1. Spread — .settings-content capped at 1280px, so on wide windows every
   label-left/control-right row left a huge void. Introduce a --settings-measure
   token (720px, macOS-like) + --settings-rail, and cap the content to it,
   left-aligned under the nav. One token now controls the reading width.

2. Overflow + bad responsiveness — the row stack break was a *viewport* media
   query (560px), but the 168px nav rail means a 760px-viewport window only has
   ~530px of content, so rows went side-by-side in a cramped box. Make
   .settings-content a container (container-type: inline-size) and stack on the
   CONTENT width via @container, keeping the viewport @media as a fallback for
   the .st-row instances used outside Settings (Splash/FirstRun/Dub/SetupWizard).

3. The shared .perfpanel__row (button/badge row reused by 6+ panels:
   RemoteBackend, HFMirror, LLMEndpoint, Pronunciation, MCPBindings, …) was an
   inline-flex with no wrap and no max-width, so it ran off the right edge —
   add flex-wrap + max-width:100% + min-width:0. Plus two rigid-width fixes that
   escaped the row cap: ApiKeys input min-width:220→0, Appearance scale floor.

Frontend builds clean; tokens, @container query, and the wrap all verified in the
emitted CSS bundle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(settings): center the settings block + tighten measure (kill the lopsided right void)

The capped content was left-aligned, so on a wide window everything jammed to the
left with a dead empty third on the right (screenshot). Center the whole settings
block (nav rail + content) as a unit via max-width + margin-inline:auto, and drop
the measure 720→660 so label→control rows read denser. The cap is computed from
the tokens (rail + gap + measure + page padding) so the content track lands
exactly at --settings-measure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:49:33 +05:30
39385feb78 docs(changelog): finalize the [0.3.8] release notes (date, ASR-hang scope, dev-launch fix) (#747)
Set the release date to 2026-06-29, extend the #730 entry to note the chunked
dub-stream path is bounded + pool-reset too (#742), and add the bun desktop
dev-launch fix (#745) under CI. release.yml extracts this section verbatim as
the GitHub Release body, so it's now tag-ready.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 13:29:33 +05:30
fa66c6a025 fix(dev): stop the Tauri dev app from killing concurrently's backend (bun desktop crash) (#745)
`bun desktop` runs concurrently[dev:api, dev:desktop] with --kill-others-on-fail.
dev:api is a uvicorn backend on :3900, but the Tauri app launched by dev:desktop
ALSO manages a backend — on boot it sees :3900 in use (and not yet healthy,
because the dev backend is still importing torch + loading 32 models) and
'takes ownership', killing the dev:api process. That exits 137, which trips
--kill-others-on-fail and tears the whole session down.

The Tauri app already supports TAURI_SKIP_BACKEND to skip backend management
(lib.rs:654) — it just wasn't wired for the concurrently-managed dev flow. Set
it on dev:desktop so the dev app attaches to concurrently's backend instead of
fighting it. Set only on dev:desktop (not dev:api, and not the standalone
`frontend` desktop script, which legitimately self-manages the backend).

bun's script shell evaluates the inline VAR=val cross-platform (verified), so no
cross-env dep / lockfile churn. Prod (desktop-prod) is unaffected — there the
Tauri app is the sole backend manager and orphan-kill is correct.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:33:09 +05:30
4a01ecdfae fix(tts): force re-download a corrupt-but-right-size model blob before giving up (#739) (#744)
snapshot_download's resume trusts an existing file by size, so a present-but-
corrupt blob is never re-fetched: the resume-repair 'succeeds' yet the reload
still raises the truncated-cache OSError, and the user was sent to a manual
delete-and-reinstall. Add a force=True path (force_download) and wire it as a
last resort — on the post-resume reload failure, force a full re-download once
(replacing corrupt blobs) and retry the load before falling back to the
actionable message. Force is reached only after a plain resume-repair didn't
fix it, so the common missing-file case still avoids re-downloading everything.

Tests: corrupt cache force-repairs on the 2nd failure (resume then force),
force_download is set only when force=True, and an unfixable cache still
surfaces the 'could not be auto-repaired' message.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 11:30:54 +05:30
cc95f526e0 fix(asr): reset the GPU pool when a chunked dub-stream chunk wedges too (#730) (#742)
The whole-file transcribe paths recover from a wedged worker via
run_transcribe_guarded's pool reset (#731), but the chunked dub transcribe-stream
only recorded a per-chunk timeout error and moved on — leaving the stuck thread
holding its GPU-pool worker, so subsequent chunks / a concurrent TTS generate
could still starve into 'can't reach backend'. Reset the pool on the per-chunk
TimeoutError via a small _reset_pool_on_wedge() helper (best-effort, no-op for a
plain executor). Closes the residual on #730.

Tests: helper resets a reset-capable pool and no-ops a plain ThreadPoolExecutor.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:36:45 +05:30
9fc18dbd89 fix(tts): retry the incomplete-cache auto-repair so a transient blip doesn't dead-end (#739) (#741)
_repair_model_cache attempted snapshot_download exactly once; a single transient
failure (the very cause of an interrupted download) returned False and sent the
user back to a manual delete-and-reinstall. Wrap the re-fetch in a bounded retry
loop (3 attempts default, linear backoff) — snapshot_download resumes between
attempts so retries are cheap and idempotent. Counts/backoff are env-tunable
(OMNIVOICE_MODEL_REPAIR_RETRIES / _BACKOFF_S) for restricted networks and set to
zero-backoff in tests. Offline mode + the actionable fallback message are
unchanged.

Tests: retry-then-succeed self-heals, exhausted-retries returns False after N
attempts, single-attempt tunable, backoff disabled so the suite stays fast.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:12:43 +05:30
Deepak VishwakarmaandClaude Opus 4.8 de3d83f14b docs: add rust as prerequisite for from-source builds (#704)
Adds Rust/Cargo as a from-source build prerequisite across the linux/macos/windows install docs. Thanks @Deepakv2104.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 03:03:23 +05:30
0fc9f2afec fix(asr): bound every transcribe path + reset the GPU pool on hang so a wedged ASR can't brick the backend (#730) (#731)
A whisperx/CTranslate2 transcribe can hang hard on some Windows+CUDA setups and
never return. ASR shares the small (1-2 worker) _gpu_pool with TTS, so one stuck
worker starved every other request — the next TTS generate then surfaced as
"Can't reach the local backend" though the process was alive (#720/#721/#723).

Two parts:
- Bound the three remaining unguarded whole-file transcribe paths (dub
  whole-file dub_core.py, batch.py, live-dictation capture_ws.py) with
  run_transcribe_guarded, matching the dub-QC/dictation/OpenAI paths that were
  already bounded by #656.
- On timeout, run_transcribe_guarded now calls executor.reset() when the pool
  supports it (_ResilientGpuPool, already built for the model-load-timeout case
  in #589/#599): the wedged worker is abandoned and the next submit gets a fresh
  one, restoring capacity without an app restart. Best-effort — a plain
  ThreadPoolExecutor (tests) just gets the bound + actionable error.

Regression tests: pool.reset() is invoked on timeout; a non-reset pool still
bounds cleanly.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 03:00:16 +05:30
a9948de99e fix(generate): classify [Errno 32] Broken pipe as a lost-pipe error, not OOM (#715) (#722)
A BrokenPipeError surfacing from generation means the backend's stdout/stderr
pipe to the desktop shell that launched it closed mid-render (an orphaned or
relaunched backend) — not out of memory. _oom_friendly_reraise mislabeled it
"ran out of memory — try Flush," which never helps. Add a BrokenPipeError /
[Errno 32] branch (same pattern as the #705 WinError-193 and #437 permission
branches) that tells the user to restart the app instead. main.py already wraps
sys.stdout/stderr to swallow EPIPE; this catches the C-level writes inside the
native engine/torch that escape that guard.

Regression test covers both the typed BrokenPipeError and a string-wrapped
"[Errno 32] Broken pipe".

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:12:50 +05:30
7f9a97b0fa fix(settings): harden control inputs against right-edge overflow (belt-and-suspenders) (#718)
Follow-up to the responsive-containment fix (#713). Make Settings control inputs
unable to overflow the available width regardless of inline widths a panel sets:

- RemoteBackend's Backend URL + API key inputs hard-coded style={flex:1,
  minWidth:220} in a right-aligned 60%-max control — on a narrow control that
  220px floor overflows. They're long-value fields, so lay them out as full-width
  stacked rows (st-row--stack) with the shrinkable .st-input class instead.
- Add a universal guard: any text-ish input/select/textarea inside .st-row__control
  gets min-width:0 / max-width:100% / box-sizing, so no panel's raw input can
  spill past the row. Pairs with the page/row minmax(0,1fr) grids.

Pure presentation; 638 frontend tests pass; build clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 17:54:30 +05:30
935b38a962 fix(dub): pass OmniVoice's ffmpeg to yt-dlp so URL merge works off PATH (#712) (#716)
Dubbing a video URL on Windows (v0.3.8) failed with 'You have requested merging
of multiple formats but ffmpeg is not installed.' The download format selector
pulls separate video+audio streams, so yt-dlp muxes them via ffmpeg
(merge_output_format=mp4) — but yt-dlp only checks PATH, while OmniVoice's ffmpeg
is typically a bundled Tauri sidecar / imageio-ffmpeg binary that isn't on PATH.

yt_download_sync now sets ydl_opts['ffmpeg_location'] = find_ffmpeg() (the same
resolver the rest of the dub pipeline uses) when ffmpeg is resolvable; if it
isn't, the key is omitted so yt-dlp falls back to PATH as before (no regression).
Tests assert the location is passed when resolved and omitted when not.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:16:45 +05:30
ea4d5e7839 fix(generate): self-heal schema + don't 500 a generated clip on a history-write fail (#710) (#714)
A synth that already produced and saved its audio could still return a 500:
'no such table: generation_history' — a DB that somehow missed schema init
(init_db's executescript never took) made the history INSERT raise after the
clip was done, losing the user's generation to a logging side-effect.

- Add db.ensure_schema(): idempotent CREATE ... IF NOT EXISTS + additive column
  reconcile (no _migrate/alembic), safe to call from a write path.
- Generation history write now self-heals: on a sqlite OperationalError it runs
  ensure_schema() and retries once; if it still fails it logs and returns the
  audio anyway. A history-logging failure can never fail the generation.

Regression test: the write raises 'no such table: generation_history' before the
heal and succeeds after (fail-before/pass-after), plus ensure_schema idempotency.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 15:04:31 +05:30
d566e5bc8e fix(settings): contain page content within available width (no right-edge clip) (#713)
Right-side control values/pills (e.g. Privacy's LOCAL SQLITE / OFFLINE
TRANSLATION / NONE — NO TRACKING, and long stored-at paths) clipped off the
right edge on wide windows.

Root cause: both settings grids used a bare '1fr' track (= minmax(auto,1fr)),
whose 'auto' minimum is the content's min-size. A non-shrinking child — a nowrap
status pill or an unbreakable path — forces the track wider than the viewport,
and .settings-content's max-width can't claw that back, so it clips at the
window edge.

Fix: minmax(0, 1fr) on both grids so the tracks can shrink below content
min-size:
- .settings-page  → 168px minmax(0, 1fr)  (the content column)
- .st-row         → minmax(0, 1fr) auto    (a long title can't shove the control
                                            off-screen; the label shrinks/wraps)

Shared layout primitives, so this contains EVERY settings page responsively.
Pure presentation; 638 frontend tests pass; build clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:45:57 +05:30
2c2e493df8 fix(dub): stream segments to disk to stop long-video RAM spikes (#639) (#709)
Takes over and completes #639 (original work by @trungthanh1288). Dub generation
held every segment's audio in RAM until final mix, so long/feature-length dubs
and big batches could exhaust memory. Segments now stream to disk as rendered;
the final track assembles from those files via a 30s-chunk memmap writer, so
peak memory stays flat regardless of length.

Completed on top of the original PR:
- Watermarking: keep the project's 'every OmniVoice audio carries the signature'
  guarantee without double-marking. Since seg_<id>.wav is BOTH the downloadable
  file AND the assembly input, mark each fresh segment once at synthesis and drop
  the per-chunk embed in the memmap writer (the final mix inherits the mark) —
  main's proven policy. Verified with real AudioSeal: 0.9999 detect confidence on
  the final track and on seg WAVs; cached/silence not re-marked.
- Fix a crash regression: zero/negative-duration segments returned an in-memory
  zero-length entry instead of writing empty audio (which raised). Regression test
  added.
- Perf: drop per-segment gc.collect(); throttle empty_cache() to every 16th call
  (the replaced code batched I/O to keep this off the hot path).
- Clean up the mix_<id> temp WAVs after assembly.
- Rewrite the watermark test for the multi-chunk (>30s) path; assert both the
  final track and the seg WAV are marked, with no double-mark.

212 passed / 1 skipped; route inventory clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: trungthanh1288 <trungthanh1288@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 14:29:38 +05:30
2fc90c44e6 docs(changelog): refresh the [0.3.8] headline for the release body (#708)
The headline predated the later 0.3.8 work. Bring it current — Settings
redesign, macOS native drag-drop (incl. macOS 26), the ASR CTranslate2-load
fallback, the pronunciation dictionary, and the more-honest error messages
(corrupt binary != OOM, model-id self-heal, stale-dub reset). release.yml
publishes this section verbatim as the GitHub Release body, so the headline is
the first thing users read on the v0.3.8 release.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 04:30:52 +05:30
aa17e3319f fix(model): route every OMNIVOICE_MODEL read through the resolver; tighten WinError 193 match (#693, #705) (#707)
Follow-up from independent verification of #693/#705.

#693 (whole-class): the resolver only guarded the model-load site. A leaked
engine id in OMNIVOICE_MODEL still hit four other raw reads — most importantly
preload_model()'s model_info() probe, which failed on the bad value and
SILENTLY disabled warm-up (first /generate then ate the full load). Plus the
Settings 'model_checkpoint' display, the loaded-models list, and the engine_id
baked into exported persona bundles. Route all of them through
resolve_omnivoice_checkpoint() (personas keeps its '' unset marker, sanitizing
only a set value). Add a source-level recurrence guard so a future raw read
can't reintroduce the class.

#705: tighten 'winerror 193' -> '[winerror 193]' so the substring can't also
match WinError 1930-1939 (the portable 'is not a valid win32 application'
clause still covers non-Windows formatting).

48 tests pass (resolver + guard + audio-guard + route inventory); edited
routers/services import clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 03:37:46 +05:30
883a06e9c0 fix(generate): classify WinError 193 as a corrupt native component, not OOM (#705) (#706)
A synth failure from a corrupt or wrong-architecture native binary on Windows
([WinError 193] %1 is not a valid Win32 application — torch, ffmpeg, or a
bundled engine binary) fell through to the generic OOM message ('ran out of
memory — try Flush'), sending the user down a path that can't help.

_oom_friendly_reraise() now detects the WinError 193 / 'is not a valid Win32
application' signature (before the OOM fallback, joining the existing
torch.compile / decode-glitch / bad-instruct cases) and surfaces an actionable
'reinstall or repair that component; Flush won't help' message. Regression test
added.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 02:13:21 +05:30
85fd8b6494 fix(desktop): enable native HTML5 file drag-drop on macOS (#700) (#703)
The app's drop zones (clone reference, dub video, stories, batch) all use HTML5
dataTransfer.files, but tauri.conf.json never set dragDropEnabled, so it defaulted
to true — Tauri intercepts the OS file-drop and the webview's HTML5 drop never
receives the files. Most visible on macOS WKWebView and fully broken on macOS 26
(Tahoe). Set dragDropEnabled: false on the main window so the webview handles
native HTML5 drops uniformly across platforms.

(The Clone/Design textarea-resize half of #700 was already fixed for 0.3.8 by
#595/#607; the reporter is on v0.3.7.)

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 01:45:25 +05:30
598b1bd911 docs(changelog): complete the [0.3.8] section for this release batch (#701)
Add the user-facing entries that landed after the initial [0.3.8] draft:
- Changed: the full Settings redesign (#686/#690/#696) and the inline first-run
  HF-token input (#687/#688).
- Fixed: OMNIVOICE_MODEL self-heal (#693), ASR CTranslate2 .so-load fallback
  (#692), and the stale-dub recovery extended to initial upload/ingest (#695).
Bump the section date to the expected cut date (set authoritatively at tag time).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 01:33:45 +05:30
ea3b565f9c fix(asr): fall back instead of crashing when CTranslate2's .so won't load (#692) (#699)
On hardened kernels / newer glibc (e.g. WSL2 glibc 2.43) CTranslate2's shared
object is rejected at load with 'libctranslate2…cannot enable executable stack'
— an OSError, not ImportError. The WhisperX/faster-whisper is_available() probes
only caught ImportError, so the OSError escaped and crashed the ASR/dub
preflight ('ASR backend initialization failed: …').

- Both probes now also catch the non-ImportError load failure and REPORT
  (False, 'failed to load …') instead of raising — a probe must never raise.
- _auto_detect() routes every probe through a never-raising _probe_available()
  so no exploding probe can crash engine selection; it falls through to
  pytorch-whisper (transformers, no CTranslate2), which works on CUDA/CPU.

Regression tests cover the raising probe, the .so-load OSError surfacing as
unavailable, and auto-detect falling back to pytorch-whisper.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:04:54 +05:30
97a020ecf0 fix(model): self-heal a leaked engine id in OMNIVOICE_MODEL instead of 500 (#693) (#698)
A stale/misconfigured OMNIVOICE_MODEL holding a bare TTS *engine id* (e.g.
"omnivoice") was passed straight to OmniVoice.from_pretrained(), which 500s
with "omnivoice is not a local folder and is not a valid model identifier
listed on huggingface.co/models".

Add resolve_omnivoice_checkpoint(): honor only a HF repo id (org/repo) or an
explicit local path (absolute / contains a separator); any bare token self-heals
to k2-fsa/OmniVoice with a logged warning — so a bad value can't brick model
load (and can't be faked by a cwd-relative folder of the same name). Regression
tests cover the leak, valid repo ids, absolute local dirs, and blanks.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:04:47 +05:30
5e6b23aeae fix(dub): reset gracefully on a stale job during initial upload/ingest (#695) (#697)
The #660 fix wired the stale-job recovery (isExpiredDubJobError → reset) into the
retry and SRT-import handlers, but NOT the two INITIAL handlers (handleDubUpload,
handleDubIngestUrl). So a job that went missing during the first upload→prep→
transcribe flow (backend reload, cache eviction, manual cleanup) surfaced the
scary "Job not found … report a bug" toast instead of quietly resetting the
stale session — exactly the reported error.

Route stale-job errors through isExpiredDubJobError() in both initial handlers
too (before the reportable fallback), matching retry/import. Add a source-level
regression guard so no dub handler can silently drop the stale-job check again
(the #660→#695 regression class).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:04:39 +05:30
5a650eeee7 fix(settings): full-width content + fix right-side wrapping/overflow/padding (#696)
Owner review of the live pages: the 760px content cap left a dead empty right
half on simple tabs (Appearance), while wide tabs (Models) showed mid-word path
breaks, an overflowing HF_ENDPOINT input, and controls flush to the border.

- Content fills full width (removed the 760px cap + redundant models opt-out);
  1280px ceiling only on ultra-wide. Comfortable side padding both sides.
- Read-only mono path values wrap only at boundaries (no `…cach/e…` mid-word).
- Inputs capped (min(360px,100%)) + box-sizing so HF_ENDPOINT/cache never overflow.
- Right padding on .st-row__control so controls aren't flush to the edge.
- Input-heavy rows (mirror preset, HF_ENDPOINT, cache location) go full-width
  below their label instead of a crushed right slot.

Pure presentation; 636 tests pass; build clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 00:04:32 +05:30
53d00ab76a refactor(settings): premium redesign — compact density, nav rail, unified controls (#690)
A design-council-driven overhaul of the Settings UI for a clean, professional,
compact-yet-gorgeous feel (Notion/Obsidian quality), addressing "looks amateur,
too tall, doesn't make sense":

- Typography: section titles move from mono-uppercase ("debug log" look) to
  sans sentence-case 600; mono reserved strictly for data values. Three clear
  type levels.
- Density: single-line ~32-40px rows (grid 1fr auto), hairline dividers instead
  of card-per-row, one muted description max per row (SettingRow hardened so the
  old double-description line is structurally impossible).
- Navigation: kill the rainbow per-tab accents → one --chrome-accent; ≥760px a
  sticky vertical nav rail + a calm 760px content column (no stretch to the rail
  height — the empty-void fix); <760px a no-wrap horizontal scroll strip.
- Controls: full-width horizontal grids for the font + theme pickers (were a
  squeezed vertical stack); unified tile/toggle/input styling via a new
  SettingsInput primitive; tokenized off-token literals.
- No tacky wrapping: descriptions wrap at a comfortable measure (text-wrap:
  pretty, no orphans); short control values never break mid-word.

Pure presentation — no behavior, handler, prop, testid, role, or i18n-string
changes. 636 frontend tests pass; build clean; --chrome-* tokens only.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 14:57:45 +05:30
823e222bd0 feat(setup): compact inline HF-token input pinned with the Continue action (#688)
Replace the bulky HF-token card (icon + title + paragraph + input row + link)
with a single-line input bar — paste a token, Save — pinned right by the
'Waiting for required models…' / Continue button. Takes only the HF token; the
explanation collapses to a one-line prompt (hidden on narrow widths) plus a
'Get one free →' link, and a slim '✓ saved' confirmation. Same save path and
i18n keys; cleaner and lighter on the page.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 10:39:41 +05:30
70857bd1ad fix(setup): pin the HF-token card next to Continue, not buried in the model list (#687)
The 'Add a free Hugging Face token for faster downloads' card sat at the bottom
of the scrolling model library, so users had to scroll past every model to find
it. Extract it into a standalone HfTokenCard and pin it in the wizard's
always-visible action area, right above the 'Waiting for required models…' /
Continue button — visible at a glance, click and paste a token without scrolling.
Compact hint so it doesn't crowd the button. No behavior change to saving.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 03:59:02 +05:30
6a64db0b8c refactor(settings): declutter + redesign the Settings UI onto a shared design system (#686)
* refactor(settings): shared design-system primitives + shell restyle (unit 0)

Foundation of the Settings redesign. Adds reusable primitives
(SettingsSection, SettingRow, InfoHint, SettingsToggle, Collapsible) styled
purely with --chrome-* tokens, and restyles the Settings shell: reordered
icon tab-nav, inline tabs (General/Hotkey/Credentials/Logs/Updates/About/
Privacy) migrated to the primitives, Proxy/FFmpeg/advanced rows tucked into
Collapsible, long prose moved into InfoHint popovers. Row() delegates to
SettingRow. No behavior changes; ModelStore table/SSE untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(settings): restyle all panels onto the design system (units A/B/C)

Migrate the 13 settings panels to the shared primitives — pure presentation,
no behavior change:
- Bucket A (Aec/Performance/Refinement/HFMirror/LLMEndpoint/MCP/Pronunciation):
  long prose (torch.compile OOM, refinement examples, etc.) moved into InfoHint
  popovers; custom checkboxes → SettingsToggle.
- Bucket B (ApiKeys/RemoteBackend/Sharing): Tailscale/help prose → InfoHint +
  Collapsible 'Advanced'; ApiKeys/Sharing CSS converted off hardcoded colors to
  --chrome-* tokens (they mis-themed on 5 of 6 themes).
- Bucket C (Appearance/Storage/Voice): VoicePanel switch → SettingsToggle;
  Appearance/Storage CSS tokenized; prose → InfoHint.
- SettingsToggle now forwards arbitrary props (data-testid/aria) to the input.

All 636 frontend tests pass; build clean; no new deps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(settings): query VoicePanel/Appearance switches by role after SettingsToggle migration

The VoicePanel enable switch moved from a testid'd checkbox to the SettingsToggle
primitive (role=switch); update the assertion accordingly. Was missed in the
panel-restyle commit because this test lives under src/test/, not components/settings/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 03:57:37 +05:30
ee638bda6c feat(tts): user pronunciation dictionary (expressive-tts slice 1) (#685)
* feat(tts): user pronunciation dictionary (expressive-tts slice 1)

Per-term, per-language pronunciation overrides applied to text before synthesis,
so names, brands, and acronyms come out right across generate, longform, and dub.
Closes part of the #1 perceived-quality gap vs ElevenLabs (pronunciation
dictionaries). First slice of docs/specs/01-expressive-tts.md.

- Schema: additive `pronunciation_entries` table (alembic 0008, mirrored into
  _BASE_SCHEMA; tested upgrade — idempotent, downgrade, converge, back-compat).
- Service: extend pronunciation.py to load enabled entries (cached) and apply
  longest-first, word-boundary-aware, per-language (global '*' + lang match,
  lang overrides global), reusing the existing ReDoS-safe matcher.
- Inline one-off `[[term|replacement]]` overrides that don't persist and don't
  collide with [voice:]/[pause]/[Name]/SSML-lite (resolved pre-chunking).
- API: /pronunciation CRUD + /test dry-run + import/export (loopback-guarded).
- Apply point: generation.py after language resolves, before chunking — covers
  native + pluggable engines.
- UI: PronunciationPanel in Settings → General; all strings via i18n.
- Tests: migration lifecycle, CRUD, per-language, precedence, inline override,
  apply-at-synth. Route snapshot regenerated (+7).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): bound inline-override regex (ReDoS) + annotate parameterized UPDATE

CodeQL flagged py/polynomial-redos on the [[...]] inline-override regex: [^\]]
also matches [, so an unterminated run of [ allowed O(n) rescans from O(n)
positions. Bound the inner class to {0,256} (linear; an inline override is a
short respelling). Annotate the dynamic UPDATE (B608) — its column fragments are
fixed literals and every value is a bound parameter; not an injection vector.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 03:16:25 +05:30
25105605f1 docs(specs): ElevenLabs-parity roadmap + Tier-1 implementation specs (#684)
Add the implementation-ready spec set mapping OmniVoice to ElevenLabs parity
while preserving local-first:

- 00-roadmap-elevenlabs-parity.md — gap analysis, prioritized tiers, sequencing,
  prior-art reconciliation, and the deliberate "won't build" list.
- 01-expressive-tts.md — engine-agnostic emotion/style intent lowered onto each
  TTS engine's real mechanism (degrade-visibly) + a DB-backed pronunciation dict.
- 02-conversational-agent.md — fully-offline full-duplex voice agent (/ws/converse,
  Silero-VAD barge-in on AEC-cleaned mic) composing existing streaming STT/TTS + LLM.
- 03-longform-studio-editor.md — per-segment edit/regenerate across dub/audiobook/
  stories, extending the existing content-addressed cache to longform.

Reconcile prior planning docs: banner the superseded parity/studio docs pointing
here; keep distinct-scope docs untouched (classification table in 00).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 03:11:00 +05:30
022a3bd6b9 feat(dictation): live local dictation via sherpa-onnx + Voice settings panel (#683)
* feat(dictation): live local dictation via sherpa-onnx + Voice settings panel

Add a sherpa-onnx ASR engine alongside the existing Whisper/NeMo dictation
path, powering a genuinely live experience: as you speak, words type straight
into the focused field (streaming partials via a new simulate_type command,
self-correcting with backspaces) and commit per pause.

Backend:
- SherpaDictationBackend + sherpa_dictation registry of the 7 models (Parakeet
  TDT v3/v2, streaming Zipformer EN/ZH/bilingual, Paraformer bilingual, Whisper
  Tiny) from csukuangfj/* int8 HF repos; CPU provider for cross-platform parity.
- /dictation/models + /dictation/prefs router; get_capture_asr_backend() honors
  the selected dictation model. get_active_asr_backend() (dub transcription) and
  the legacy WebM/Opus capture path are untouched.
- True streaming over /ws/transcribe (OnlineRecognizer: live partials +
  per-endpoint finals); offline models surface partials via short re-decode.

Frontend:
- New "Voice" settings panel (enable, Toggle/Hold mode, model picker with
  offline/streaming/recommended badges + per-model download/delete).
- Live word-by-word typing via simulate_type (enigo) with prefix-diff delta and
  backspace correction; paste fallback retained, no double-insertion.

Deps: sherpa-onnx>=1.13.3 (+ sherpa-onnx-core); uv.lock regenerated, Docker
frozen-install verified. API route-inventory snapshot updated. 40+ new tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(dictation): register sherpa-onnx-asr engine in README + features inventory

Fixes the docs-drift CI guard: the new sherpa-onnx-asr ASR engine existed in
the registry but not in docs/features.yaml or README. Adds the live-dictation
engine row to the ASR Engines table, bumps the engine counts (8→9), and adds
the inventory entry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): fold live-dictation into the [0.3.8] section

main is 0.3.8 (untagged), so the dictation feature belongs in that release, not
a separate [Unreleased] block. Merge the two Added lists under one [0.3.8],
refresh the headline to lead with live dictation, and correct the capture
description to reflect live word-by-word typing (not paste-on-pause).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 02:43:53 +05:30
e7d358b541 fix(design): don't forward a clone profile_id in design mode (gender attribute no-op) (#674) (#679)
In Voice Design, choosing "Male" (or any gender) could have no audible effect.
Root cause: the design synthesize branch forwarded the selected `profile_id`
alongside the design instruct. If that profile is a CLONE (reference audio, no
instruct) — e.g. the demo voice selected by default — the backend clones it, and
the reference voice's gender/timbre overrides the "male" attribute, so the design
slider appears to do nothing.

Fix: a pure `designModeProfileId(selectedProfile, profiles)` decides what to send
in design mode — it suppresses a KNOWN clone (no instruct) so the design
attributes drive the voice, while a design profile (carries an instruct) still
passes through to re-render a designed voice. Conservative: an unknown id
(profiles not loaded) or a design profile is unchanged, so this only removes the
gender-hijacking case. Threaded `profiles` into useTTS.

Test: voiceInstruct.test.js — clone (no/empty instruct) → null, design profile →
its id, empty/null → null, unknown id → passthrough.

Closes #674

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 17:12:32 +05:30
2339cc8e85 fix(model): actionable "reinstall transformers" hint on a corrupted-install model-load error (#676)
A model load failed with `[Errno 2] No such file or directory:
'…/site-packages/transformers/models/qwen3/modeling_qwen3.py'` — the user's
transformers install was incomplete (the file is missing while a correct 5.3.0
install has it; an interrupted `uv sync` / antivirus / partial update drops it).
The System Check showed the raw path + "Check logs and try restarting", which is
useless — restarting can't restore a missing file.

Two fixes:
1. core.failure.classify(): recognize this corrupted-install variant. It's a
   FileNotFoundError, not an ImportError, so the existing TRANSFORMERS_IMPORT
   match ("could not import module"/"AutoFeatureExtractor") missed it. Now also
   matches a "no such file"/"errno 2" + "transformers" + "site-packages" signal
   (substrings checked separately so it works on POSIX `/` and Windows `\`
   paths). An unrelated package's missing file is NOT mislabelled.
2. model_manager._load(): build the /model/status error via build_failure so it
   carries the classified hint AND strips the home dir, instead of storing the
   raw str(exc). The System Check now shows "Your transformers install is
   incomplete. Reinstall it (uv pip install --reinstall transformers) or switch
   ASR to faster-whisper" — the existing TRANSFORMERS_IMPORT hint.

Docs: troubleshooting §1a documents the error + the reinstall fix.

Test: test_failure_classify.py pins the POSIX + Windows path forms classify as
TRANSFORMERS_IMPORT with a "reinstall" hint, and that an unrelated package's
missing file does not.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 14:56:29 +05:30
0af744bc3a fix(setup): use the backend's authoritative aggregate for live download progress (#675)
The first-run download line showed wrong numbers — e.g. "8% · 1 KB/s · 0.0 MB
left" on a 2.4 GB model that was barely started. The #657 display summed the
PER-FILE tqdm SSE events on the frontend, but under parallel/segmented fetch the
big weight shards report total/rate as 0, so the sum was garbage (tiny total →
"0.0 MB left", a couple small files → "1 KB/s").

The backend already solves this: download_aggregator emits a throttled
`phase:"aggregate"` event with one windowed rate + ETA + bytes_done/total_bytes
(+ files done/total), seeded by the dry-run preflight totals — precisely because
summing per-file on the client is unreliable. But WizardLibrary dropped that
event (`if (!ev.filename) return prev`) and never used it.

Fix: capture the `aggregate` event into per-repo state and render from it
(new pure `progressFromAgg`), falling back to the per-file sum only until the
first aggregate arrives. Now the line shows real, live values, e.g.
"8% · 5.2 MB/s · 2.2 GB left · ~7m", updating in real time and landing on 100%.

Test: wizardLibraryAggregate.test.js — progressFromAgg yields correct
pct/remaining/rate/ETA from real totals (2.2 GB left, 5.2 MB/s — not 0.0 MB /
1 KB/s), returns null until totals are known, and caps pct at 100.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 14:30:41 +05:30
87cce2e2b8 docs(changelog): draft the [0.3.8] release section (#673)
Renames [Unreleased] → [0.3.8] — 2026-06-24 with a one-paragraph headline in the
house style, and adds the entries merged since v0.3.7 that weren't yet logged:
faster default downloads + the surfaced HF-token card (#669/#657), the auto-play
toggle (#666), the status-bar version badge (#671), and the Windows/stability
fixes — WhisperX-on-Windows (#630), transcribe timeout (#656), preview playback
(#653/#659), stale dub session (#660), bad-instruct 400 (#664/#612), Insert
popover clipping (#672), and the M1 startup-hang bound (#632). A fresh empty
[Unreleased] is left above it for the next cycle.

This makes cutting v0.3.8 a single `git tag` away: release.yml extracts this
section verbatim as the GitHub Release body, so the tag ships real notes instead
of the auto-generated fallback. (Owner adjusts the date if tagged on another day;
no version files touched — this is docs only.)

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:39:36 +05:30
8c45d4a9f2 fix(clone): cap the ⊕ Insert popover height so it can't clip off the top of the window (#672)
On the Voice Clone tab, the ⊕ Insert popover (15 expression-token chips) opens
upward from the lifted button (`bottom: 60px`) but had NO max-height — so the
wrapping chip grid grew unbounded and, when the button sat high in a tall script
panel, the popover shot past the top of the app window and the first rows were
clipped behind the title bar (reported with the tokens overflowing above the
OmniVoice header).

Cap it: `max-height: min(280px, calc(100vh - 120px))` + `overflow-y: auto`
(+ `overscroll-behavior: contain`). The popover is now a compact, scrollable box
that sits just above the button and always stays within the viewport, regardless
of how tall the script is or where the button lands. Horizontal guard (#481) and
the upward anchor are unchanged.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:50:37 +05:30
dba8934aa2 feat(footer): clickable version badge → Updates, with an update-available indicator (#671)
The bottom status bar showed no version and had no quick path to updates. Add a
small `v<version>` badge next to the network/share icon; clicking it opens
Settings → Updates. When an update is available (or downloaded and ready), the
badge highlights and shows a pulsing notification dot, and its tooltip names the
new version — so users can see at a glance that an update is waiting and one
click takes them to install it.

Mechanism: a one-shot `pendingSettingsTab` hand-off in the UI store (mirrors the
existing `pendingProfileId` pattern) + an `openSettingsTab(tab)` convenience that
sets the tab and navigates in one call. Settings consumes it as its initial tab
and clears it (an effect covers the already-open case). The indicator reads the
existing `updateStatus`/`updateVersion` from the updater slice — no new update
plumbing. Version from the shared APP_VERSION constant; new strings via i18n.

Test: openSettingsTab.test.js — the convenience sets mode=settings + the pending
tab, and the value can be cleared after consumption.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:48:51 +05:30
b7cecde57e feat(setup): faster downloads by default + prominent, encouraged HF-token entry (#669)
Two changes that make first-run downloads faster and easier to speed up further.

1. Segmented (multi-connection) downloader is now ON by default. The app forces
   the legacy-LFS path (HF_HUB_DISABLE_XET=1) for clear progress, but that path
   is single-stream and slow — which is why downloads felt sluggish. The built-in
   IDM/uGet-style segmented accelerator (parallel byte-ranges, live speed/ETA)
   was already implemented but defaulted OFF. Flip it ON: it only engages when
   Xet is inactive (the default), and ANY failure falls back to snapshot_download
   ("can never compromise a correct install"). Pure-httpx, cross-platform,
   auth-safe (token never forwarded to a CDN). Override with
   OMNIVOICE_SEGMENTED_DOWNLOAD=0.

2. The Hugging Face token field is now a prominent, always-visible card right
   above Continue — was a collapsed "advanced" fold almost nobody opened. A free
   token gives authenticated downloads (higher rate limits, fewer stalls), so it
   pairs with change #1 to keep the parallel fetch from getting throttled. The
   card leads with the speed benefit, shows a saved-state, and adds a one-click
   "Get one free →" link to huggingface.co/settings/tokens.

Docs: downloading-models.md updated — the legacy-LFS section now documents the
default-on segmented accelerator + the HF-token speed tip, and the tuning table
reflects OMNIVOICE_SEGMENTED_DOWNLOAD=0 as the disable knob (docs-sync).

Test: test_segmented_download_default.py pins the new default ON and that the
env override still disables it; existing FDL-08 behavior tests stay green.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:35:57 +05:30
46abe2e29a feat(settings): add opt-out for auto-playing the preview after a render (#666) (#667)
After a render finishes in Voice Clone / Design / a profile's try-it box, the
output preview auto-plays unconditionally — `autoPlay` was hardcoded on the
WaveformPlayer. A user batch-generating Korean clone segments asked to turn it
off so each finished clip doesn't start playing on its own.

Add a persisted `autoPlayPreview` pref (default ON — preserves current behavior)
with a Settings → Appearance toggle, and thread it into the two preview call
sites (VoicePreview.jsx, VoiceProfile.jsx) so `autoPlay={autoPlayPreview}`.
WaveformPlayer already gates playback on the prop, so off = no auto-play; the
manual Play button is unaffected. Cross-platform-parity safe: it's a pure UI
preference that behaves identically on macOS/Windows/Linux, default unchanged.
New strings go through i18n.

Test: AppearancePanel.test.jsx — the toggle defaults checked (ON) and flipping
it sets the store to false.

Closes #666

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:48:46 +05:30
81a007552b fix(generate): classify a bad-instruct error as a 400, not a 500 "ran out of memory" (#664) (#665)
A user typed free-form prose ("Speak with high energy … like a podcast host")
into the voice-design instruct field and got a **500** whose message read "TTS
engine stopped mid-generation. This usually means it ran out of memory. Try the
Flush button …" — with the real cause ("Unsupported instruct items found …")
buried as the underlying error. The user is told to Flush for an OOM that never
happened; the actual problem is a rejected instruct.

Root cause: `_resolve_instruct` raises on unknown/conflicting instruct items, but
by the time the error reaches `_oom_friendly_reraise` it's no longer a bare
`ValueError` (a lower layer wraps it), so the route's `except ValueError -> 400`
guard misses it and it falls through to the generic OOM `RuntimeError`. v0.3.7
has had that guard since v0.3.6 yet still produced the OOM message — proving the
error arrives wrapped, so type-based detection is insufficient.

Fix: in `_oom_friendly_reraise`, detect the instruct-validation **message
signature** ("unsupported instruct items" / "conflicting instruct items" / "in a
single instruct") regardless of exception type and re-raise a clean `ValueError`,
so the route returns a **400 with the instruct guidance** instead of a 500 OOM.
This is version-independent and complements the client-side guard (#658/#612):
it also covers API/MCP callers and stored profiles whose instruct slips through.

Test: two cases in test_generation_audio_guard.py — a bare instruct `ValueError`
and one wrapped in a `RuntimeError` both reclassify to a ValueError without the
"ran out of memory" text; the generic OOM path is unchanged.

Closes #664

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 07:22:51 +05:30
db016d4675 fix(dub): reset stale dub session gracefully instead of erroring "Job not found" (#660) (#661)
A persisted `dubJobId` outlives the backend's in-memory job store — after a
backend restart (or once a job is cleaned up), resuming/retrying a dub returns
404 "Job not found. It may have been cleaned up or was never created." The UI
surfaced this expected stale-session state as a hard error toast *with a "report
a bug" prompt* (toastErrorWithReport), so a user who just reopened the Dub tab
(#660: only action was view:dub) got a scary, un-actionable error for what is
really "your old session is gone — start a new one."

Fix the class: add a pure `isExpiredDubJobError(err)` predicate (matches the
dub_core preflight message, the dub_generate expired-session message, and a bare
404 "Job not found") and a `_resetStaleDubSession()` helper that clears the dead
job id/state, drops any pill, and shows a calm info toast inviting a fresh
upload. Wired into the two handlers that operate on a pre-existing job —
retry-transcribe (the #660 path) and SRT import. The fresh upload/ingest paths
are intentionally left reporting real errors: a just-created job going missing
*is* a bug worth reporting.

Test: dubExpiredJobError.test.js pins the predicate against both backend
messages + a bare 404, and asserts unrelated failures (stream dropped, CUDA OOM,
abort) stay reportable.

Closes #660

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 04:53:14 +05:30
9d2e395437 fix(net): Windows preview playback — 127.0.0.1 loopback + quieter decode-fallback log (#659)
Two coupled Windows fixes for the preview/blob audio path (the "playBlobAudio
decode error: EncodingError: Unable to decode audio data" users see in
Logs → Frontend on Windows).

1. apiBase 127.0.0.1, not localhost (Tauri context). The backend binds IPv4
   127.0.0.1 only; on Windows "localhost" often resolves to ::1 (IPv6) first, so
   requests miss the backend. The main client (api/client.ts) already did this
   since #174, but utils/apiBase.ts lagged on "localhost" — and its one consumer
   is utils/media.js's preview upload, the #653 fallback. So #653's streamed
   fallback fetched http://localhost:3900/preview/upload and FAILED on Windows,
   leaving preview playback broken even after #653. Align the two resolvers.

2. Quieter, accurate logging in playBlobAudio. The Web Audio decodeAudioData
   path is EXPECTED to fail for long-form / AAC renders on WebView2 and is
   recovered by the streamed fallback — yet it logged at error level, so users
   saw a red "decode error" even when playback succeeded. Downgrade that branch
   to console.warn ("falling back to streamed playback"); reserve error level for
   the real failure (both decode AND fallback failed). With fix #1 the fallback
   now actually reaches the backend on Windows, so the recovery completes.

Tests: apiBase.test.ts asserts Tauri → http://127.0.0.1:3900; the existing
playBlobAudioFallback.test.js (#653) still passes (fetch hits /preview/upload,
plays the HTTP URL, never a blob:).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 03:15:41 +05:30
10b9d6950d fix(synthesize): validate clone-path instruct client-side so non-EN/ZH prose can't 400 (#612) (#658)
A Vietnamese user typed a free-form Vietnamese description into the voice style
(instruct) field and got "400 Bad Request: Unsupported instruct items found in
quảng cáo, sôi nổi và thu hút". The instruct field is a fixed EN/ZH style-tag
vocabulary (the model's trained tokens: gender/age/pitch/accent/dialect/whisper);
the backend _resolve_instruct deliberately *raises* on unknown items.

The design path already guarded this: it runs the free-text through
buildDesignInstruct(), keeping valid tags, dropping the rest, and surfacing a
localized warning toast (#115/#114). But the *clone* path
(defineMethod === 'audio') appended the raw `instruct` string straight to the
request — so a clone + free-text style in any non-EN/ZH language round-tripped to
a 400 instead of being handled locally.

Fix (localized client-side guard, the chosen approach): route the clone path's
free-text through the same buildDesignInstruct({}, instruct) guard. Valid style
tags survive (a clone can still ask for "whisper"); unsupported items drop with
the existing localized `tts_errors.ignored_unsupported` toast; synthesis proceeds
in the user's language without style control instead of failing outright. No
backend/engine change — the model genuinely can't honor non-EN/ZH instructs, so
this makes the failure graceful and understandable rather than a raw 400.

Test: two cases in voiceInstruct.test.js pin the clone scenario — a fully
Vietnamese instruct yields "" + all items in the unsupported bucket, and a mixed
"whisper, sôi nổi" keeps "whisper" while flagging the prose.

Closes #612

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 03:02:39 +05:30
e87c13e919 feat(setup): show live download rate + size-remaining, surface HF token as a speed lever (#657)
The first-run Models & Engines page showed only "downloading…" (and, once totals
arrived, a bare percent + ETA). Users asked to see the actual download rate, the
size remaining, and a way to speed downloads up.

The backend already streams per-file byte counts and a windowed rate over SSE —
the UI just wasn't surfacing it. Changes (frontend-only):

- aggregate() now also returns live rate + bytes-remaining (was pct + ETA only),
  and is exported so the speed/remaining math is unit-tested.
- The download line now reads e.g. "38% · 5.2 MB/s · 1.2 GB left · ~3m", each
  part shown only once the stream has it (still degrades to "downloading…" early).
- New fmtBytes()/fmtRate() helpers (MB/GB, MB/s↔KB/s).

The Hugging Face token field already existed but was buried in an "advanced"
fold and framed only as "unlocks gated models" — so users hunting for a faster
download never found it. Reframed the title/hint to lead with what they want:
authenticated downloads are faster, have higher rate limits, and stall less
(and still unlock gated models like pyannote diarization). Token persistence and
the segmented/faster downloader (segmented_download.py) are unchanged — this just
makes the existing speed levers visible.

Test: frontend/src/test/wizardLibraryAggregate.test.js — aggregate sums bytes,
ignores completed-file rate, returns nulls before totals; fmtBytes/fmtRate
formatting + idle blanks.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 02:54:18 +05:30
252f0d4fac fix(asr): bound whole-file transcription so a stall isn't reported as "can't reach backend" (#656)
A Windows/CUDA user (Vietnam) hit "Can't reach the local backend" only when
dubbing/transcribing. Their log proves the backend started fine — model loaded,
preload complete, 25 models — and the log ends right after
`whisperx transcribing …tmp.wav`. The backend was alive; the *transcription*
stalled (large-v3 ASR contending with the resident TTS model for VRAM on an
8 GB-class GPU), which the UI surfaces as an unreachable backend.

Root cause (class, not instance): the chunked dub pipeline already bounds each
chunk (OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S), but the *whole-file* transcribe
paths ran unbounded:
  - dub QC re-transcribe (dub_export)
  - dictation (capture)
  - OpenAI-compat /audio/transcriptions
A slow/stuck transcribe on any of these hung the request AND held a GPU-pool
worker — indistinguishable from a dead backend.

Fix: add run_transcribe_guarded() in services/asr_backend.py — a shared
asyncio.wait_for wrapper (ASRTimeoutError, a TimeoutError subclass) with a
generous env-tunable bound (OMNIVOICE_ASR_TRANSCRIBE_TIMEOUT_S, default 300 s).
On timeout the request returns 504 with actionable guidance (backend is alive;
free VRAM / pick a smaller ASR model / use CPU; restart to clear the stuck
worker) instead of hanging forever. Wired into all three whole-file paths.

Docs: new troubleshooting §14 — "Can't reach the local backend during
transcription/dubbing" — explains it's ASR weight/VRAM pressure, not a network/
mirror problem, and corrects the misconception that a "Network → Restricted/Global
mirror" Settings toggle exists (the Network control is LAN sharing). Serves the
#602/#585/#567 "can't reach backend" cluster.

Test: backend/tests/test_asr_transcribe_timeout.py — slow fn raises ASRTimeoutError
with the actionable message, fast fn passes through, subclass-of-TimeoutError so
the openai_compat broad catch still maps to 504.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 02:46:14 +05:30
d3cebe58a0 fix(asr): cross-platform speechbrain lazy-import guard — unblock WhisperX on Windows (#630) (#655)
WhisperX (the default ASR) aborts transcription with zero segments on Windows
only, surfacing "Lazy import of LazyModule(...speechbrain.integrations.k2_fsa...)
failed" (#630) or its generic wrapper "Transcribe stream dropped..." (#611, #647).

Root cause is in speechbrain 1.x. It exposes optional integrations (k2_fsa,
numba losses, spacy/flair nlp) as LazyModule redirects in sys.modules. Stray
introspection during whisperx.load_model (pyannote -> speechbrain) — PyTorch's
op-registration machinery, pickling, a dir()/hasattr walk — touches one of these
redirects. speechbrain suppresses such inspect-triggered imports via a guard, but
the guard checks filename.endswith("/inspect.py") with a hardcoded POSIX
separator. On Windows the frame filename uses backslashes, the guard misses, the
redirect actually imports k2_fsa -> import k2 -> k2 not installed -> ImportError
that bubbles out and kills ASR. macOS/Linux use forward slashes, so the guard
fires and the feature works — a Windows-only break of a cross-platform default
(P0 parity).

Fix the whole class (every optional-integration redirect, not just k2) by
re-implementing LazyModule.ensure_module with a separator-agnostic basename check
(normalise both "\\" and "/"), applied right before whisperx loads. Idempotent;
a no-op on macOS/Linux and when speechbrain is absent; genuine missing-dep
accesses from real user code still raise ImportError — only inspect-triggered
spurious imports are suppressed, now on every platform.

Regression test fakes the importer frame with Windows- and POSIX-style inspect.py
paths plus a real-caller path, so it pins the behaviour on any CI host (fails
before the fix on the Windows-path case, passes after).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 02:06:21 +05:30
1f29e374be fix(audiobook): play long-form preview via streaming HTTP, not decodeAudioData (#653) (#654)
In-app preview of a finished audiobook/story did nothing on Windows. playBlobAudio
(Tauri path) decodes the whole render into one PCM AudioBuffer via Web Audio
decodeAudioData, which throws "EncodingError: Unable to decode audio data" on a
long-form .m4b/AAC under WebView2. The catch-block fallback used a blob: URL,
which the file's own fileToMediaUrl notes does NOT play in a Tauri <audio>
element — so it silently played nothing.

The fallback now uploads the blob to /preview/upload (ffmpeg-extracts a
streamable WAV server-side) and plays the returned HTTP URL via <audio> — the
exact pattern video previews already use. Streams instead of whole-file-decode,
so it also fixes hour-long renders regardless of platform. Short WAV TTS previews
keep the fast decodeAudioData path. Regression test pins that the fallback hits
/preview/upload and plays an HTTP URL (never blob:). No version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 00:21:48 +05:30
4eac1b50d5 chore(desktop-prod): add --keep-models for fast fresh-app runs (#650)
`bun desktop-prod` (clean) wipes everything including the HF model cache, so
every fresh-install emulation re-downloads multi-GB weights — slow and bandwidth
-heavy, and the exact pain users on flaky networks hit. --keep-models wipes
app/backend data, logs, and webview state for an honest first-run, but KEEPS the
model cache so the weights aren't re-pulled. Ignored under --keep-data (which
keeps everything). Adds the `desktop-prod:keep-models` convenience script.
Scripts-only package.json change — no deps, bun.lock unaffected.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:30:14 +05:30
1dc8eb1adb feat(setup): calmer descriptions + surface platform-tuned models by default (#649)
Two first-run setup polish items:
1. Descriptions dimmed + tightened (opacity 0.72->0.55 / 0.68->0.5, smaller
   line-height/reserve) and shortened (subtitle, compute, channel, mode copy).
2. "Models & engines" (WizardLibrary) now surfaces optional models tuned for the
   detected platform — those whose catalog "platforms" tag matches the host
   (MLX mac-ARM on Apple Silicon, CUDA variants on NVIDIA) — up-front with a
   green "recommended" chip + their note, folding only the universal long tail.
   Generic across platforms; graceful when none match. No backend change (the
   /models API already ships "platforms" + the host "platform_tags").

isPlatformPick extracted as a pure exported helper; 6 vitest cases. en.json +
JSX fallbacks synced; orphan check clean; vite build green. No version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:22:58 +05:30
41e866b46c test(onboarding): guard demo clip stays un-ignored + a bundled resource (#621) (#648)
A Windows user's log showed 'Demo audio not found … demo_voice.wav — skipping
onboarding seed'. The local bundle DOES ship the clip (verified), so that user
just has a pre-#633 build — but the existing test only checks the file exists in
the repo. It misses the two ways the clip could silently drop from ALL builds
while still sitting in the repo: (1) the .gitignore un-ignore allowlist
(!backend/assets/samples/*.wav) being weakened — gitignore-aware build walkers
would then skip it; (2) backend/ being removed from tauri.conf.json bundle
resources. Pin both. test-only.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 16:57:48 +05:30
Palash Debnathandmergetest 8e4044bb37 fix(i18n): clear orphan-key advisory — add en bootstrap.lines, drop dead gallery.cat_* (#646)
The locale orphan-key judge flagged 20 non-en locales carrying keys absent from
en. Two distinct causes:

1. bootstrap.lines (used at BootstrapSplash.jsx:467, t('bootstrap.lines',{count}))
   existed in de/es/fr/ja but NOT en — so English (and 16 locales falling back to
   it) rendered the literal key instead of '{{count}} lines'. Added to en.
2. gallery.cat_* (anime/books/celebs/disney/gaming/marvel/news/politicians) were
   renamed to archetypes.use_* long ago (VoiceGallery.jsx:309) but left orphaned
   in 20 locales — 160 dead keys. Removed.

Zero orphans remain. Flipped the probe test to assert the judge now PASSES
(regression guard). Locale files edited losslessly (json indent=2, ensure_ascii
=False, trailing-newline preserved). No version bump.

Co-authored-by: mergetest <test@local>
2026-06-23 13:27:43 +05:30
d8b059813a fix(startup): timeout-bound MCP session-manager start to stop M1 startup hang (#632) (#645)
* fix(startup): timeout-bound MCP session-manager start to stop M1 hang (#632)

A reporter's faulthandler thread dump showed the asyncio loop alive but the
lifespan suspended at an await with an idle pool worker + a leaked semaphore —
the MCP Streamable-HTTP session manager hanging on its anyio task group during
startup (Apple-Silicon M1). Because `enter_async_context(_sm.run())` is awaited
before yield, the hang meant 'Application startup complete' never fired and the
backend was unreachable with no error — a P0 (default feature dead on a platform).

The MCP layer is explicitly best-effort, but the old guard only caught
exceptions, not hangs. Bound the start with asyncio.wait_for
(OMNIVOICE_MCP_START_TIMEOUT_S, default 30s): a hang → logged warning + backend
serves without MCP. Extracted _enter_mcp_session_manager + _mcp_start_timeout_s;
4 regression tests (hang→False fast, healthy→True, None→noop, env override). No
version bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(startup): run MCP in its own task (anyio task-affinity) — fix CI cancel-scope error

The first attempt wrapped enter_async_context in wait_for, which entered the MCP
anyio task group in a throwaway sub-task while the AsyncExitStack exited it on the
lifespan task → 'Attempted to exit cancel scope in a different task' (caught by
test_coverage_critic's real backend boot). Correct fix: _serve_mcp owns the full
enter→exit in ONE task; _start_mcp_session_manager only waits (with timeout) on a
ready Event. A hang still can't block startup, and enter/exit share a task.
Shutdown signals stop + bounded-awaits the task. Tests updated (5; incl broken-
manager case). test_coverage_critic now boots+shuts down clean.

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 12:49:39 +05:30
6819feb8b2 fix(dub): skip yt-dlp mtime stamp to avoid [Errno 22] on Windows (#642) (#644)
Dubbing a URL could fail with 'Unable to download video: [Errno 22] Invalid
argument' on Windows: yt-dlp stamps the downloaded file's mtime with the video's
upload date, and an out-of-range/invalid timestamp makes os.utime raise
[Errno 22], aborting the ingest. We download to a throwaway original.* and never
use its mtime, so set updatetime=False (yt-dlp --no-mtime). Regression test
asserts the opt is set. No version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 11:53:44 +05:30
79f3e35682 docs(troubleshooting): add stuck-download / incomplete-cache recovery (#622) (#643)
The 'stuck on the download page, model folder has only refs/ no weights' case
(a connection dropping/blocking mid-pull) is a recurring support report but
wasn't in the install troubleshooting guide. Add section 13 with the recovery
steps + antivirus/VPN/mirror escalation + a huggingface-cli manual fallback.
Docs-only; no version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 11:50:42 +05:30
b15acbaae9 fix(dub+generate): yt-dlp 403 player-client fallback (#625) + non-finite audio guard (#629) (#635)
Two independent fixes from issue triage; no version bump.

#625 — yt-dlp 403 on the media download (some videos serve formats
signature-protected to the default player client) is not transient, so the
existing broken-pipe retry (#579) kept 403ing. The URL download now escalates
the YouTube player client (tv → android → web_safari) on a 403 before giving up;
a 403 no longer counts against the transient-retry budget.

#629 — a numerical glitch in the model (seen on MPS) could leave NaN/inf samples
that write an unreadable WAV; a downstream decode then failed with an opaque
"ffmpeg returned error code: 183 / Invalid data", surfaced to the user as a
misleading "ran out of memory". Sanitize non-finite samples to silence in
_apply_effect_chain (single chokepoint, covers the raw path too) so the WAV is
always decodable, and classify a decode/ffmpeg failure as unreadable-audio
rather than OOM in _oom_friendly_reraise.

Tests: 403 escalation order + success-on-alternate-client; NaN/inf sanitize +
finite-passthrough + decode-error classification. Full suite 1851 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:36:32 +05:30
0cf6bb3087 feat(startup): watchdog that dumps thread stacks on a startup hang (#632) (#634)
A silent hang during the FastAPI lifespan startup (reported as a Mac M1 hang
after 'Loading weights: 527/527') leaves the app unusable with no error: weights
load, then 'Application startup complete' never fires. Without a thread dump the
deadlock is invisible.

Arm faulthandler.dump_traceback_later at the top of the lifespan and cancel it
the instant startup completes (just before the yield). If startup stalls past
the window (default 300s, OMNIVOICE_STARTUP_WATCHDOG_S to tune, 0 to disable),
every thread's stack is dumped to stderr → backend_err.log, capturing the hang
point for #632 and any future startup deadlock. Best-effort + exit=False, so the
diagnostic can never itself break or kill startup; a normal (even slow-download)
boot disarms it first and never trips.

No version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:56:22 +05:30
a28a19dc63 fix(onboarding): commit + bundle the demo voice clip (#621) (#633)
backend/assets/samples/demo_voice.wav is a build artifact (generated by
scripts/build_demos.sh) that was never committed, so it shipped absent from
installs: onboarding logged 'Demo audio not found', seeded nothing, and the
Launchpad was empty on first run + the /demo_audio route was unavailable.

The file is already un-ignored in .gitignore and bundled via the Tauri
'backend' resource — it just needed to exist in git. Commit it (regenerated
via the script's say/Samantha path, 24kHz mono 16-bit, content matching
DEMO_REF_TEXT) so first-run works on every platform. Onboarding keeps its
graceful skip (now with a regenerate hint) for a partial checkout.

Regression test guards the asset is present + valid and that onboarding seeds
the demo profile from it (and is a no-op on a non-empty DB). No version bump.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:56:07 +05:30
a63c8e851b fix(dub): speaker-aware re-split so merged speaker turns separate (#486) (#616)
Segmentation groups words into sentences BEFORE diarization, so a two-speaker
exchange can land in one segment; assign_speakers_* then only relabels it with
the majority speaker, losing the turn boundary (the second half of #486 — the
per-speaker voice auto-assign was fixed in #490).

Add a post-diarization pass that re-splits any segment whose words span >1
speaker at the word-level boundary, assigning each piece its speaker:
- backend/services/segmentation.py: resplit_segments_by_diarization /
  resplit_segments_by_turns + a pure _resplit_core. Single-speaker segments are
  returned BYTE-FOR-BYTE UNCHANGED (same dict/id/text/start/end) — the
  no-single-speaker-regression guarantee. Pieces keep the segment's outer
  start/end (preserving onset-snap) and use word times for interior splits, so
  they exactly cover the original span. A lone mis-attributed word is smoothed,
  not split (diarization noise).
- backend/api/routers/dub_core.py: accumulate global-timeline words alongside
  segments; apply the re-split after both the pyannote and FunASR-turns assign.
  Heuristic fallback (no word-speaker data) is untouched.

8 regression tests pin the invariant + the split/3-way/noise-smoothing/label
behaviour. Full suite: 1836 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:39:07 +05:30
1fe68ba11e fix(setup): weight-aware install-state so truncated model cache isn't read as installed (#622) (#626)
A first-run user whose model download was interrupted after the config/
tokenizer files landed but before the weight shard got stranded on the
Models & Engines page: GET /models computed "installed" purely from cache
size on disk, so a size-positive-but-weight-less cache reported installed=true,
the wizard hid the re-download button, and the model manager (Settings → Models)
that could repair it was unreachable behind the wizard gate.

Make install-state weight-aware. The boolean weight-floor scan now lives in
models.py (the lowest module in the setup import graph) as snapshot_has_weights()
+ cache_is_complete(); list_models() and recommendations() downgrade a truncated
cache to installed=false (+ an explicit incomplete=true on /models), so the
existing "install" action re-appears and the user can re-download in-wizard.

Fixes the whole class, not just /models: download.py's install-time validator
now delegates to the same shared scan (one source of the floors, can't drift),
matching the load-time repair in model_manager.py (#581/#606).

config_only repos (pyannote/speaker-diarization-3.1 — a pipeline whose real
weights live in referenced sub-repos and whose own cache is legitimately tiny)
carry a new config_only:true hint in models.yaml and are exempt, so they're not
false-flagged as incomplete.

Tests: tests/test_mm2_lifecycle.py — snapshot_has_weights truncated-vs-complete,
cache_is_complete on a truncated weight repo + config-only exemption, and
list_models downgrading a size-positive truncated cache to installed=false /
incomplete=true. Full backend suite green (1832 passed).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 22:38:38 +05:30
d21fb765e2 feat(stories): tagged-script [Name] parsing → auto multi-voice cast (#487) (#615)
Paste a `[Alice] … [Bob] …` podcast/audiobook script into Stories and Auto-cast
now builds the cast and assigns a voice per character — no manual setup. This
sits entirely on the existing Stories pipeline (autoCast → storyToSpans →
/longform/render); the only missing piece was recognizing the `[Name]` tag
format, which parseScript now auto-detects and routes through a new
parseTaggedScript (alongside `NAME:` screenplay + quoted prose).

- parseTaggedScript: `[Name] dialogue`, multi-line blocks join until the next
  tag, prose before the first tag → Narrator. Inline synthesis markers
  ([pause], [pause 500ms], [voice:ID], [fast], [spell]) are NOT treated as
  speakers (no colon + reserved-keyword guard), so they stay in the text.
- parseScript auto-routes tagged scripts so the existing Auto-cast button works
  unchanged; single-line re-render is already covered by the content-addressed
  chapter cache.
- autocastHint advertises all three formats. 16 parseScript tests pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 12:55:20 +05:30
de80856cd9 test: backend route-inventory + webUI feature-coverage guards (#609)
* test: backend route-inventory snapshot + webUI feature-coverage guards

A reusable testing system that verifies every feature surface is present:
- tests/test_api_route_inventory.py: boots the app, diffs all 213 routes vs a
  committed snapshot (tests/fixtures/api_routes.txt), guards a critical-endpoint
  set, and floors the route count — any endpoint drift fails CI.
- scripts/dump_api_routes.py: regenerates the snapshot.
- frontend featureCoverage.test.js: every AppMode has a render branch, every
  lazy-imported page file exists, every feature has an i18n namespace.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(changelog): note the feature-coverage test system

* test(api-inventory): isolate via subprocess + exclude env-dependent mounts

CI surfaced two flaws in the first cut:
- the in-process app import + sys.modules purge polluted later DB-touching
  tests (a cascade of 404s in test_dub_subtitles_309 etc.);
- the snapshot included StaticFiles mounts (/demo_audio) and a conditional
  GET / root that register based on filesystem state, so a macOS-generated
  snapshot didn't match a fresh Linux CI runner.

Compute routes in an isolated subprocess (scripts/dump_api_routes.py --print)
and cover only the deterministic router surface (drop Mounts + root). 209
routes; inventory + previously-polluted tests now pass together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:33:00 +05:30
1575baca36 fix(design): heal validator-rejecting instruct on design voices (#594/#571/#596) (#600)
* fix(design): heal validator-rejecting instruct on design voices (#594/#571/#596)

A designed voice could persist an `instruct` the engine validator rejects —
either the literal "[object Object]" from a pre-fix build (#550) or freeform
prose typed into the style field — so every Generate/Dub that used the voice
failed with `Unsupported instruct items found in …` (400/500, and "Can't reach
the local backend" when it tore down mid-render). Migration 0006 only *blanked*
"[object Object]", which silently discarded the design — an Indonesian female
voice then rendered male (#594).

Fix the whole class by healing at every seam and rebuilding from the
authoritative source (the design's saved `vd_states` category picks):

- omnivoice/utils/voice_design.py: add sanitize_instruct / instruct_from_vd_states
  / heal_design_instruct — forgiving (never raise), drop poison/prose to valid
  tags, and rebuild tags from vd_states when the stored value is unusable.
- profiles.py: sanitize + rebuild at save (POST) and sanitize at edit (PUT), so
  no poisoned instruct can ever be persisted again.
- generation.py + dub_generate.py: heal whenever a profile drives synthesis, so
  legacy poisoned rows resolve to valid tags instead of 400-ing.
- migration 0007: heal existing profiles in place (recovers gender/age/pitch
  from vd_states), self-contained (frozen vocab snapshot) so it never drags
  torch into startup; supersedes 0006's blanking. Backward-compatible.

Tests: unit coverage for the healer, a migration test driving 0006->0007 on the
real schema, a parity guard so the frozen snapshot can't drift, and two API
guards. Corrected one existing test that had encoded the #594 behaviour.

Resolves #571, #594, #596; removes a major driver of the "Can't reach backend"
reports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(cjk): allowlist migration 0007's frozen dialect-tag snapshot (#564)

The 0007 instruct-heal migration carries a frozen copy of the design-tag
whitelist (incl. Chinese dialect tags) so it stays self-contained; add it to
the hardcoded-CJK allowlist like omnivoice/utils/voice_design.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:21:19 +05:30
31ba6d3d27 fix(transcribe): surface the real ASR-load failure instead of a generic "stream dropped" (#578) (#608)
When WhisperX (or any ASR backend) failed to load its model, the transcribe
SSE stream dead-ended on a generic "Transcribe stream dropped … Likely ASR
backend failed to load" message with no actionable cause.

Two root causes, both fixed:

1. WhisperX loads lazily inside transcribe(), so a load failure (faster-whisper
   weights, CTranslate2/cuDNN mismatch, torch-2.6 weights-only VAD regression)
   was buried in per-chunk errors and retried on every chunk. Added
   ASRBackend.ensure_loaded() (no-op default; WhisperX triggers its lazy
   loader) and call it in the transcribe pre-flight so the genuine cause
   surfaces once, up front, as a structured error event.

2. The pre-flight and audio-load error paths closed the SSE stream with a bare
   `error` and no terminal `done`, so the browser's native EventSource
   connection-drop could race and win against the structured error — discarding
   the real cause. Every terminal error now emits `done`, and the frontend
   latches the structured cause so a connection drop can't overwrite it with
   the generic message.

Adds a fail-before/pass-after regression test driving the stream's async
generator through the ASR-load-failure path; updates the existing #516 fake
backend to the new ensure_loaded() contract; CHANGELOG ### Fixed entry.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:11:29 +05:30
2ad83f37fd fix(ui): dub play button + designer script resize on Windows (#595) (#607)
Two frontend bugs reported on v0.3.7, both Windows/Chromium-flavoured:

1. The PLAY button on the dubbed-video preview did nothing. WaveSurfer
   builds its AudioContext at mount (before any user gesture), so on
   Windows WebView2 / Linux FF/Chrome it stays "suspended" and
   playPause() resolves with no sound. This is the same autoplay-policy
   trap #510 fixed for WaveformPlayer, but the dub timeline player was
   missed. togglePlay and the per-segment playRange now await the shared
   unlockAudio() on the click before starting playback, and swallowed
   play() rejections are logged. A source-contract regression test pins
   the invariant (fail-before/pass-after verified).

2. The designer Script text field couldn't be expanded. It was a
   `flex: 1` item in a flex column, so flex-grow recomputed its height
   each reflow and snapped the resize-drag back — `resize: vertical` is
   ignored on a flex-grown item in Chromium/WebView2. The textarea now
   owns its height (flex: 0 1 auto + a taller min-height) so the corner
   grip grows it reliably on every platform.

Gates: `bun run build` and `bunx vitest run` (563 tests) both pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:10:49 +05:30
2767e2995d fix(tts): auto-repair incomplete model cache instead of dead-ending (#581) (#606)
An interrupted first download leaves the HF cache with config/tokenizer
files but no weight shard. transformers then raises an OSError ("does not
appear to have a file named pytorch_model.bin or model.safetensors") on
load, which model_manager translated into a 500 with a manual "delete the
model and install it again" instruction — a dead-end for the user.

Make the load path self-repair: on the truncated-cache OSError, re-fetch
just the missing files via snapshot_download (already-present blobs are
skipped, so a near-complete cache repairs fast and a healthy cache never
reaches this branch), then retry the load once. HF offline mode is
respected, and the actionable delete-and-reinstall message is preserved
as the fallback when repair can't fix it.

Adds tests/test_model_cache_repair.py covering completeness detection,
the fast path (no repair on healthy cache), self-repair + retry, the
offline guard, and the repair-failure fallback.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:10:18 +05:30
7b70d82322 fix(dub): retry transient broken-pipe on URL download (#579, #598) (#605)
Pasting a video URL into the dubber could fail outright with
`download: Unable to download video: [Errno 32] Broken pipe`. A broken
pipe raised while the write side of a pipe closes mid-stream (a killed
ffmpeg merge child, a CDN reset during muxing) aborts the whole
`extract_info` call and is NOT covered by yt-dlp's own per-fragment
retries, so a single transient blip killed the entire ingest.

Root cause: no download-level retry around `yt_download_sync`'s
`extract_info`. The failure was already classified as
`VIDEO_DOWNLOAD_NETWORK` (#554/#536) and carried a "just retry" hint, but
nothing actually retried.

Fix: wrap the download in a bounded retry (1 + 2 attempts) that retries
only on transient/broken-pipe-class failures, reusing the single
`failure.classify() == VIDEO_DOWNLOAD_NETWORK` taxonomy (plus the
BrokenPipeError/ConnectionError classes) rather than a parallel keyword
list. Partial `original.*` files are wiped between attempts so a
half-written download can't poison the next try. Unsupported links still
fail fast with their own hint (no wasted retries); after retries are
exhausted the existing actionable network hint is surfaced.

Adds tests/test_dub_download_retry.py: retryability classification +
retry-then-recover, bounded give-up, and no-retry-on-unsupported-URL.
Fails before (no retry loop / helper), passes after.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:09:51 +05:30
2549d9539e feat(support): Contact page, Ko-fi/PayPal donate, simpler license (#604)
GitHub Sponsors isn't available for this account, so route donations to Ko-fi /
PayPal instead, add a standalone Contact page, and trim the commercial-license
page to the essentials.

- Donate: drop GitHub Sponsors. Pick an amount ($10 / $20 / $50) then choose
  Ko-fi or PayPal; PayPal.me carries the amount into checkout. Updated
  .github/FUNDING.yml (ko_fi + custom PayPal) and the README badges to match.
- Contact page (new `mode: 'contact'`, ContactPage.jsx): Discord, email, GitHub
  issues, and website (palash.dev) as clean one-tap rows; reachable from a new
  footer button. Routed in App.jsx, sidebar hidden like the other full pages.
- Commercial License: cut the 6-tile benefit grid + 3-item FAQ down to the
  three deciding factors (IP ownership, no per-minute cost, direct support) and
  one clear "request a quote" email CTA.
- All new copy goes through i18n (en.json: donate.choose_method*,
  enterprise.hero_simple/contact_lead, contact.*, logs.contact*).

Build (vite) + vitest (561 passed) green; en.json validated.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:09:29 +05:30
6e3e4bcfdd fix(bootstrap): gate venv on omnivoice import + source fallback (#564) (#603)
* fix(bootstrap): gate venv on omnivoice import + source fallback (#564)

`No module named 'omnivoice'` is a venv that starts uvicorn but can't import the
project's OWN package: an interrupted/offline `uv sync` installed deps yet never
laid the editable record (`_editable_impl_omnivoice.pth`), or antivirus removed
it. The bootstrap health gate only checked `import uvicorn` + `import
pkg_resources`, so it handed back the broken venv and the app failed only at the
first model call (the dub/generate SSE error in #564). #573's source fallback in
main.py wasn't enough on its own because the editable record, not the source
tree, was the missing piece.

Fix the root cause at the gate and harden the runtime:
- bootstrap.rs: add an `omnivoice` import check beside the uvicorn/pkg_resources
  gates, using `importlib.util.find_spec` (resolves without importing, so no
  torch load). When it fails, fall through to the repair `uv sync`, which
  re-lays the editable install. Mirrors the #248 pkg_resources pattern exactly.
- core/omnivoice_path.py (new): `ensure_omnivoice_importable()` — a tested
  helper that no-ops when the install resolves and otherwise appends the sibling
  source root to sys.path, with a precise diagnostic when neither is found.
- main.py: replace the inline #573 block with the helper.
- model_manager._lazy_omnivoice: self-heal on ModuleNotFoundError at the actual
  import site so the model-load path recovers and logs the searched roots.

Regression tests cover the path-resolution logic (env override, append-not-
insert precedence, no-source-found). cargo check passes for the Rust change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(omnivoice-path): patch via live module object to survive core reloads (#564)

The #603 CI flake: other suites importlib.reload(core.*), leaving the
top-level-imported ensure_omnivoice_importable closed over a stale module whose
_already_importable a string-form monkeypatch didn't touch, so it returned None.
Resolve the function + the patch target from sys.modules together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 05:07:07 +05:30
c965a7fbdd fix(backend): self-healing GPU pool so a reset can't strand requests (#589/#599) (#601)
`_reset_gpu_pool()` fires on a model-load timeout to recover a wedged worker —
it shut the ThreadPoolExecutor down and rebuilt a fresh one on next access. But
several request handlers (generation, dub_generate, dub_core, dub_translate,
openai_compat) did a *module-level* `from services.model_manager import
_gpu_pool`, capturing the executor object at import time. After a reset those
references pointed at the dead pool, so the next generate/dub/transcribe/
translate raised `RuntimeError: cannot schedule new futures after shutdown` —
surfacing as a 500 or "Can't reach the local backend" (#589 #599).

Make `_gpu_pool` a single long-lived `_ResilientGpuPool` wrapper (a
concurrent.futures.Executor) whose *inner* ThreadPoolExecutor is swapped:
- every submit() resolves the live pool, and a submit that races a shutdown
  rebuilds once and retries, so a stale captured reference self-heals;
- `_reset_gpu_pool()` now drops only the inner pool (fresh worker on retry)
  while preserving the wrapper identity every importer holds;
- pool sizing stays lazy, so we still probe the device after torch's lazy
  import (the reason for the original __getattr__ indirection).

Fixes the whole class — all importers share one wrapper, module-level or
function-level. Regression tests cover stale-ref-survives-reset, identity
stability, submit-after-inner-shutdown self-heal, and asyncio.run_in_executor
compatibility; updated the load-timeout test to the new reset semantics.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 04:42:16 +05:30
8f2c4bbc5c fix(tts): NFC-normalize text + dense-script-aware chunking for long-form quality (#502/#505) (#587)
Two defensive fixes for non-Latin / long-form synthesis quality:

#502 (Vietnamese clone distorted/unintelligible): the /generate text path never
NFC-normalized its input, so pasted decomposed (NFD) Vietnamese — base letter +
combining diacritic instead of the single composed codepoint — reached the
tokenizer/model as two characters and rendered as garbled speech. Normalize the
input text to NFC at the endpoint (no-op for already-composed text), mirroring
what the duration estimator already does so the estimate and synthesis agree.

#505 (long-form 5+ min degrades — repeated/skipped/mispronounced): the chunker
split purely by character count (800), but CJK/kana/Hangul pack ~1 char =
1 syllable, so an 800-char chunk is ~4-5 minutes of audio in a single shot —
past the model's reliable range, where it starts repeating/skipping. When a
chunk is predominantly dense-script, cap it to max_chars/2.5 so each chunk's
spoken length stays bounded; Latin/spaced text is unchanged. Dense-script
detection is by code point (no literal CJK in source — no-literal-CJK gate stays
clean).

Tests: _dense_char_count, _effective_max_chars (shrink-when-dense, unchanged-for-
Latin, disabled-passthrough, floor), and that a 400-CJK-char string now splits
(was one chunk) while a Latin paragraph still doesn't.

Note: #502's exact distortion still wants a user sample to fully confirm; this is
the defensive NFC fix that's correct regardless.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 03:18:15 +05:30
b8e219e7d1 fix(gpu): prevent the 8 GB-card OOM crash behind the "backend unreachable" wave (#567/#570/#571/#580+) (#586)
The wave of "Can't reach the local backend" reports — all on ~8 GB NVIDIA cards,
all during generate bursts — is the backend *process* dying, not a transport
blip. Root cause: the GPU pool was sized at 2.5 GB/job, so an 8 GB card (~7 GB
free) got 2 workers. The interactive clone path co-loads WhisperX large-v3 ASR
(~3 GB) alongside TTS (~1.6 GB), so two concurrent clone jobs is ~10 GB on an
8 GB card → a sticky CUDA "illegal memory access" that aborts the whole
interpreter (uncatchable by the per-request OOM guard, which only re-raises a
clean torch.cuda.OutOfMemoryError as HTTP 500).

Budget 5 GB/job (the real TTS+ASR concurrent footprint) instead of 2.5 GB:
≤10 GB cards now serialize to a single GPU worker — no concurrent-kernel
contention, so the crash can't happen — while 16/24 GB cards still parallelize.
Overridable via OMNIVOICE_GPU_WORKERS. This *prevents* the crash; the
auto-restart supervisor (#572) *recovers* from any other cause — defense in
depth.

Extracted `_workers_for_free_vram()` (pure) with tests pinning 8 GB → 1 worker,
the larger-card ladder, the floor/cap, and a guard on the budget constant so a
regression toward 2.5 GB can't silently re-enable the crash.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:51:45 +05:30
7393ae80f5 feat(design): seed pin / re-roll for designed voices (#526) (#577)
Voice design rolled a brand-new random seed on every synth, so tweaking an
attribute also re-rolled the whole base timbre — you could never iterate on the
"same voice, slightly different". #526 asks for the seed to be shown with a
"keep this seed" control.

- Backend: `/generate` already accepted `seed` and echoed `X-Seed`, but left
  `used_seed=None` when nothing supplied one (non-deterministic, unreproducible,
  empty X-Seed). Now it materializes a concrete random seed when none resolves,
  so every take is reproducible and the real seed is always returned and stored
  — this also helps the clone/profile paths, not just design.
- Frontend: new store slice (`designSeed`, `keepSeed`); the design synth reuses
  the pinned seed when "keep this seed" is on (via `pickDesignSeed`) and reads
  the authoritative seed back from `X-Seed`. Design tab gains a Seed field +
  "keep this seed" checkbox + "New seed" (re-roll) button.

Test: `pickDesignSeed` (pin when kept+valid, re-roll otherwise, range guard).
i18n keys added to en.json (other locales fall back; parity probe is advisory).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 04:39:22 +05:30
21b0b1f0b2 fix(dub): auto-assign per-speaker cloned voices to segments (#486) (#576)
Multi-speaker dubbing diarizes the speakers and clones each one from the video
(the Voice dropdown shows "From Video → Speaker 1 / Speaker 2"), but every
segment was left on "Default" — the user had to set the voice on each row by
hand. The clone→segment binding simply never happened: the transcribe `final`
handler stored `speaker_clones` but set the segments without filling their
`profile_id`.

Bind them up front: new `applySpeakerCloneDefaults(segments, speakerClones)`
sets each segment's `profile_id` to its speaker's `auto:<safe>` clone id when a
clone exists and the user hasn't already chosen a voice. The id is computed by
`autoProfileId()`, which mirrors the backend clone-resolution key
(`speaker_id.lower().replace(" ","_")`) and the DubTab dropdown option value, so
all three agree. Only an *empty* profile_id is filled — an explicit per-speaker
or per-segment choice is never clobbered.

Pure helper + unit test (assign-when-cloned, never-clobber, no-clone-stays-
Default, no-op-without-clones).

Note: the issue's second symptom — different speakers' turns merged onto one
line — is a separate diarization/segment-grouping concern (speaker-turn
re-split) tracked as a follow-up; this fixes the per-speaker voice assignment.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 21:16:18 +05:30
0e17caa52a fix(install): actionable torch-wheel-download failure + local-wheel recovery (#569) (#574)
#569: on a restricted network the first-run install fails downloading the
~2.5 GB cu128 PyTorch wheel from download.pytorch.org, and the app won't launch.
Two problems: the error told users to "set UV_DEFAULT_INDEX to a mirror" — which
CANNOT redirect torch, because it comes from a *named, explicit* uv index
(uv 0.11 rejects index-name override values and `--frozen` pins the exact wheel
URLs); and there was no way to supply a manually-downloaded wheel.

- Detect a torch/pytorch-host `uv sync` failure and emit torch-specific guidance
  (Clean & Retry → VPN → drop the wheel locally) instead of the wrong mirror
  advice.
- Add a local wheel-drop dir `<env_root>/wheels` (survives Clean & Retry) wired
  via `UV_FIND_LINKS`. On a frozen-sync torch-download failure WITH wheels
  present, retry NON-frozen with find-links so uv re-resolves from the local
  wheels. Verified empirically: a non-frozen find-links sync installs from a
  local wheel fully offline, while a `--frozen` sync ignores find-links — so the
  retry is the only mechanism that can consume a dropped wheel. Best-effort: if
  it can't satisfy, it fails identically to before and the actionable error
  still fires.
- docs/install/troubleshooting.md: new "#12 CUDA PyTorch wheel download fails"
  entry (docs-sync) — the offline wheel-drop path + why a PyPI mirror can't fix
  this index.

Note: an automatic mirror redirect for the cu128 index is intentionally NOT
shipped — uv provides no working override for a named explicit index, so it
couldn't be verified; the offline wheel path is the reliable escape hatch.

Test: sync_failure_is_torch_download host/keyword detection + negative guard.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 21:04:31 +05:30
ff168048be fix(backend): import omnivoice from source when the editable install is missing (#564) (#573)
#564 ("No module named 'omnivoice'") is the backend failing to import its OWN
package at the first model call (the dub SSE error on dub:upload). `omnivoice`
is an editable install, so an interrupted/offline `uv sync` that installed deps
but never laid the editable record, an antivirus-quarantined
`_editable_impl_omnivoice.pth`, or an upgrade where only the lock-gated drift
sync ran leaves the venv able to start uvicorn yet unable to import omnivoice —
it boots fine and only fails at runtime, so the bootstrap health gate and the
exit-based broken-venv self-heal (which only see a process that won't start)
never catch it.

Fix the whole class at the import layer: main.py now also appends the project
root (the parent of backend/, where the desktop layout always copies
omnivoice/) to sys.path, guarded on omnivoice/__init__.py existing. The backend
then resolves omnivoice from source regardless of the editable-install state —
covering every variant above. Appended (not inserted) so a real
site-packages/editable install keeps precedence and it can't shadow a different
omnivoice; a no-op in Docker (no sibling omnivoice/) and a harmless duplicate
in a dev checkout.

Also routes "No module named 'omnivoice'" through failure.classify() →
BROKEN_VENV so, if it ever still surfaces, the toast points at Clean & Retry
instead of a bare import error. Regression test covers the classify mapping and
its negative guard (a legitimately-named omnivoice_* helper must not match).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 20:46:26 +05:30
e14644f77a fix(backend): auto-restart supervisor + client transport-retry (#567/#570/#571) (#572)
The "Can't reach the local OmniVoice backend" cluster was a long-standing
supervision gap (dates to v0.3.0/#38), not a v0.3.7 regression: the backend
was spawned once and never watched again — `spawn_backend_and_wait` returned
the instant it was healthy. When the uvicorn process then died mid-session (a
CUDA OOM/context fault under a burst of generations — #571's log shows the
startup banner replaying 6× during a 20-generate burst — an antivirus kill, any
crash), nothing restarted it, so every later request threw connection-refused
and the user was stuck on the toast until a full app restart.

Two layers, both default-mode and platform-neutral:

1. Backend auto-restart supervisor (bootstrap.rs). After Ready, the bootstrap
   thread (which used to just return) keeps watching the child and respawns it
   on a *confirmed process exit* (try_wait — never a slow health probe, so a
   busy-but-alive backend is never killed). Bounded to 5 restarts/60s (then
   Failed) so a deterministic startup crash can't fork-bomb; the #314
   broken-venv self-heal stays the venv-failure path. Strictly gated on
   AppFlags.quitting so it never resurrects the backend during shutdown. A
   single-supervisor guard (compare_exchange) prevents duplicate loops when
   Retry re-enters concurrently. Emits backend-restarting/backend-restored
   events (the splash poll stops post-Ready, so the stage alone can't show it).

2. Client transport-retry (client.ts). A *thrown* fetch (the backend briefly
   down while it respawns) is retried a bounded few times with backoff
   (~2.9s total) before surfacing the actionable ApiError, making the restart
   window invisible. HTTP errors and deliberate aborts are never retried.

Resolves the whole cluster regardless of the crash trigger. Tests: Rust
backoff-policy unit test (cap + window-pruning); 4 client-retry vitest cases
(retry-then-succeed, no-retry-on-HTTP-error, no-retry-on-abort, bounded
give-up). Also corrects a stale Cargo.lock omnivoice-studio version (0.3.6→0.3.8).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 20:32:59 +05:30
b0c93a598e docs(changelog): complete the v0.3.7 section (items that landed but were under-listed) (#568)
The v0.3.7 notes were missing several user-facing changes that shipped between
v0.3.6 and the tag: Stories global reading-speed (#508), the Settings sparse-tab
fill + Appearance i18n (#507), the donate progress correction (#513), and a
### Changed (version single-source #503, preview-nightly #500) + ### Internal
(frozen-backend version #501) section. Restructured to the 0.3.6 house style.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 16:56:40 +05:30
github-actions[bot] a22f029c2c chore(version): main -> 0.3.8 after v0.3.7 release 2026-06-20 09:26:01 +00:00
8e6344ee4e docs(changelog): add the non-English language fixes to v0.3.7 (#533/#505/#502) (#566)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:55:21 +05:30
8ed76b40a6 fix(lang): propagate profile/request language into generation + longform (#533/#505/#502) (#565)
Non-English voices drifted to English/wrong-language because the request's or
profile's language wasn't reaching the model:

- #533: generate_speech() read instruct/ref_text/seed from a resolved profile row
  but never row['language'] (and collapsed Auto→None), so a German archetype
  previewed in German yet generated in English on the user's own call (and via
  Docker/API). Fall back to the profile's stored language when the request didn't
  pin one; an explicit non-Auto request language still wins. (Frontend already
  sets the dropdown on profile-select; this is the authoritative backend fix.)
- #505 (B2): the audiobook/longform synth hardcoded language=None, so the engine
  re-autodetected per chunk and a non-English clone flipped language mid-render.
  Add _resolve_default_language (request → profile → autodetect) and thread the
  resolved language through _build_synth/_prepare_synth/_render_longform_sse, the
  three longform request models, the preview path, and the resume manifest.
  Genuine Auto/unset behavior is unchanged.
- #502 (partial): the duration estimator weights combining marks (U+0300–036F)
  at 0.0, so NFD/decomposed text under-allocated frames → rushed audio. NFC-
  normalize text at the estimator entry — fixes the whole diacritic-script class
  (no-op for precomposed text). (The residual "distorted" core still needs the
  reporter's sample; tracked separately.)

Tests (fail-before/pass-after): profile language reaches the engine (German→de;
explicit/Auto override semantics); longform synth gets the resolved language
(→ja), not None; NFD vs NFC duration parity (Korean Hangul diverges ~3x pre-fix).
Full suite: tests/ 1740 passed, backend/tests/ 114 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:45:58 +05:30
ad78a418bc docs(changelog): v0.3.7 release notes (#563)
The stabilization release: tags the startup-crash fixes already on main + the
wave of 0.3.6-line fixes (voice design [object Object], consent_audio_path schema
self-heal, Linux WebKitGTK buttons, pip install, ASR float16 fallback, audiobook
import, Windows auto-play, download errors, relocated-venv self-heal, About
version), and folds in the MOSS-TTS/dots.tts engines (#498/#531) + the
Linux/Android audio-playback fix (#510) that landed on main.

release.yml extracts this section verbatim as the v0.3.7 GitHub release body.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:49:55 +05:30
b8cc0de44a fix: actionable download errors + self-heal a relocated venv (#554/#536/encodings) (#562)
Two robustness fixes that share the failure taxonomy:

- Video downloads (#554 douyin "Unsupported URL", #536 "Broken pipe"): yt-dlp's
  raw error surfaced with no next step. classify() now names UNSUPPORTED_VIDEO_URL
  (non-downloadable link shape — paste a direct video page or drop a file) and
  VIDEO_DOWNLOAD_NETWORK (transient CDN/network drop — just retry; the partial
  download is already cleaned up), each with an actionable hint. yt-dlp is on a
  current pin, so this is graceful classification, not a dependency bump.

- "No module named 'encodings'" (relocated/copied/restored venv whose interpreter
  can't bootstrap its stdlib — exit 1, not 106): slipped BOTH #314 self-heal
  matchers, so the user saw the error forever. Widen
  backend_exit_indicates_broken_venv to also match the full quoted phrase, routing
  it into the existing rebuild-once self-heal. Kept narrow so an app-level import
  of an 'encodings'-prefixed package can't trigger a rebuild. Plus a BROKEN_VENV
  hint for the case where the rebuild itself can't run.

Tests: classify() maps the 3 new classes with hints (a generic reason still ""),
and the 'encodings'-prefixed-package negative guard holds; the Rust matcher test
gains the encodings positive + negative cases. 5 pytest passed; the matcher is
compiled by CI's Tauri shell check.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:46:30 +05:30
a7ab148483 fix(asr): float16-unsupported GPUs fall back to int8 instead of "no segments" (#561)
#551: both CTranslate2 ASR backends request compute_type="float16" on CUDA with
NO fallback. On GPUs without efficient fp16 (older Maxwell/Pascal, GTX 16xx) or a
CTranslate2/cuDNN binary mismatch, WhisperModel/whisperx.load_model raise a
ValueError at construction — which escaped the existing OOM-only `except
RuntimeError`, so every chunk failed and the user got "Transcription produced no
segments". Add a per-device compute_type fallback chain (cuda: float16 →
int8_float16 → int8; cpu: int8 → float32) to both backends + the ASR sidecar,
alongside (not replacing) the existing OOM→CPU path, with an ASR_COMPUTE_TYPE
override for exotic hardware (documented in README).

Also in the same ASR-robustness pass:
- #549: PyTorchWhisperBackend._ensure_pipe wraps the transformers pipeline load
  and re-raises an actionable error (reinstall transformers / use faster-whisper)
  instead of a bare "Could not import module 'AutoFeatureExtractor'".
- #516: the /dub/transcribe SSE generator is wrapped so it can NEVER close
  without a terminal event — any unanticipated exception now yields a structured
  `error` (with build_failure's hint) + `done`, turning "stream dropped, likely
  ASR failed" into the real cause + Retry.
- failure.py: COMPUTE_TYPE_UNSUPPORTED + TRANSFORMERS_IMPORT classes so the
  no-segments toast is actionable.

Tests (fail-before/pass-after): float16-unsupported → int8 for both WhisperX +
FasterWhisper; a generic non-OOM RuntimeError still raises; classify() maps the
two new classes; the SSE stream always terminates with error→done. 7 + 1 passed,
17 in the failure suite (no regression).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:34:06 +05:30
51ad469ef2 fix(projects): preview finished renders in-app, not via window.open (#532) (#538)
The audiobook/story library card called window.open(audioUrl, '_blank').
Under Tauri's WebView2 on Windows that handed the file to a new webview/OS
media surface, spawning a separate black playback window with centered
controls that couldn't be closed without force-quitting the whole app.

Route the render through the shared single-playback manager (playBlobAudio)
so it previews inside the app — identical behavior on macOS/Windows/Linux,
and starting another preview stops this one. This is the only raw
window.open on a media URL in the frontend, so it fixes the whole class.

Adds a regression test: the card plays in-app and never calls window.open.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:15:44 +05:30
7678d4cca3 fix(audiobook): add AudiobookPlan.chapter_count so import doesn't 500 (#544)
POST /audiobook/import ended with `plan.chapter_count`, but AudiobookPlan
only exposed `char_count` (a property) and emitted `chapter_count` from
`to_dict()` — so the attribute access raised AttributeError, surfacing as
"500 Internal Server Error: 'AudiobookPlan' object has no attribute
'chapter_count'". The parse itself succeeded, so this hit every import
format (.txt/.md/.epub/.pdf), not just PDF.

Add a `chapter_count` property mirroring `char_count`, and have `to_dict()`
derive its key from it so the attribute and serialized key can't drift.
No API/schema/data change.

Tests: unit property test + a direct-handler /audiobook/import regression
(pdf/md/txt) that fails-before with the AttributeError.

Fixes #543

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 09:15:40 +05:30
6dccf0390d fix(settings): About → Version never blank (web/Pinokio build) (#560)
In the web/Pinokio (non-Tauri) build with the backend idle, Settings → About
rendered an EMPTY Version cell. Both sources were unavailable: appVersion is set
only inside the isTauri()-gated effect, and info.app_version comes from
/system/info which never resolves while the backend is IDLE (retry: Infinity).
So `appVersion || info?.app_version || '—'` produced nothing.

Add resolveAboutVersion() in a small util that falls back to the build-time
__APP_VERSION__ (Vite-injected from package.json — always present regardless of
Tauri/backend), and use it at BOTH the About → Version row AND the diagnostics-
copy block (which had no fallback at all — the whole-class fix). Tauri/live-
backend sources still take precedence, so the packaged build is unchanged.

Test: resolveAboutVersion prefers Tauri → backend → build constant, and is never
blank/dash. vitest 2 passed; typecheck clean.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:15:36 +05:30
3f23a77eab fix(engines): pin uv pip install to the running interpreter (#529/#527) (#559)
The engine Install chip (deep_translator / openai / argostranslate) 500'd with
"No virtual environment found". run_pip shells out to bare `uv pip install`,
which discovers its target venv from VIRTUAL_ENV / a .venv in CWD — NOT from the
running interpreter. The desktop spawns `<venv>/bin/python -m uvicorn` without
exporting VIRTUAL_ENV and CWDs outside the venv, so uv finds nothing. The
existing `--system` fallback never fires because it's gated on _in_virtualenv()
being False, but the running interpreter genuinely IS in a venv (it just can't be
auto-discovered) — the heuristic answers the wrong question.

Pass `--python sys.executable` for uv install/uninstall: targets the same
interpreter _probe()/is_installed() import from, fixing the whole class
(spawned-venv, system, conda). It takes precedence when both flags are present,
so the Docker `--system` path is untouched. (#527's openai is already bundled by
#484; this hardens the chip for the remaining runtime-installed engines.)

Test: run_pip's spawned argv contains `--python <sys.executable>` after
install/uninstall. 2 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:11:20 +05:30
cb8ff16d86 fix(ui): fill the shell on WebKitGTK so Generate/Settings buttons aren't clipped (#558)
#523 "voice synthesis not pressable" / #524 "Settings button not visible" — and
the Discord "where is the Generate button / the clone button disappeared" reports
on 0.3.6/0.3.7 with the backend UP. The #504 zoom fix only half-worked: it sizes
.app-container as calc(100vw/scale) magnified back by `zoom`, which round-trips to
the viewport on Chromium — but older WebKitGTK (Linux AppImage/deb) treats `zoom`
as a LAYOUT NO-OP, so the box stays shrunk to 77vw, leaving a ~23% black band and
pushing the bottom-pinned Synthesize/Generate CTA + the NavRail Settings footer
off-screen. The old comment's "FILLS the window (no black bands)" claim was false.

No single static rule satisfies both engines (they disagree on whether `zoom`
lays out), so detect it at runtime:
- App.jsx: a one-shot probe measures a real `zoom:2` element; if its rect isn't
  magnified, the engine treats zoom as a no-op → set html[data-zoom-layout=off].
  Robust where @supports(zoom)/UA-sniffing aren't (both lie on WebKitGTK), and
  future-proof (flips back to the zoom path on engines that start honoring it).
- index.css: html[data-zoom-layout=off] .app-container renders at 1.0 filling
  100vw/100vh — no band, no clipped CTAs. Chromium keeps the scaled zoom path.
- appShellScale.test.js: guard BOTH branches (the existing calc+zoom path AND
  the 100vw/100vh fallback) so a future "simplification" can't re-break one engine.

vitest 4 passed; typecheck clean. NOTE: CI e2e is chromium-only, so the WebKitGTK
fallback path must be eyeballed on a real Linux build before the v0.3.7 tag.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:05:26 +05:30
915256cb8c fix(db): self-heal additive schema columns so consent_audio_path 500s stop (#557)
A profile/persona/consent endpoint 500'd with "no such column:
consent_audio_path" (#552/#547) — and the same class for kind/vd_states/is_demo.
The 0003/0005 migrations exist and are wired, but init_db's
CREATE TABLE IF NOT EXISTS never adds columns to a pre-existing table, the legacy
_migrate only knows pre-0.3 columns, and _run_alembic_upgrade swallows every
failure. So on a DB whose alembic_version is stamped at a revision no longer in
versions/ (common after running a preview build) or where alembic isn't
importable, the alembic-era columns silently never land.

Fix the whole class: add _reconcile_additive_columns(conn), which builds the
canonical schema from _BASE_SCHEMA in-memory and ALTER TABLE ADD COLUMN any
column an existing table is missing (additive only — never drops/retypes;
names/types from _BASE_SCHEMA so injection-safe). Call it in init_db (so the
schema converges regardless of alembic) and again in the alembic-failure branch.
Also correct the false "_BASE_SCHEMA guarantees the schema regardless" comment.

Test (fail-before/pass-after): init_db on a legacy voice_profiles whose
alembic_version is a removed revision now lands consent_audio_path/kind/etc.
without raising; reconcile converges to the canonical column set; idempotent +
additive-only. 21 passed (incl. existing 0003/0005 migration tests).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 09:00:47 +05:30
1ee4b3c2a0 fix(voice): stop poisoning design profiles with "[object Object]" instruct (#556)
Voice Studio "Save design as profile" passed buildDesignInstruct()'s RETURN
OBJECT ({instruct, unsupported, duplicates}) straight to FormData.append, which
string-coerced it to the literal "[object Object]". That got persisted into
voice_profiles.instruct and 400'd on first preview/use with "Unsupported instruct
items found in [object Object]" (#550 #545 #542 #537 #530 #525).

Fix the whole class + heal already-poisoned data (backward-compatible-data rule):
- CloneDesignTab.jsx:609 — pass `.instruct` (the string), not the builder object.
- useProfiles.js — append via new instructToFormValue() helper, which extracts
  `.instruct` if an object ever slips through again (defense-in-depth).
- omnivoice/_resolve_instruct — drop the "[object Object]" sentinel instead of
  raising, so any value that slips through (e.g. generation_history) degrades to
  neutral conditioning rather than a hard 400. A genuine unsupported token still
  raises (keeps the #114/#115 user feedback).
- migration 0006 — UPDATE voice_profiles SET instruct='' WHERE it's the sentinel,
  idempotent + table-guarded, to heal profiles saved on the buggy build.

Tests (fail-before/pass-after): frontend voiceInstruct (instructToFormValue never
yields "[object Object]"); backend _resolve_instruct tolerates the sentinel but
still rejects a real bad token; alembic 0006 heals a poisoned row, leaves a
healthy one untouched. Frontend 9 passed + typecheck clean; backend 5 passed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 08:50:42 +05:30
Palash Debnath 9a1d549686 Merge pull request #531 from debpalash/debpalash/feature-adding-moss-tts-v1.5-8b-model-as-an-alte
feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498)
2026-06-20 08:11:34 +05:30
mergetestandClaude Opus 4.8 3777d3a62c feat(tts): add MOSS-TTS-v1.5 (8B) and dots.tts (2B) as opt-in engines (#498)
Adds two zero-shot voice-cloning TTS engines requested in #498, both
opt-in and subprocess-isolated with their own dedicated venv — the same
pattern as IndexTTS-2. The dedicated venv is forced, not just chosen:
each upstream pins a transformers version that conflicts with the
parent's >=5.3 (MOSS-TTS-v1.5 ==5.0.0, dots.tts ==4.57.0), so they cannot
share the parent interpreter.

Because they use the clone+venv bootstrap (env var -> clone -> uv venv),
this touches no pyproject.toml / uv.lock / bun.lock — `uv sync
--all-extras` and Docker's `bun install --frozen-lockfile` are unchanged,
so main's CI/Docker matrix stays green.

Engines:
- moss-tts-v15: 8B, 31 langs, ~16 GB weights, 24 kHz. AutoModel/
  AutoProcessor via trust_remote_code. gpu_compat=(cuda,cpu) — MPS is
  undocumented/untested upstream so it is never claimed; on a Mac it runs
  on CPU. Apache-2.0, no license gate.
- dots-tts: 2B, 24 langs, ~9 GB weights, 48 kHz. DotsTtsRuntime;
  continuation cloning (prompt_audio_path+prompt_text). Upstream is
  Linux/macOS-only, so is_available() gates it off cleanly on Windows
  (cross-platform parity rule — it is opt-in, never a broken default).

Wiring: registered in _LAZY_REGISTRY + _INSTALL_HINTS. list_backends()
surfaces both as subprocess/[cuda,cpu]/available-until-installed; the
data-driven Settings engine picker needs no frontend change.

Tests (19, fail-before/pass-after): registry resolution, subprocess
marker, no-MPS gpu_compat, the Windows gate, not-installed honesty, and
the parent-side generate() kwarg arbitration. Existing engine suite still
55 passed / 5 skipped. Sidecar inference follows the upstream-documented
APIs but, like IndexTTS/Supertonic, can't be executed in CI without the
multi-GB model clones.

Docs (same-PR per docs-sync rule): README + README_CN engine tables, new
docs/engines/moss-tts-v15.md + dots-tts.md, disk-usage.md (torch-dedup
note), CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 00:02:27 +05:30
Palash Debnath 80891d3239 Merge pull request #519 from debpalash/fix/ui-scale-clipping-504
fix(ui): shrink layout box by uiScale so zoomed shell fits viewport (#504)
2026-06-17 17:27:48 +05:30
mergetest c0727c261d fix(ui): shrink layout box by uiScale so zoomed shell fits viewport (#504)
CSS `zoom` magnifies visually but does NOT enlarge the layout box. At the
default `uiScale=1.3` the shell was drawn at 130vw x 130vh while the window
stayed 100vw x 100vh, so the bottom/right edges were clipped by `overflow:
hidden` on `#root` and `.app-container`. This hid Settings save, Dub transcribe,
and Clone buttons (only reachable via Tab).

Fix: shrink the layout box with `calc(100vw / var(--ui-scale))` /
`calc(100vh / var(--ui-scale))`, then let `zoom` magnify it back to exactly the
viewport on Chromium. On WebKitGTK `zoom` is a no-op, so the UI renders smaller
but still fills the window - no black-band regression.

Updates the appShellScale regression test to match the new intentional
`calc(... / --ui-scale)` rule and keeps the `transform: scale()` blacklist.
2026-06-17 17:27:21 +05:30
Palash Debnath 2eac93c645 Merge pull request #518 from debpalash/docs/readme-reorder-header
docs(readme): nav links above badges, download badges to Quickstart
2026-06-17 16:55:12 +05:30
mergetest 4ec0ece457 docs(readme): move nav links above badges, download badges to Quickstart
- Nav links (Quickstart, Features, Why OVS, etc.) now sit above the
  badge row in the header for better scannability
- Download badges (macOS/Windows/Linux/Debian) moved from the header
  to the Quickstart section where the user actually installs
2026-06-17 16:54:47 +05:30
Palash Debnath 50fa5c206f Merge pull request #509 from debpalash/discord/fix-508
fix(stories): apply global reading speed to preview + stem export
2026-06-17 16:54:01 +05:30
Palash Debnath ef93835e22 Merge pull request #517 from debpalash/docs/readme-shorten-why-heading
docs(readme): shorten 'Why OmniVoice Studio?' to 'Why OVS?'
2026-06-17 16:04:31 +05:30
mergetest 25a471fe59 docs(readme): shorten 'Why OmniVoice Studio?' to 'Why OVS?' 2026-06-17 16:03:10 +05:30
Palash Debnath e02b7941a8 Merge pull request #513 from debpalash/fix/donation-progress-10-of-200
fix(donate): correct progress from $137.50 to actual $10/$200 raised
2026-06-17 12:23:23 +05:30
mergetest ad443e853a fix(donate): correct progress from $137.50 to actual $10/$200 raised
The in-app goal bar and its bundled snapshot showed $137.50 / $200 (23
sponsors) — fabricated numbers. Updated both the runtime JSON and the
TypeScript fallback to reflect the real amount: $10 / $200, 1 sponsor.

Also updates the README badge color from red to yellow (in-progress).
2026-06-17 12:23:03 +05:30
Palash Debnath 1fa2dda7c8 Merge pull request #511 from paoloantinori/fix/audio-mime-and-audiocontext-unlock
fix(web): audio playback on Linux/Android (MIME + AudioContext unlock)
2026-06-17 12:17:31 +05:30
Palash Debnath abb5f00d81 Merge pull request #512 from debpalash/docs/readme-update-donate-cta-and-features
docs(readme): add donate CTA, update features/engines/roadmap for v0.3.6
2026-06-17 12:17:29 +05:30
mergetest 69b8c7c0bc docs(readme): add donate CTA, update features/engines/roadmap for v0.3.6
- Add Sponsor/Donate section with Ko-fi, PayPal, GitHub Sponsors
  links and progress bar ($10/$200 agent bill fund)
- Add Ko-fi + GitHub Sponsors badges to header
- Expand features table: Audiobook, Stories, Diagnostics,
  Engine Routing, Portable Personas, Unlimited TTS,
  Remote Backend, Dictation+LLM (3x4 → 5x4)
- Update TTS engines: 6 → 11 (add GPT-SoVITS, Sherpa-ONNX,
  IndexTTS 2, OmniVoice GGUF, Supertonic 3)
- Update ASR engines: 7 → 8 (add isolated Faster-Whisper)
- Expand roadmap Shipped with v0.3.6 additions (Longform,
  engine routing, diagnostics, MCP server, remote backend,
  reliability, etc.)
- Update architecture diagram (100+ endpoints, engine routing)
- Add comparison table rows: Audiobook/Stories, TTS count,
  ASR count, MCP Server, Self-check
- Update FAQ engine count to 11
- Fix CTranslate2 link typo
- Remove already-shipped Audiobook Editor from Up Next
2026-06-17 12:16:27 +05:30
Paolo Antinori f4e7a76886 docs(changelog): note MIME + AudioContext fixes for Linux/Android playback 2026-06-17 07:25:37 +02:00
Paolo Antinori 74d63f099c test(audioUnlock): unit-test the gesture-driven AudioContext resume
Locks down the resume path from the parent commit:

- Every AudioContext is tracked at construction (the wrap is active)
- unlockAudio() resumes all suspended tracked contexts in parallel
- Idempotent: repeated calls do not re-resume
- Contexts created after unlock are not re-resumed by a second call
- resume() rejections are swallowed — one bad context doesn't block others
- installAudioUnlock() is idempotent (the _installed gate works)

The unlock path was the fix for the "click does nothing" bug on Linux
Firefox/Chrome and Android Chrome, where AudioContexts created before a
user gesture stay suspended — decodeAudioData hangs → WaveSurfer's ready
never fires → the play button never enables. Without this test, breaking
the gesture wiring silently regresses every non-macOS browser.

Adds the __resetForTesting() export so the unlock can be exercised
repeatedly against the same module instance (the unlock is meant to be
a one-shot per page load).
2026-06-17 06:48:43 +02:00
Paolo AntinoriandClaude Opus 4.7 f448c1c73c fix(web): resume AudioContext on first gesture so play button works on Linux/Android (#fix)
Browser autoplay policy (Linux Firefox/Chrome, Android Chrome, mobile
Safari): AudioContexts created before a user gesture start in "suspended"
state — decodeAudioData hangs and WaveSurfer's `ready` event never fires.
The play button gated on `ready` stays disabled forever, so the click
silently does nothing and no /audio/ request ever fires.

macOS Safari/Chrome are more lenient (typically auto-resume on first
interaction) which masked the bug cross-platform.

Fix has three parts:

1. `frontend/src/utils/audioUnlock.js` (new) — monkey-patches
   `window.AudioContext` (and `webkitAudioContext`) to track every
   instance ever created. Exports `installAudioUnlock()` which wires a
   one-time pointerdown/keydown/touchstart listener that resumes all
   suspended contexts on the first user gesture. The patch MUST install
   before any module constructs an AudioContext, so this file is imported
   first in main.jsx before the dynamic import of main-app.jsx.

2. `frontend/src/main.jsx` — imports and installs the unlock before any
   other module loads.

3. `frontend/src/components/WaveformPlayer.jsx` — three changes:
   - Remove the `Loader` spinner that gated on `ready`. The spinner
     itself was a visual signal that the user was waiting on a state
     the browser refuses to produce without user interaction.
   - Button is now `disabled={!resolvedUrl}` — clickable as soon as the
     audio URL exists, so the user's click IS the gesture that unlocks
     the AudioContext.
   - `togglePlay()` explicitly awaits `unlockAudio()` before calling
     `playPause()` to close any race with the global gesture listener.

The console warning "An AudioContext was prevented from starting
automatically" may still appear once on page load — that's the
informational signal that the pre-gesture context was created suspended;
it's harmless because we explicitly resume on first interaction.

Tested: Linux Firefox 151.0.3 — before fix, clicking play did nothing
(no /audio/ request fired, button never enabled). After fix, single
click on play resumes AudioContext + starts playback, waveform animates.

Pairs with the audio/wav MIME fix in the same PR — both bugs had the
same user-visible symptom (silent play button on Linux) but different
root causes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-17 06:33:21 +02:00
mergetestandClaude Opus 4.8 35e7dbc11c fix(stories): apply global reading speed to preview + stem export (#508)
The #415 global speed only flowed through the full longform export
(storyToSpans). Per-segment preview and stem export resolved speed with a
hardcoded `track.speed || 1.0`, silently dropping the global → generated
audio played at 1.0x even with the global set to e.g. 0.70x.

Add a pure `effectiveSpeed(track, globalSpeed)` helper (mirrors
effectiveProfile / storyToSpans precedence: per-line override → global →
engine default) and use it at both call sites so all three generation
paths agree. Regression-tested in storyCast.test.js.

Fixes #508

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 00:13:57 +05:30
05369e748a fix(settings): fill sparse tabs + route Appearance strings through i18n (#507)
* fix(settings): fill the panel on short tabs instead of a stunted box in a void

Settings tabs with little content (Appearance — UI scale/theme/font; About;
Privacy) rendered the accent-bordered .settings-content as a short box with the
rest of the page as empty black void below it (reported on Appearance).

Make .settings-page a flex column with min-height:100% (a FLOOR — tall tabs grow
past it and scroll exactly as before) and let .settings-content flex:1 1 auto
grow to fill. Safe by design: it only adds space when there's slack, and if the
parent height is ever indeterminate the rule no-ops rather than constraining
content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(settings): route AppearancePanel strings through i18n

The Appearance panel hardcoded English ("UI scale", "Color theme", "Font", and
the help paragraph) — against the localization hard rule. Wire them through
t('settings.*', { defaultValue }) matching the ApiKeysPanel pattern; reuse the
existing settings.appearance/ui_scale/theme keys and add color_theme/font/
appearance_help to en.json (the reference superset — other locales fall back to
English and backfill via the translation pipeline; the i18n parity gate only
requires valid JSON). Also renamed the THEMES.map(t =>) variable to `th` so it
no longer shadows the translation `t`.

typecheck + Appearance vitest (3) + i18n parity probe (4) all pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:53:49 +05:30
ade951688e fix(test): resolve package.json version reference in the desktop probe spec (#506)
#503 made tauri.conf.json derive its version from package.json
("version": "../package.json"), but the L3 desktop probe spec
(desktop_smoke.probe.yaml) asserts config.version == pyproject_version and was
reading the literal path string "../package.json" — reddening main.

Resolve the package.json reference in load_tauri_config() the way Tauri does, so
the integrity check sees the effective bundle version. The check now validates
the *real* thing end-to-end: the resolved desktop bundle version matches the
project version (and implicitly that package.json == pyproject).

Full suite: 1691 passed, 20 skipped, 11 xfailed, 3 xpassed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:08:57 +05:30
5a28c04c44 chore(version): make frontend/package.json the single source of truth (#503)
Five hand-maintained version literals (pyproject, Cargo.toml, tauri.conf.json,
package.json, version.py) drifting is what shipped a 0.3.6 build calling itself
0.3.5 (package.json lagged; the frozen backend's literal lagged). Collapse to
one canonical source.

- frontend/package.json is canonical: vite already injects __APP_VERSION__ from
  it (first-run setup footer + bug reports).
- tauri.conf.json now reads its bundle version from it ("version":
  "../package.json", a supported Tauri v2 feature) — the MSI/dmg/updater version
  can no longer drift from the UI. Removes the most error-prone literal.
- Cargo.toml + pyproject.toml + version.py's _FALLBACK_VERSION remain as
  toolchain-required CI-guarded mirrors, bumped in lockstep from the canonical.
- release.yml: the preview-stamp and version-bump jobs now read/write
  package.json (the canonical) and no longer touch the derived tauri.conf.json.
- tests/test_app_version.py: new test_tauri_version_derives_from_package_json
  guards the path; the lockstep test now checks the mirrors against the
  canonical package.json.
- CLAUDE.md versioning rule updated to document the single-source model.

6 version tests pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:50:39 +05:30
Paolo AntinoriandClaude Opus 4.7 8db5feccc7 fix(web): serve .wav/.flac with IANA-canonical MIME so Linux/Android browsers play inline (#fix)
Python's `mimetypes.guess_type()` returns `audio/x-wav` for `.wav` and
`audio/x-flac` for `.flac` — vendor-experimental types (x- prefix) that
were never IANA-registered. macOS Chrome/Safari MIME-sniff leniently via
CoreAudio so playback works there, but Linux Chrome/Firefox (FFmpeg) and
Android Chrome (ExoPlayer) strictly honor the declared type and treat
the x- variants as download-only.

Result: the play button in the browser web app silently did nothing on
Linux/Android (download prompt instead of inline playback), while the
Tauri desktop shell worked because its WebView is lenient. The
diagnostic signal — Chromium short-circuits to download BEFORE the
<audio> element sees the response, so no MEDIA_ERR_SRC_NOT_SUPPORTED
fires; just a download prompt that's easy to miss.

Fix: register `audio/wav` and `audio/flac` (the IANA-canonical types)
via `mimetypes.add_type()` before the StaticFiles mounts in main.py.
No browser-side workaround exists (no chrome://flags, no about:config
pref) — the server is the only place this can be fixed.

Existing comment in dub_export.py:766 already acknowledges this exact
quirk for video files; this applies the same treatment to audio.

Test: regression test in test_api.py asserts `/audio/<file>.wav` returns
`Content-Type: audio/wav`. Without the fix this returns `audio/x-wav`.

Ref: https://www.iana.org/assignments/media-types/media-types.xhtml#audio

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 18:42:18 +02:00
0337e15a2e fix(version): frozen backend reports real version, not the 0.3.5 fallback (#501)
The desktop app's About panel, /health, /system/info, diagnostics, bug reports,
and exported persona/marketplace bundle metadata all read core.version.APP_VERSION.
In a synced env that resolves from package metadata (correct), so CI and the
lockstep test were green — but the PyInstaller-frozen backend has no omnivoice
.dist-info, hits PackageNotFoundError, and fell back to a hardcoded
APP_VERSION = "0.3.5". The version-bump job never touched that literal, so every
0.3.x desktop build has been reporting 0.3.5 regardless of its real version.

Fix (belt and suspenders, so it can't recur):
- backend.spec: copy_metadata('omnivoice') so importlib.metadata resolves in the
  frozen build — the primary path now works there too.
- backend/core/version.py: resolution chain is metadata → pyproject (walked up,
  correct for raw source checkouts) → a named _FALLBACK_VERSION literal as last
  resort (no longer the only fallback).
- tests/test_app_version.py: _FALLBACK_VERSION joins the lockstep (now FIVE
  sources); + a test that the fallback resolves to pyproject, + a test that
  backend.spec copies the metadata (so the frozen path can't silently regress).
- release.yml version-bump: also bumps _FALLBACK_VERSION so the lockstep guard
  never reddens main after a release.

Already-shipped binaries can't be fixed, but every build from here (tonight's
preview, the next stable) reports its real version. 5 version tests pass.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:05:11 +05:30
e58106d552 fix(updater): preview channel builds nightly from main + prerelease/parity guards (#500)
The Preview update channel was effectively dead: its only build trigger was a
manual workflow_dispatch, so "preview = main" was never enforced — the live
preview manifest was stuck at 0.3.5-41 (June 7) while main moved to 0.3.7.
It also shipped two latent hazards: the `preview` GitHub release had drifted to
isPrerelease=false (a non-prerelease `preview` is eligible to become GitHub's
"Latest" — the exact URL the *stable* updater reads, so it could hijack the
Stable channel), and its updater manifest dropped darwin-x86_64 (Intel-Mac
preview users silently got no updates — a cross-platform-parity breach).

Changes (release.yml):
- Add a nightly `schedule` (07:00 UTC) that rebuilds the rolling `preview`
  prerelease from main. A new `preview-gate` job no-ops the 4-platform matrix
  on nights when main didn't move, so idle days cost only a ~30s gate job.
- Centralize the preview-vs-stable decision in `preview-gate.outputs.is_preview`
  (schedule OR workflow_dispatch+publish_preview), consumed by the stamp step,
  tauri-action, and preview-notes — replacing the repeated inline conditions.
- Harden the prerelease flag: preview-notes' `gh release edit` now re-asserts
  `--prerelease` every run, and a new post-publish step fails the run if the
  preview release isn't a prerelease or its manifest is missing any platform
  stable ships (catches the Intel-Mac regression in CI).

Docs (docs-sync): update docs/update-channels.md — previews are no longer
"manual / no scheduled spend"; they build nightly from main (+ on demand).

The live `preview` release was re-flagged prerelease out-of-band to close the
hazard immediately; this makes it recurrence-proof.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:04:27 +05:30
364611f7d7 docs(changelog): backfill high-quality v0.3.6 release notes + make it a hard rule (#499)
v0.3.6 was tagged with the "Auto-generated release for v0.3.6…" fallback body
because CHANGELOG.md had no matching section (release.yml's "Extract CHANGELOG
section for tag" step found nothing). Backfill a real, user-facing
`## [0.3.6] — 2026-06-16` section (Longform suite, engine routing, dubbing +
install reliability, FSL→AGPL relicense) and fold the shipped `.ovsvoice`
persona entry out of [Unreleased]. The live GH release body was updated
in-place to match (notes + the per-platform checksum blocks preserved).

Add a "Release notes / changelog" hard rule to CLAUDE.md: every tagged release
(and preview build) gets a high-quality, house-style CHANGELOG section before
the tag — never the auto-generated fallback — since release.yml ships that
section verbatim as the release body.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 19:04:22 +05:30
github-actions[bot] 4471bc0770 chore(version): main -> 0.3.7 after v0.3.6 release 2026-06-16 12:30:21 +00:00
812 changed files with 87663 additions and 30586 deletions
+6 -3
View File
@@ -1,6 +1,9 @@
# These are supported funding model platforms
# GitHub Sponsors isn't set up for this account — fund via Ko-fi or PayPal.
github: [debpalash]
# ko_fi: omnivoice
ko_fi: debpalash
custom:
- "https://paypal.me/palashCoder"
- "https://github.com/debpalash/OmniVoice-Studio/blob/main/SPONSORS.md"
# github: [debpalash] # not available
# open_collective: omnivoice-studio
# custom: ["https://omnivoice.palash.dev/sponsor"]
+78
View File
@@ -0,0 +1,78 @@
name: 🤝 Sponsorship inquiry
description: Support OmniVoice and (optionally) claim a logo slot. Not for bugs or feature requests.
title: "Sponsorship inquiry: "
labels: ["sponsor"]
body:
- type: markdown
attributes:
value: |
Thanks for considering sponsoring **OmniVoice Studio** 💛
OmniVoice is free, local-first, and AGPL-3.0 — sponsorship keeps development going.
See **[SPONSORS.md](https://github.com/debpalash/OmniVoice-Studio/blob/main/SPONSORS.md)** for tiers, placements, and logo guidelines.
Prefer to just donate? [Ko-fi](https://ko-fi.com/debpalash) (recurring) or [PayPal](https://paypal.me/palashCoder) (one-time) — you don't need this form for that.
- type: input
id: name
attributes:
label: Name or organization
description: How you'd like to be credited (person or company).
validations:
required: true
- type: input
id: website
attributes:
label: Website / link
description: The URL your name or logo should link to (homepage, product page, profile…).
placeholder: https://example.com
- type: input
id: logo
attributes:
label: Logo URL (optional)
description: Link to your logo (SVG preferred, else 2× PNG, transparent background). You can also attach it in the description below.
placeholder: https://example.com/logo.svg
- type: dropdown
id: tier
attributes:
label: Tier you're interested in
description: See SPONSORS.md for what each tier includes. Not sure? Pick "Not sure yet".
options:
- Backer
- Bronze
- Silver
- Gold
- Not sure yet — let's talk
- Custom / annual arrangement
validations:
required: true
- type: dropdown
id: method
attributes:
label: How you'd like to support
options:
- Ko-fi (recurring)
- Ko-fi (one-time)
- PayPal (one-time)
- Not sure yet — let's discuss
validations:
required: true
- type: input
id: contact
attributes:
label: How should we reach you?
description: Email or another contact. (GitHub will also notify you on this issue.)
validations:
required: true
- type: textarea
id: notes
attributes:
label: Anything else?
description: Questions, constraints, timeline, or context. Attach your logo here if you didn't link it above.
- type: checkboxes
id: ack
attributes:
label: Acknowledgements
options:
- label: I understand sponsorship is a thank-you, not a paywall — OmniVoice stays fully free and AGPL-3.0, and sponsors don't get gated features.
required: true
- label: If I provide a logo, I have the right to use it and grant OmniVoice permission to display it in the README, the app, and the project website.
required: false
+18 -2
View File
@@ -105,9 +105,25 @@ jobs:
working-directory: frontend
run: bun run typecheck:ci
# oxlint gate — fast Rust linter, blocks on errors so lint debt can't
# re-accumulate (warnings, incl. the react-compiler advisories in
# `lint:hooks`, are non-blocking). See frontend/.oxlintrc.json.
- name: Frontend lint (oxlint)
working-directory: frontend
run: bun run lint
# oxfmt format gate — JS/TS/JSX only (CSS/JSON/Tauri excluded; see
# frontend/.oxfmtrc.json). `bun run format` fixes locally.
- name: Frontend format check (oxfmt)
working-directory: frontend
run: bun run format:check
# `bun run test` (frontend/package.json), not `bunx vitest` — bunx
# resolves by npm package name and can miss workspace-hoisted bins,
# then falls back to fetching from npm (#962 class).
- name: Run Vitest (frontend)
working-directory: frontend
run: bunx vitest run
run: bun run test
# Legacy node:test runner for tests/frontend/*.test.mjs
- name: Run frontend node:test (legacy)
@@ -133,7 +149,7 @@ jobs:
- os: windows-2022
label: Windows
rust_target: x86_64-pc-windows-msvc
- os: ubuntu-22.04
- os: ubuntu-24.04
label: Linux
rust_target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
+135 -32
View File
@@ -4,12 +4,16 @@
# - push of a tag matching `v*` (e.g. `v0.2.0`) → full STABLE release,
# publishes artifacts + signed updater manifest (`latest.json`) to the
# tag's GH Release. This is the default Stable updater channel.
# - workflow_dispatch (publish_preview=true) → builds the selected branch and
# publishes a rolling `preview` PRERELEASE with its own signed
# `latest.json` at releases/download/preview/. This feeds the opt-in
# Preview updater channel (Settings → About → Update channel). The stable
# `latest` release is untouched. Run this manually whenever you want to cut
# a preview from `main`.
# - schedule (nightly, 07:00 UTC) → rolling `preview` PRERELEASE built from
# `main` with its own signed `latest.json` at releases/download/preview/.
# Feeds the opt-in Preview updater channel (Settings → About → Update
# channel). The `preview-gate` job skips the matrix on nights when `main`
# didn't move, so an idle day costs only a ~30s gate job — keeping Preview
# ≤24h behind `main` at a predictable ~1-matrix/day cost. The stable
# `latest` release is untouched.
# - workflow_dispatch (publish_preview=true) → the same preview build on
# demand from the selected branch (e.g. to preview a feature branch, or to
# refresh immediately without waiting for the nightly).
# - workflow_dispatch (publish_preview=false) → on-demand build (prior
# behavior; draft release named after the branch).
#
@@ -28,6 +32,10 @@ name: Desktop Release
on:
push:
tags: ['v*']
schedule:
# 07:00 UTC daily — rolling `preview` prerelease from `main`. The
# preview-gate job no-ops the matrix when main hasn't moved in a day.
- cron: '0 7 * * *'
workflow_dispatch:
inputs:
draft:
@@ -121,8 +129,41 @@ jobs:
working-directory: frontend
run: node --experimental-strip-types --no-warnings --test ../tests/frontend/*.test.mjs
# Decide preview-vs-stable, and for nightly runs whether `main` actually
# moved in the last day. Outputs gate the expensive matrix (`build`) and the
# `preview-notes` job, so a no-commit night costs only this ~30s job.
preview-gate:
name: Preview gate
runs-on: ubuntu-22.04
outputs:
is_preview: ${{ steps.decide.outputs.is_preview }}
proceed: ${{ steps.decide.outputs.proceed }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 50
- id: decide
shell: bash
run: |
set -euo pipefail
event="${{ github.event_name }}"
if [ "$event" = "schedule" ] || { [ "$event" = "workflow_dispatch" ] && [ "${{ inputs.publish_preview }}" = "true" ]; }; then
echo "is_preview=true" >> "$GITHUB_OUTPUT"
else
echo "is_preview=false" >> "$GITHUB_OUTPUT"
fi
# Nightly: skip the matrix when main hasn't moved in the last day.
if [ "$event" = "schedule" ] && [ -z "$(git log --since='25 hours ago' --oneline)" ]; then
echo "No new commits on main in the last day — skipping nightly preview."
echo "proceed=false" >> "$GITHUB_OUTPUT"
else
echo "proceed=true" >> "$GITHUB_OUTPUT"
fi
build:
needs: test
needs: [test, preview-gate]
# Nightly runs with no new commits on main skip the 4-platform matrix.
if: needs.preview-gate.outputs.proceed == 'true'
strategy:
fail-fast: false
matrix:
@@ -149,6 +190,14 @@ jobs:
# backlog that motivated the original drop is contained by
# fail-fast:false — a slow Intel leg can delay the release run but
# can't fail the other targets.
#
# #889 (2026-07): Intel macOS is now UNSUPPORTED for the local
# backend — torch ≥2.3 ships no macOS x86_64 wheels, so the venv
# bootstrap can never succeed on Intel. The shipped x64 artifact is
# effectively UI-only (usable with a remote backend); the app now
# pre-fails first-run bootstrap with an honest message on Intel.
# Whether to keep shipping this x64 leg (UI-only) or drop it is an
# OWNER CALL — deliberately not changed in the #889 PR.
- os: macos-15-intel
arch: x86_64-apple-darwin
label: "macOS Intel"
@@ -164,13 +213,24 @@ jobs:
bundles: "msi,updater"
# Linux: ship .AppImage only. AppImage is universal (no distro
# package-manager dep), runs on any glibc-2.31+ host, and is the
# package-manager dep), runs on any glibc-2.39+ host, and is the
# Linux auto-update target. The .deb target was dropped: tauri-bundler
# fails it with "Failed to create control scripts: No such file or
# directory" (no custom deb config of ours is at fault) — revisit on a
# tauri-cli bump. FUSE unavailability on GH runners is handled via
# APPIMAGE_EXTRACT_AND_RUN=1.
- os: ubuntu-22.04
#
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
# repos resolve (see the "Linux system deps" step below) — 22.04's
# was meaningfully stale relative to what current Ubuntu/Fedora
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
# take priority over a healthy system WebKitGTK at runtime. Raises
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
# something that old has come in, and the project's own install
# docs already assume Debian 12 / Ubuntu 22.04+.
- os: ubuntu-24.04
arch: x86_64-unknown-linux-gnu
label: "Linux x64"
rust_target: x86_64-unknown-linux-gnu
@@ -427,11 +487,14 @@ jobs:
# the Windows MSI ProductVersion (which strips the prerelease → 0.3.6)
# is also correctly above the last stable.
- name: Stamp preview version
if: github.event_name == 'workflow_dispatch' && inputs.publish_preview
if: needs.preview-gate.outputs.is_preview == 'true'
shell: bash
run: |
set -euo pipefail
CONF=frontend/src-tauri/tauri.conf.json
# package.json is the single source of truth; tauri.conf.json reads its
# version from it ("version": "../package.json"), so stamping
# package.json restamps the whole bundle.
CONF=frontend/package.json
BASE=$(jq -r .version "$CONF")
# MSI/WiX requires the semver pre-release identifier to be numeric-only
# (and <= 65535). "preview.N" hard-fails the Windows bundler, so the
@@ -471,11 +534,13 @@ jobs:
# rolling `preview` prerelease for the updater's Preview channel.
# Every other invocation — crucially the `v*` tag-push stable release
# — evaluates these expressions to exactly their prior values.
tagName: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'preview' || github.ref_name }}
releaseName: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'OmniVoice Studio (Preview)' || format('OmniVoice Studio {0}', github.ref_name) }}
tagName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'preview' || github.ref_name }}
# Version-first so the tag is readable in GitHub's truncated
# release-list sidebar (which clips the title mid-string).
releaseName: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'Preview — OmniVoice Studio' || format('{0} — OmniVoice Studio', github.ref_name) }}
releaseBody: ${{ steps.changelog.outputs.body }}
releaseDraft: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) && 'false' || (inputs.draft || 'true') }}
prerelease: ${{ (github.event_name == 'workflow_dispatch' && inputs.publish_preview) || false }}
releaseDraft: ${{ (needs.preview-gate.outputs.is_preview == 'true') && 'false' || (inputs.draft || 'true') }}
prerelease: ${{ needs.preview-gate.outputs.is_preview == 'true' }}
updaterJsonPreferNsis: false
includeUpdaterJson: true
@@ -661,8 +726,8 @@ jobs:
# preview-only — stable `v*` releases keep their CHANGELOG section + the
# appended checksums.
preview-notes:
needs: build
if: github.event_name == 'workflow_dispatch' && inputs.publish_preview
needs: [build, preview-gate]
if: needs.preview-gate.outputs.is_preview == 'true'
runs-on: ubuntu-22.04
permissions:
contents: write
@@ -693,16 +758,51 @@ jobs:
echo ""
echo "$CONTRIB"
} > /tmp/preview-notes.md
gh release edit preview --repo "$REPO" --notes-file /tmp/preview-notes.md
# --prerelease re-asserts the flag every run: a non-prerelease
# `preview` release is eligible to become GitHub's "Latest", which is
# the exact URL the Stable updater channel reads — so it must never
# flip off.
gh release edit preview --repo "$REPO" --prerelease --notes-file /tmp/preview-notes.md
echo "Applied auto-generated release notes + contributors to the preview release."
# ── Post-release version bump (versioning hard rule, owner-set 2026-06-11) ──
# main is always last-release + 1 patch. The moment a stable v* tag is
# released, bump the three version sources on main to the next patch so every
# PR and preview build identifies as the next version. Pushes directly to
# main with the workflow token (a metadata-only commit; CI runs on PRs).
- name: Verify preview updater manifest (prerelease + platform parity)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
# The preview release must stay a prerelease (or it can hijack the
# Stable channel's releases/latest endpoint), and its updater manifest
# must cover every platform stable does (else those users — e.g. Intel
# Mac — silently get no preview updates).
is_pre=$(gh release view preview --repo "$REPO" --json isPrerelease -q .isPrerelease)
test "$is_pre" = "true" || { echo "::error::preview release is not a prerelease"; exit 1; }
curl -fsSL "https://github.com/$REPO/releases/download/preview/latest.json" -o /tmp/preview-latest.json
curl -fsSL "https://github.com/$REPO/releases/latest/download/latest.json" -o /tmp/stable-latest.json
python3 - <<'PY'
import json, re
prev = json.load(open("/tmp/preview-latest.json"))
stab = json.load(open("/tmp/stable-latest.json"))
v = prev.get("version", "")
assert re.fullmatch(r"\d+\.\d+\.\d+-\d+", v), f"preview version not X.Y.Z-N: {v!r}"
pk, sk = set(prev.get("platforms", {})), set(stab.get("platforms", {}))
missing = sk - pk
assert not missing, f"preview manifest missing platforms vs stable: {sorted(missing)}"
print(f"preview manifest OK: {v} platforms={sorted(pk)}")
PY
# ── Post-release version bump (OWNER-GATED as of 2026-07-01) ──────────────
# Previously auto-ran after every stable v* tag to keep main = release + 1.
# The owner now controls bumps manually ("keep 0.3.8; I say when to bump"), so
# this job is OPT-IN: it runs ONLY when the repo variable AUTO_VERSION_BUMP is
# set to 'true' (Settings → Secrets and variables → Actions → Variables).
# Unset/anything-else → main stays at whatever it is after release. Re-enable
# by setting the variable; disable again by unsetting it.
version-bump:
if: github.event_name == 'push' && github.ref_type == 'tag' && !contains(github.ref, '-')
if: >-
github.event_name == 'push' && github.ref_type == 'tag'
&& !contains(github.ref, '-')
&& vars.AUTO_VERSION_BUMP == 'true'
runs-on: ubuntu-22.04
permissions:
contents: write
@@ -718,23 +818,26 @@ jobs:
RELEASED="${GITHUB_REF_NAME#v}"
IFS=. read -r MAJ MIN PAT <<< "$RELEASED"
NEXT="$MAJ.$MIN.$((PAT + 1))"
CURRENT=$(jq -r .version frontend/src-tauri/tauri.conf.json)
# frontend/package.json is the SINGLE SOURCE OF TRUTH: vite injects
# __APP_VERSION__ from it, and tauri.conf.json reads its bundle version
# from it ("version": "../package.json"). Read CURRENT from it.
CURRENT=$(jq -r .version frontend/package.json)
if [ "$(printf '%s\n' "$NEXT" "$CURRENT" | sort -V | tail -1)" = "$CURRENT" ] && [ "$NEXT" != "$CURRENT" ]; then
echo "main is already at $CURRENT (>= $NEXT) — nothing to bump"; exit 0
fi
tmp=$(mktemp)
jq --arg v "$NEXT" '.version = $v' frontend/src-tauri/tauri.conf.json > "$tmp"
mv "$tmp" frontend/src-tauri/tauri.conf.json
# frontend/package.json drives __APP_VERSION__ (vite.config.js) — the
# first-run footer + every auto bug report. Keep it in lockstep too,
# set absolutely (jq) so any prior drift self-heals. (#248-sweep finding)
# Bump the canonical (package.json), set absolutely so any prior drift
# self-heals. tauri.conf.json needs no edit — it derives from this.
tmp=$(mktemp)
jq --arg v "$NEXT" '.version = $v' frontend/package.json > "$tmp"
mv "$tmp" frontend/package.json
# The remaining files are CI-guarded mirrors (cargo/uv require a
# literal; the version.py literal is the frozen-backend last resort) —
# bump them in lockstep with the canonical.
sed -i "0,/^version = \"$CURRENT\"/s//version = \"$NEXT\"/" frontend/src-tauri/Cargo.toml
sed -i "0,/^version = \"$CURRENT\"/s//version = \"$NEXT\"/" pyproject.toml
sed -i "0,/_FALLBACK_VERSION = \"$CURRENT\"/s//_FALLBACK_VERSION = \"$NEXT\"/" backend/core/version.py
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add frontend/src-tauri/tauri.conf.json frontend/package.json frontend/src-tauri/Cargo.toml pyproject.toml
git add frontend/package.json frontend/src-tauri/Cargo.toml pyproject.toml backend/core/version.py
git commit -m "chore(version): main -> $NEXT after $GITHUB_REF_NAME release"
git push origin main
+1047
View File
File diff suppressed because it is too large Load Diff
+17 -176
View File
@@ -3,7 +3,7 @@
**OmniVoice Studio**
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The latest stable release is **v0.3.5**; `main` rolls ahead at **v0.3.6** (latest release + 1 patch — see the Versioning rule below).
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
@@ -16,188 +16,36 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
- **Local-first guarantee preserved**: Auto bug reporting (new addition) must be **opt-in**, must submit only to GitHub Issues (no third-party telemetry endpoint), and the app must remain fully functional with reporting disabled. No required cloud calls, accounts, or API keys.
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release (currently **v0.3.5**). ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
<!-- GSD:project-end -->
<!-- GSD:stack-start source:research/STACK.md -->
## Technology Stack
## Recommended Stack — Per Capability
### Capability 1 — HuggingFace Token Persistence (issue #35)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `huggingface_hub` (already pinned transitively by `transformers>=5.3.0`) | `≥1.12.x` (latest 2026) | Auth + cache + token storage | Canonical, used by every HF library already in the stack. `HfFolder` is **superseded** in v1.x by the higher-level `login()` / `auth_list()` / `auth_switch()` API. |
| `keyring` (Python) | `≥25.x` | Optional OS-keychain backing | Only adopt if a future hardening pass wants Keychain/Credential-Manager/SecretService. **Not recommended for this milestone** — adds a native dep (`dbus`, `pywin32`) per platform with no real security win over `0600` file storage in `HF_HOME`. |
| Shell | One-liner to persist `HF_TOKEN` |
|-------|---------------------------------|
| macOS zsh (default since 10.15) | `echo 'export HF_TOKEN=hf_xxx' >> ~/.zshrc && source ~/.zshrc` |
| Linux bash | `echo 'export HF_TOKEN=hf_xxx' >> ~/.bashrc && source ~/.bashrc` |
| Windows PowerShell (user scope) | `[Environment]::SetEnvironmentVariable("HF_TOKEN","hf_xxx","User")` (new shells only) |
| Windows cmd | `setx HF_TOKEN "hf_xxx"` (user scope, new shells only) |
- [HF environment variables docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH confidence (official, current)
- [HF authentication API docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH confidence
- [Microsoft `setx` docs](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH confidence
### Capability 2 — In-App Structured Bug Reporting (opt-in, GitHub Issues)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| GitHub REST API `POST /repos/{owner}/{repo}/issues` | `2026-03-10` API version | Server-side issue creation | Official, stable. Requires auth. |
| **Prefilled-URL pattern** (`github.com/{owner}/{repo}/issues/new?title=…&body=…&labels=…`) | n/a | Zero-auth fallback | **This is the recommended primary path for v0.3.x.** No token needed, no GitHub App registration needed, user's browser opens with a prefilled form, they review and click Submit. They own the issue, the OSS project gets the report, and OmniVoice never holds a credential. |
| `gh-app-jwt` + GitHub App (Rust crate `octocrab` or Python `pygithub`) | only if we later want fully-automated submission | Programmatic posting under an app identity | **Defer to a later milestone.** Requires registering a public GitHub App, hosting a token-exchange endpoint, and managing rate-limit quotas — disproportionate for stabilization scope. |
| `platform`, `psutil`, `torch.cuda` (already in deps) | already pinned | Capture OS, CPU/GPU/VRAM info | No new deps. |
| `httpx` (already in `dev-dependencies`, promote to runtime if needed) | `≥0.28.1` | HTTP for the API call path (if/when we add auth) | Modern async-first, already used in test suite. |
- ✓ No token storage in OmniVoice → no security surface
- ✓ Opt-in by definition (user has to click Submit on github.com)
- ✓ User owns the issue → can be replied to, edited, closed by them
- ✓ Zero infra cost — no proxy, no app, no rate-limit management
- ✓ Works identically on macOS / Windows / Linux via Tauri's `shell.open`
- ✓ Survives our project being forked (just change the URL)
- OS name + version (`platform.platform()`)
- Python version (`sys.version`)
- OmniVoice version (`pyproject.toml`)
- Backend git SHA (if installed from source) or installer build ID
- CPU model, RAM (`psutil.cpu_count()`, `psutil.virtual_memory()`)
- GPU vendor/model/VRAM (`torch.cuda.get_device_name()`, `torch.cuda.mem_get_info()`, MPS detect)
- Active TTS engine + list of installed engines
- Frontend: bun version, OS shell
- Last error message + stack trace if launched from an error toast
- Audio file contents (privacy — reference samples may contain user's voice)
- File paths containing `/Users/<name>/` (strip home dir → `~/`)
- HF token, OpenAI keys, any env var matching `*TOKEN*|*KEY*|*SECRET*`
- [GitHub URL query parameters for issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/creating-an-issue#creating-an-issue-from-a-url-query) — HIGH confidence
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (widely used reference impl)
- [GitHub REST API: Create an issue](https://docs.github.com/en/rest/issues/issues#create-an-issue) — HIGH confidence (for the future auto-submit path)
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — reviewed, **rejected for milestone** due to local-first constraint
### Capability 3 — `uv venv` Mirror Fallback for Restricted Networks (issues #57, #60)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `uv` (already used) | `≥0.5.x` | Python+venv bootstrap | Existing dep. |
| `UV_PYTHON_INSTALL_MIRROR` env var | uv `0.4.x`+ | Override python-build-standalone download URL | **Official, current.** Replaces `https://github.com/astral-sh/python-build-standalone/releases/download/...` in download URL construction. No built-in fallback if mirror fails. |
| `UV_PYTHON_PREFERENCE=only-system` (or CLI flag `--python-preference only-system`) | uv `0.4.x`+ | Skip the python-build-standalone download entirely; use the user's system Python | **The reliable escape hatch** when no mirror works. Requires a compatible Python `>=3.11` to already be on PATH. |
| `UV_HTTP_TIMEOUT`, `UV_HTTP_CONNECT_TIMEOUT`, `UV_HTTP_RETRIES` | uv `0.4.x`+ | Tune retry behavior for flaky links | Defaults are 30s / 10s / 3 — bump to 120s / 30s / 5 for restricted networks. |
# Pseudocode for the bootstrap
# Final fallback: don't download Python at all
- `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (Tsinghua — fastest in China)
- `UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple` (Aliyun fallback)
- Russia: no major government-blessed PyPI mirror; users typically tunnel via VPN. Document this honestly rather than ship a broken default.
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (official)
- [uv issue #5224 — python-build-standalone mirror support](https://github.com/astral-sh/uv/issues/5224) — HIGH (the feature was added)
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms real user pain, no built-in fallback)
- [uv python-versions concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH (documents `python-preference` semantics)
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained mirror list; verify each URL still works before shipping)
### Capability 4 — Supertonic-3 TTS Engine
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `supertonic` (PyPI) | `1.3.1` (latest, May 18 2026 — Phase 3 Wave 1 to verify constructor signature before bump) | Official Supertonic-3 inference SDK | Authoritative wrapper from Supertone Inc. Wraps the ONNX session orchestration so we don't have to. |
| `onnxruntime` | `≥1.17.x` (any recent) | ONNX inference runtime | Already a transitive dep of WhisperX (via CTranslate2 path is separate, but `onnxruntime` itself ships for kittentts and audioseal). Verify with `uv tree` after adding — should resolve cleanly. |
| `huggingface_hub` (already pinned) | `≥1.12.x` | Model weight download (~400 MB on first use) | Reuses existing HF token + cache infrastructure. The user's existing `HF_TOKEN` (Capability 1) works for the Supertonic model download too. |
| `numpy`, `soundfile` (already pinned) | already pinned | Audio I/O + array math | No new deps. |
- `text_encoder.onnx`
- `latent_denoiser.onnx`
- `voice_decoder.onnx`
- 44.1 kHz sample rate, 24-dim latent, 128-dim style
- ~99M parameters total
- Tokenizer: `AutoTokenizer.from_pretrained(model_path)` — loads from `tokenizer.json` shipped with model
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official)
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official)
- [supertonic PyPI page](https://pypi.org/project/supertonic/) — HIGH (`1.3.1` confirmed 2026-05-18; same publisher, MIT, same 4 deps)
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure details)
### Capability 5 — Cross-Platform Documentation Tooling
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| Plain Markdown in `docs/` + GitHub-rendered (current state) | n/a | Install tutorial, troubleshooting | Zero new infra. Renders inline on GitHub for issue-replies. No build step to break. |
| Existing `scripts/smoke-test.sh` + Playwright `tests/` (already in `package.json`) | already pinned | Verify install paths actually work | **This is the real solution to "docs drift."** If smoke-test exercises the install path described in docs, docs that drift will break CI. |
| **Future** (defer): Astro Starlight | `≥0.30` | Standalone docs site at `docs.omnivoice.studio` | Adopt only when docs exceed ~20 markdown files and need search/versioning. Tauri, the framework OmniVoice already depends on, uses Starlight — well-traveled choice. Material for MkDocs entered maintenance mode in November 2025 per Docsio's 2026 review — **avoid** for new docs. |
| Project | What they do |
|---------|--------------|
| **OBS Studio** | Docs at `obsproject.com/docs` (Sphinx, separate repo). Install paths in README, wiki for community-contributed. CI doesn't gate on docs drift. |
| **Audacity** | Manual at `manual.audacityteam.org` (MediaWiki). README is minimal. Install path = "use the installer." No automated sync. |
| **Tauri** | Docs at `v2.tauri.app` (Astro Starlight, separate repo `tauri-apps/tauri-docs`). README is minimal. Heavy reliance on community contributions and PR review. |
| **VS Code** | Docs at `code.visualstudio.com/docs` (separate repo, Markdown). README is minimal. Manual sync; docs team is staffed. |
- [Tauri docs (Astro Starlight)](https://github.com/tauri-apps/tauri-docs) — HIGH (reference for "if we ever move off README")
- [OBS Studio docs](https://docs.obsproject.com/) — HIGH (Sphinx, separate site reference)
- [Audacity Manual](https://manual.audacityteam.org/) — HIGH (MediaWiki reference)
- [Docsio: Material for MkDocs 2026 review (maintenance mode)](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with project's own GitHub activity)
- [Docsio: Starlight 2026 review](https://docsio.co/blog/starlight-docs) — MEDIUM
## Installation
# No new Python dependencies needed for Capabilities 1, 2, 3, 5.
# Only Capability 4 adds a runtime dep:
# Verify no regressions:
# Should show single versions of each; no duplicates.
## Alternatives Considered
| Recommended | Alternative | When to Use Alternative |
|-------------|-------------|-------------------------|
| HF token via in-app Settings → `huggingface_hub.login()` | OS keyring via `keyring` package | Only if a security hardening milestone later demands OS-native credential storage. Not worth the cross-platform native-dep cost for v0.3.x. |
| Prefilled-URL GitHub Issues | GitHub App + device flow + authenticated POST | When milestone budget can afford registering a public GitHub App and hosting a token-exchange function. Defer. |
| Prefilled-URL GitHub Issues | Sentry / `sentry-tauri` | Never — violates the "no third-party telemetry endpoint" constraint in PROJECT.md. |
| `UV_PYTHON_INSTALL_MIRROR` chain + `only-system` fallback | Bundle Python in the Tauri installer | Adds ~30 MB to every installer for ~5% of users. Revisit if the bootstrap is still a top complaint in v0.4. |
| In-repo Markdown docs | Astro Starlight standalone site | When docs grow past ~20 pages and need full-text search. Tauri provides a precedent if/when we get there. |
| In-repo Markdown docs | MkDocs / Material for MkDocs | **Avoid** for new sites — Material for MkDocs is in maintenance mode as of Nov 2025. |
## What NOT to Use
| Avoid | Why | Use Instead |
|-------|-----|-------------|
| `HfFolder.save_token()` directly | Older API; v1.x `login()` does the same plus git-credential integration and is the documented path | `huggingface_hub.login(token=val, add_to_git_credential=False)` |
| Setting `HF_TOKEN` via shell rc files as the *only* persistence mechanism | Different per OS, fragile, opaque to the user, breaks in installer-launched processes that don't source shell rc | Write to `$HF_HOME/token` via `login()`. Document env var as override only. |
| `setx` for HF token persistence | Doesn't propagate to current shell; common source of "I set it but it's empty" bug reports | `[Environment]::SetEnvironmentVariable(...,"User")` in PowerShell, or the in-app Settings field |
| PAT-based GitHub Issues posting from OmniVoice | Would require shipping or asking for a token; breaks local-first promise | Prefilled-URL pattern (user submits from their browser) |
| `sentry-tauri` for OmniVoice | Third-party telemetry endpoint — violates PROJECT.md constraint | Local-only `backend.log` rotation + opt-in prefilled-URL reporter |
| `hf_transfer` for downloads | Deprecated in favor of `hf-xet` per HF docs | Default `huggingface_hub` (uses `hf-xet` automatically when available) |
| `--python-preference managed` (default) without mirror config in restricted-network installers | Hits GitHub CDN, times out, user sees raw `uv` error | Configure `UV_PYTHON_INSTALL_MIRROR` + retry chain + `only-system` final fallback |
| Material for MkDocs as a *new* docs choice | Entered maintenance mode November 2025 | If docs site is eventually needed, use Astro Starlight (Tauri precedent) |
## Stack Patterns by Variant
- Set `UV_PYTHON_INSTALL_MIRROR` to one of the gh-proxy URLs at install time
- Set `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (China) or document VPN requirement (Russia)
- Fall back to `UV_PYTHON_PREFERENCE=only-system` if all mirrors fail
- Increase `UV_HTTP_TIMEOUT=120`, `UV_HTTP_RETRIES=5`
- Default path: in-app Settings field → `login()` → file at `$HF_HOME/token`
- Power-user path: `export HF_TOKEN=...` in shell rc (documented but not promoted)
- Both paths are read at HF library import time; env var wins on conflict
- Default path: in-app "Report a bug" → prefilled GitHub Issues URL → user reviews + submits in browser
- All optional capture toggles default ON except "include reproduction file" (privacy)
- No path posts to any URL except `github.com/{owner}/{repo}/issues/new` (rendered locally as a URL, opened via `shell.open`)
- `uv add supertonic` → new TTSBackend subclass in `backend/services/tts_backend.py`
- Auto-detected and added to the engine picker in Settings
- ~400 MB model download on first synthesize call, cached in `$HF_HUB_CACHE`
- Existing IndexTTS/CosyVoice/etc. installs are untouched (no shared model weights)
## Version Compatibility
| Package A | Compatible With | Notes |
|-----------|-----------------|-------|
| `supertonic@1.3.1` | `onnxruntime>=1.17`, `numpy>=1.24`, `huggingface_hub>=0.20` | All deps already satisfied transitively by current `pyproject.toml`. |
| `huggingface_hub>=1.12` | `transformers>=5.3.0` (current pin) | `HfFolder` retained as deprecated alias; `login()`/`get_token()` are the canonical APIs. |
| `uv>=0.5` | `UV_PYTHON_INSTALL_MIRROR`, `UV_PYTHON_PREFERENCE` | Both env vars stable since uv 0.4.x. |
| Tauri v2 + `@tauri-apps/api/shell` | `shell.open()` for the prefilled-URL pattern | Already in the desktop app; no new permission needed beyond what the existing "open external link" plugin grants. |
## Sources
- [Hugging Face Hub environment variables](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH (verified against v1.12.1 docs, current 2026)
- [Hugging Face Hub authentication API](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH (verified `login()` is the canonical 1.x API)
- [Microsoft `setx` reference](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH (confirms "current shell" gotcha)
- [PowerShell `about_Environment_Variables`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables) — HIGH
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (verified all mirror + retry env vars)
- [uv issue #5224 — python-build-standalone mirror](https://github.com/astral-sh/uv/issues/5224) — HIGH (feature shipped)
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms user pain, justifies fallback chain)
- [uv `python-preference` semantics](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official, 99M params, 31 languages, OpenRAIL-M)
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official inference API)
- [supertonic 1.3.1 on PyPI](https://pypi.org/project/supertonic/) — HIGH (released 2026-05-18, MIT code license; bumped from 1.2.3 after Phase 3 research)
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure)
- [GitHub Docs: Authenticating to the REST API](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api) — HIGH
- [GitHub Docs: Generating a user access token for a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) — HIGH (device flow reference)
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (canonical prefilled-URL reference impl)
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — MEDIUM (reviewed, rejected on PROJECT.md constraint, not on quality)
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained, verify URLs are still live before pinning in production)
- [Tauri 2 docs (Astro Starlight reference)](https://v2.tauri.app/) — HIGH (precedent for docs framework if we ever migrate)
- [Docsio: Material for MkDocs entered maintenance mode Nov 2025](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with the project's own GitHub commit activity)
The May-2026 stack research that used to live here served five capabilities that have all since shipped (HF-token Settings panel, prefilled-URL bug reporting, uv mirror fallback for restricted networks, the Supertonic-3 engine, in-repo Markdown docs). Follow the patterns in the code itself; the durable *don'ts* that research established:
- **No third-party telemetry endpoints, ever** (`sentry-tauri` was evaluated and rejected) — bug reporting stays opt-in via prefilled GitHub-issue URLs.
- **No PAT/token-based GitHub posting from the app** — the user submits from their own browser.
- **Don't recommend `setx` for env vars on Windows** (silent truncation, no current-shell propagation) — use the in-app Settings panel or PowerShell `[Environment]::SetEnvironmentVariable`.
- **Don't adopt Material for MkDocs** for any future docs site (maintenance mode since Nov 2025) — Astro Starlight is the precedent if docs ever outgrow the repo.
- **`hf_transfer` is deprecated** — default `huggingface_hub` (hf-xet) handles downloads.
For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/package.json`, and check `uv tree` for conflicts before adding a dependency.
<!-- GSD:stack-end -->
<!-- GSD:conventions-start source:CONVENTIONS.md -->
## Conventions
**Versioning (hard rule, owner-set 2026-06-11):** main is always **latest release + 1 patch**. The moment `vX.Y.Z` is released, main's version files (`frontend/src-tauri/tauri.conf.json`, `frontend/src-tauri/Cargo.toml`, `pyproject.toml`, **and `frontend/package.json`** keep all **four** in lockstep; `package.json` drives the runtime `__APP_VERSION__` via vite, shown in the first-run footer + every auto bug report, so a drift ships a build that misreports its own version — guarded by `tests/test_app_version.py::test_all_version_files_in_lockstep`) bump to `X.Y.(Z+1)`. Consequences:
**Versioning (hard rule, owner-set 2026-06-11; single-source 2026-06-16):** main is always **latest release + 1 patch**. **`frontend/package.json` is the SINGLE SOURCE OF TRUTH for the app version** — vite injects `__APP_VERSION__` from it (first-run footer + every auto bug report), and `frontend/src-tauri/tauri.conf.json` reads its bundle version from it (`"version": "../package.json"`, so the MSI/dmg/updater version can't drift from the UI). Three toolchain-required **mirrors** are kept equal to it and bumped in lockstep `frontend/src-tauri/Cargo.toml` + `pyproject.toml` (cargo/uv need a literal) and `backend/core/version.py`'s `_FALLBACK_VERSION` (the frozen-backend last resort; at runtime the backend reads its version from package metadata via `importlib.metadata`, which `backend.spec`'s `copy_metadata('omnivoice')` makes work in the frozen build too). Never hand-edit any mirror or re-hardcode a literal in `tauri.conf.json`. Guarded by `tests/test_app_version.py` (`test_all_version_files_in_lockstep` + `test_tauri_version_derives_from_package_json`). The moment `vX.Y.Z` is released, bump `package.json` (+ the mirrors) to `X.Y.(Z+1)`. Consequences:
- Every PR and preview build identifies as the **next** version. Preview builds stamp `X.Y.(Z+1)-N` (run number), which semver-sorts **above** the last stable `X.Y.Z` — the updater ordering is natural, no comparator tricks needed.
- Releasing = tag `vX.Y.(Z+1)` from main (version files already match), then immediately bump main to `X.Y.(Z+2)`. The post-release bump is automated by the `version-bump` job in release.yml; if it fails, do it manually in the same day.
- Releasing = tag `vX.Y.(Z+1)` from main (version files already match), then immediately bump main to `X.Y.(Z+2)`. **Owner override (2026-07-01): the post-release bump is now MANUAL — the `version-bump` job in release.yml is opt-in behind the `AUTO_VERSION_BUMP` repo variable (default off), so `main` stays at the released version until the owner explicitly asks to bump.** (Historically the bump auto-ran; re-enable that by setting `AUTO_VERSION_BUMP=true`.) When pinned, `main` == the released tag; preview-build ordering and "release + 1" only resume once a bump is requested.
- Docker: `ghcr.io/debpalash/omnivoice-studio:latest` = **main** (rolling preview); `:X.Y.Z` + `:X.Y` + `:stable` = tagged releases. `:latest` is the preview channel by design — stable users pin `:stable` or a version tag.
- Do not bump minor/major or invent RCs/codenames without the owner asking. No "defer to next version" labels — scope is absorbed or declined, never re-versioned.
**Docs-sync (hard rule, owner-set 2026-06-11):** any change that alters something these docs describe — README.md, CONTRIBUTING.md, SECURITY.md, SUPPORT.md, LICENSE, or `docs/**` (install flows, Docker tag semantics, platform support, versioning/release behavior, review process, supported versions) — must update those docs **in the same PR** as the change. If a doc impact is discovered after merge, the docs fix is the immediate next commit, not backlog. Stale docs are treated as bugs.
**Release notes / changelog (hard rule, owner-set 2026-06-16):** every tagged release gets a **high-quality, user-facing `## [X.Y.Z] — DATE` section in `CHANGELOG.md`** before (or in the same hour as) the tag — never the "Auto-generated release for vX.Y.Z…" fallback. `release.yml` extracts that section verbatim as the GitHub Release body (the `Extract CHANGELOG section for tag` step), so a missing/empty section ships a bare release. Quality bar = the existing house style: a one-paragraph headline, then `### Added` / `### Fixed` / `### Changed` / `### License` / `### CI` subsections; each entry is a **bold one-line lead** (what the user gets), 13 lines of plain-English why, and the `(#NNN)` issue/PR ref — grouped by theme, written for users, **not** raw commit dumps. This applies to **preview builds too**: preview release notes summarize what's new on `main` since the last stable, in the same style. Workflow: as features merge, keep `## [Unreleased]` current; at release time rename it to the version + date. If a release was already cut with the fallback body, the next action is to backfill `CHANGELOG.md` **and** `gh release edit <tag>` the live body — not backlog.
**Localization (hard rule):** No hardcoded non-English (CJK) **user-facing text** anywhere in the codebase except the translation layer (`frontend/src/i18n/`). All UI strings go through i18n (`t('...')` keys in `locales/*.json`); native language names live in `i18n/index.ts` (`LANGUAGES`). Functional CJK is allowed and tracked via the allowlist in `tests/test_no_hardcoded_cjk.py` — text-processing regexes, model/engine vocabulary & identifiers (e.g. CosyVoice speaker IDs), localized error matching, demo/eval data, and test fixtures. CI fails on any hardcoded CJK outside the allowlist; to add legitimate functional CJK, extend `_ALLOWED_FILES` there with a justification.
**Fix quality (hard rule, owner-set 2026-06-16):** Fix issues *properly* and future-maintenance-proof — don't stop at the symptom. Root-cause fully, fix the whole **class** of the bug (not just the one reported instance), add a fail-before/pass-after regression test, and harden against recurrence (e.g. if a lockfile drift only fails in Docker, also make CI catch it). Go the extra mile where it durably pays off. Be token-efficient about it — extra **effort**, not extra **verbosity**: no padding, no redundant re-checks, the smallest correct change that is also recurrence-proof. Don't be shy to spend the effort a proper fix needs; do be shy about wasting tokens.
@@ -220,16 +68,9 @@ No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skill
<!-- GSD:skills-end -->
<!-- GSD:workflow-start source:GSD defaults -->
## GSD Workflow Enforcement
## Workflow
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
Use these entry points:
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
- `/gsd-debug` for investigation and bug fixing
- `/gsd-execute-phase` for planned phase work
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command gate that used to live here referenced `/gsd-quick` / `/gsd-debug` / `/gsd-execute-phase` skills that are not installed in this environment; the owner chose to keep working directly rather than restore them. The working conventions that matter are in **Conventions** above — versioning, docs-sync, changelog, localization, fix quality, keep-main-green — plus: gate every merge on the "Tests (backend + frontend)" check passing and the PR being MERGEABLE, and check the open-PR queue before implementing any community-reported fix (contributors may have already submitted one).
<!-- GSD:workflow-end -->
+26 -1
View File
@@ -18,6 +18,7 @@ Thanks for your interest in improving OmniVoice Studio! This guide covers everyt
### Prerequisites
- [Git](https://git-scm.com/)
- `curl` (used by the Bun / uv / rustup install one-liners on macOS and Linux)
- [Bun](https://bun.sh/) (frontend package manager)
- [uv](https://docs.astral.sh/uv/) (Python environment manager)
- [ffmpeg](https://ffmpeg.org/) (audio/video processing)
@@ -159,7 +160,7 @@ class MyEngineBackend(TTSBackend):
- **Components**: Functional components with hooks
- **State**: Zustand stores in `src/stores/`, organized by slice
- **CSS**: Vanilla CSS in component-level files — no Tailwind
- **CSS**: **Utilities-first + shadcn/ui, one stylesheet.** UI is built on the shadcn/ui primitives in `src/components/ui/` (wrapped by the `src/ui/` barrel, themed to the OmniVoice palette), composed with Tailwind v4 utility classes. **All styling now lives in a single file — `src/index.css`**: the `@theme` / `[data-theme]` token foundation plus the irreducible set utilities can't express (`@keyframes`, glassmorphism/`backdrop-filter`, pseudo-elements, `:has()`, unlayered cascade overrides, and styling hooks on library-generated DOM like virtualized rows / WaveSurfer). The per-component `.css` files were eliminated in the CSS→Tailwind/shadcn migration — **do not create new ones.** Reach for shadcn primitives + utilities; if a rule is genuinely irreducible, add it to `src/index.css` with a provenance comment. (The only other `.css` is the test-only visual harness. See `docs/shadcn-migration.md`.)
- **Naming**: `PascalCase` for components, `camelCase` for hooks and utils
### Rust (Tauri)
@@ -169,6 +170,30 @@ class MyEngineBackend(TTSBackend):
---
## Frontend file structure & size limits
Frontend code stays modular so an edit loads one small file, not a 1900-line
one. The rules:
- **Size caps:** **soft 300 lines**, **hard 500 lines** per `.jsx` file.
Anything over 500 lines must be split. (The cap does **not** apply to
`src/index.css` — it is the single, intentional styling foundation and the
only app stylesheet; see the CSS rule above.)
- **Pages are thin orchestrators.** A file in `frontend/src/pages/` is just
layout + routing + state wiring that composes feature components — no inline
sub-component over ~50 lines.
- **One component per file.** Co-locate `Foo.jsx` + `Foo.test.jsx` together in a
per-page feature folder under `frontend/src/components/` (e.g.
`components/settings/`, `components/dub/`). Styling is **not** co-located —
it's utilities + shadcn, with any irreducible rules in `src/index.css`.
- **Shared bits go in a `primitives/` folder** inside the feature folder
(`components/settings/primitives/` is the existing example).
- **Enforced by ESLint `max-lines`** (`max: 500`) — **warn-only for now** so it
never breaks CI, with the goal of upgrading to `error` once the backlog of
oversized files clears.
---
## Commit Messages
Write clear, concise messages. The PR title becomes the squash-merge commit.
+389 -175
View File
@@ -4,45 +4,49 @@
<h3>The open-source ElevenLabs alternative.</h3>
<p>Real-time dictation, zero-shot voice cloning, and cinematic video dubbing — all on your desktop.<br/>Open-source, no API keys, fully local. <b>646 languages.</b></p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Stars" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="Release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
</p>
<p>
<a href="#quickstart">Quickstart</a> ·
<a href="#features">Features</a> ·
<a href="#why-omnivoice-studio">Why OmniVoice Studio?</a> ·
<a href="#why-ovs">Why OVS</a> ·
<a href="#tts-engines">TTS Engines</a> ·
<a href="#asr-engines">ASR Engines</a> ·
<a href="#openai-api">API</a> ·
<a href="#sponsors">Sponsors</a> ·
<a href="#sponsor--donate">Donate</a> ·
<a href="#contributing">Contributing</a> ·
<a href="https://discord.gg/bzQavDfVV9">Discord</a> ·
<a href="README_CN.md"><strong>简体中文</strong></a>
</p>
<p>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<!-- Pre-built macOS bundle is Apple Silicon. Intel Macs: build from source (docs/install/macos.md); a pre-built Intel target is tracked in #279. -->
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
</p>
<p>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
<a href="https://github.com/debpalash/OmniVoice-Studio/stargazers"><img src="https://img.shields.io/github/stars/debpalash/OmniVoice-Studio?style=flat-square&color=f59e0b" alt="Stars" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/github/v/release/debpalash/OmniVoice-Studio?style=flat-square&color=10b981" alt="Release" /></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue?style=flat-square" alt="License" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/issues"><img src="https://img.shields.io/github/issues/debpalash/OmniVoice-Studio?style=flat-square&color=ef4444" alt="Issues" /></a>
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white" alt="Discord" /></a>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_Us-FF5E5B?style=flat-square&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=flat-square&logo=paypal&logoColor=white" alt="PayPal" /></a>
</p>
</div>
<br/>
<div align="center">
<img src=".github/assets/social-preview.png" alt="OmniVoice Studio — The open-source ElevenLabs alternative" width="100%"/>
<img src="docs/screenshot-launchpad.png" alt="OmniVoice Studio — Launchpad" width="100%"/>
</div>
> **Your voice is the most personal data you have. So why rent it back from a cloud?** Every mainstream voice tool ships your audio to someone else's server and bills you monthly for the privilege. OmniVoice Studio flips that: clone, design, dub, and dictate on your own hardware — 646 languages, no meter running, nothing leaving your machine.
<div align="center">
| 🔑 No API keys | 🙅 No accounts | ☁️ No cloud | 💳 No subscription |
|:---:|:---:|:---:|:---:|
| nothing to paste in | nothing to sign up for | your audio stays home | it's your computer |
</div>
> [!WARNING]
> **OmniVoice Studio is in active beta.** Things may break between releases. For the latest features and fixes, clone the repo and run from source rather than using pre-built installers. Bug reports and PRs are very welcome [open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) or [join Discord](https://discord.gg/bzQavDfVV9).
> **OmniVoice Studio is in active beta.** Things may break between releases — for the latest features and fixes, clone the repo and run from source rather than the pre-built installers. Bug reports and PRs are very welcome: [open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) or [join Discord](https://discord.gg/bzQavDfVV9).
<div align="center">
<br/>
@@ -54,79 +58,157 @@
<br/>
## Features
<a id="screenshots"></a>
## 📸 See it in action
<table>
<tr>
<td align="center" width="50%">
<img src="docs/screenshot-studio.png" alt="Studio" width="100%"/>
<br/><b>Studio</b><br/>
<sub>Generate &amp; clone in one workspace — a 3-second clip mirrors any voice, 646 languages, zero-shot.</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="Voice Design" width="100%"/>
<br/><b>Voice Design</b><br/>
<sub>Build new voices from scratch — gender, age, accent, pitch, emotion, dialect.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="Voice Gallery" width="100%"/>
<br/><b>Voice Gallery</b><br/>
<sub>Browse ready-made archetype voices with language filters — or build your own library.</sub>
</td>
<td align="center">
<img src="docs/screenshot-dub.png" alt="Video Dubbing" width="100%"/>
<br/><b>Video Dubbing</b><br/>
<sub>A real dub, end to end: 37 segments transcribed, translated to Bengali, re-voiced, and timed — ready to export as MP4.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-engines.png" alt="Settings — Engines" width="100%"/>
<br/><b>Settings → Engines</b><br/>
<sub>The engine compatibility matrix — 14 TTS engines with per-engine GPU preflight, no silent CPU fallback.</sub>
</td>
<td align="center">
<img src="docs/screenshot-settings.png" alt="Settings — Models" width="100%"/>
<br/><b>Settings → Models</b><br/>
<sub>One-click model store — auto-detects your platform (CUDA / MPS / CPU) and recommends the right models.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-openapi.png" alt="Settings — API Reference" width="100%"/>
<br/><b>API Reference</b><br/>
<sub>The full local REST API, embedded — every endpoint documented with copy-paste client snippets.</sub>
</td>
<td align="center">
<img src="docs/screenshot-updates.png" alt="Settings — What's New" width="100%"/>
<br/><b>What's New</b><br/>
<sub>In-app changelog reader — see exactly what shipped in each release without leaving the app.</sub>
</td>
</tr>
</table>
---
<a id="features"></a>
## ✨ Features
The eight headliners — and twelve more waiting under the fold.
<table>
<tr>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎙️ Voice Cloning</h3>
<p>3-second clip → mirror any voice.<br/><b>646 languages</b>, zero-shot.</p>
</td>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎨 Voice Design</h3>
<p>Gender, age, accent, pitch, speed,<br/>emotion, dialect — <b>dial it in</b>.</p>
</td>
<td align="center" width="33%">
<td align="center" width="25%">
<h3>🎬 Video Dubbing</h3>
<p>YouTube URL or file → transcribe →<br/>translate → re-voice → <b>MP4</b>.</p>
</td>
<td align="center" width="25%">
<h3>📖 Audiobook Editor</h3>
<p>Import text, EPUB, or PDF. Auto-chapter,<br/>loudnorm, metadata. Export <b>.m4b</b>.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🎭 Stories</h3>
<p>Multi-voice editor. Assign voices<br/>per-line, preview, <b>export full cast</b>.</p>
</td>
<td align="center" valign="top">
<h3>⌨️ Dictation Widget</h3>
<p><code>⌘+⇧+Space</code> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
<p><kbd>⌘</kbd>+<kbd>⇧</kbd>+<kbd>Space</kbd> from <b>any app</b>.<br/>Transcribes, auto-pastes, disappears.</p>
</td>
<td align="center" valign="top">
<h3>🔊 Vocal Isolation</h3>
<p>Demucs-powered. Splits speech<br/>from music, <b>keeps the background</b>.</p>
</td>
<td align="center" valign="top">
<h3>👥 Speaker Diarization</h3>
<p>Pyannote + WhisperX.<br/><b>Auto-identifies</b> who said what.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>📦 Batch Queue</h3>
<p>Drop <b>50 videos</b>, walk away.<br/>Progress bars per job.</p>
</td>
<td align="center" valign="top">
<h3>🤖 MCP Server</h3>
<p>Use OmniVoice from <b>Claude</b>,<br/>Cursor, or any MCP client.</p>
</td>
<td align="center" valign="top">
<h3>🛡️ AI Watermark</h3>
<p>AudioSeal (Meta). <b>Invisible</b>,<br/>survives compression.</p>
</td>
</tr>
<tr>
<td align="center" valign="top">
<h3>🔐 100% Local</h3>
<p>No keys, no cloud, no accounts.<br/><b>Your machine only</b>.</p>
</td>
<td align="center" valign="top">
<h3>⚡ GPU Auto-Detect</h3>
<p>CUDA · MPS · ROCm · CPU.<br/>≤8 GB? <b>Auto-offloads</b>.</p>
</td>
<td align="center" valign="top">
<h3>🧩 Extensible</h3>
<p>Subclass <code>TTSBackend</code>,<br/>add any engine in <b>~50 lines</b>.</p>
<h3>🤖 MCP Server</h3>
<p>Use OmniVoice from <b>Claude</b>,<br/>Cursor, or any MCP client.</p>
</td>
</tr>
</table>
<details>
<summary><b>…and 12 more</b> — isolation, diarization, batch, watermarking, diagnostics, and friends</summary>
<br/>
- 🔊 **Vocal Isolation** — Demucs-powered: splits speech from music and keeps the background bed.
- 👥 **Speaker Diarization** — Pyannote + WhisperX auto-identify who said what.
- 📦 **Batch Queue** — drop 50 videos, walk away; per-job progress bars.
- 🛡️ **AI Watermark** — AudioSeal (Meta): invisible, survives compression.
- 🔬 **Diagnostics** — self-check suite, error journal, scrubbed diagnostic bundles.
-**GPU Auto-Detect** — CUDA · MPS · ROCm (Linux, opt-in) · CPU; ≤8 GB VRAM auto-offloads.
- 🧭 **Engine routing** — preflight GPU check per engine; no silent CPU fallback.
- 🧩 **Extensible** — subclass `TTSBackend`, add any engine in ~50 lines.
- 🎒 **Portable personas** — export voices as `.ovsvoice` bundles: identity + watermark.
- ♾️ **Unlimited TTS** — sentence-chunked generation, no length cap, streaming via WebSocket.
- 🌐 **Remote backend** — point the UI at a remote server; Tailscale-friendly, bearer auth.
- 🧠 **Dictation + LLM** — local-LLM cleanup of transcripts, optional echo cancellation.
</details>
---
## Quickstart
<a id="quickstart"></a>
Per-OS install guides — pick yours and follow it end-to-end:
## ⚡ Quickstart
- **macOS** — [docs/install/macos.md](docs/install/macos.md)
- **Windows** — [docs/install/windows.md](docs/install/windows.md)
- **Linux** — [docs/install/linux.md](docs/install/linux.md)
- **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
<div align="center">
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<br/>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
<br/>
<sub><b>Intel Macs are not supported for the local backend:</b> the app UI installs, but the Python backend cannot run because PyTorch no longer ships Intel-Mac (x86_64) wheels (<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>) — see <a href="docs/install/macos.md">docs/install/macos.md</a>.</sub>
</div>
Stuck? Run the built-in self-check first — **Settings → About → "Run
Pick your OS and follow the guide end-to-end:
- 🍎 **macOS** — [docs/install/macos.md](docs/install/macos.md)
- 🪟 **Windows** — [docs/install/windows.md](docs/install/windows.md)
- 🐧 **Linux** — [docs/install/linux.md](docs/install/linux.md)
- 🐳 **Docker** — [docs/install/docker.md](docs/install/docker.md) · [Docker Hub: `palashdeb/omnivoice-studio`](https://hub.docker.com/r/palashdeb/omnivoice-studio)
<details>
<summary><b>🧰 Stuck? Self-checks, tokens &amp; restricted networks</b></summary>
<br/>
Run the built-in self-check first — **Settings → About → "Run
self-check"** in the app, or `uv run python backend/main.py --diagnose` from
a checkout (`--deep` also test-loads the active engine). Then see
[docs/install/troubleshooting.md](docs/install/troubleshooting.md) for the
@@ -141,57 +223,13 @@ diarization-specific gating, see
speed, the ⚡ fast-download (Xet) status, and restricted-network / mirror
options, see [docs/downloading-models.md](docs/downloading-models.md).
## Screenshots
<table>
<tr>
<td align="center" width="50%">
<img src="docs/screenshot-clone.png" alt="Voice Clone" width="100%"/>
<br/><b>Voice Clone</b><br/>
<sub>Drop a 3-second clip → mirror any voice. 646 languages, zero-shot.</sub>
</td>
<td align="center" width="50%">
<img src="docs/screenshot-design.png" alt="Voice Design" width="100%"/>
<br/><b>Voice Design</b><br/>
<sub>Build new voices from scratch — gender, age, accent, pitch, style.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-dub.png" alt="Video Dubbing" width="100%"/>
<br/><b>Video Dubbing</b><br/>
<sub>Upload or paste a YouTube URL. Transcribe, translate, re-voice, export.</sub>
</td>
<td align="center">
<img src="docs/screenshot-gallery.png" alt="Voice Gallery" width="100%"/>
<br/><b>Voice Gallery</b><br/>
<sub>Search YouTube, browse categories, download clips, build your library.</sub>
</td>
</tr>
<tr>
<td align="center">
<img src="docs/screenshot-settings.png" alt="Settings — Models" width="100%"/>
<br/><b>Settings → Models</b><br/>
<sub>15 models. One-click install. Auto-detects your platform (CUDA / MPS / CPU).</sub>
</td>
<td align="center">
<img src="docs/screenshot-libraryprojects.png" alt="Projects" width="100%"/>
<br/><b>Projects</b><br/>
<sub>Dub projects, voice profiles, generation history, exports — all searchable.</sub>
</td>
</tr>
<tr>
<td align="center" colspan="2">
<img src="docs/screenshot-logs.png" alt="Settings — Logs" width="100%"/>
<br/><b>Settings → Logs</b><br/>
<sub>Live backend, frontend, and Tauri runtime logs. Filter, refresh, clear.</sub>
</td>
</tr>
</table>
</details>
---
## Why OmniVoice Studio?
<a id="why-ovs"></a>
## 💡 Why OmniVoice?
ElevenLabs charges **$5$330/mo** and processes your audio on their servers. OmniVoice Studio runs **on your hardware, with no usage limits.**
@@ -200,15 +238,20 @@ ElevenLabs charges **$5$330/mo** and processes your audio on their servers. O
| **Pricing** | $5$330/mo, per-character billing | Free & open-source (AGPL-3.0) · [Commercial license](#license) for proprietary use |
| **Voice Cloning** | ✅ 3s clip | ✅ 3s clip, zero-shot |
| **Voice Design** | ✅ Gender, age | ✅ Gender, age, accent, pitch, style, dialect |
| **Audiobook / Stories** | ❌ | ✅ Full audiobook editor + multi-voice stories (EPUB/PDF import, .m4b export) |
| **Languages** | 32 | **646** |
| **Video Dubbing** | ✅ Cloud-only | ✅ Fully local |
| **Data Privacy** | Audio sent to cloud | **Nothing leaves your machine** |
| **API Keys** | Required | Not needed |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm · CPU |
| **GPU Support** | N/A (cloud) | CUDA · Apple Silicon · ROCm (Linux) · CPU |
| **Desktop App** | ❌ | ✅ macOS · Windows · Linux |
| **TTS Engines** | 1 | **14** (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS) |
| **ASR Engines** | 1 | **9** (WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR, isolated Faster-Whisper, sherpa-onnx live dictation) |
| **MCP Server** | ❌ | ✅ Use from Claude, Cursor, any MCP client |
| **Self-check** | ❌ | ✅ Diagnostics suite, error journal, scrubbed debug bundles |
| **Customizable** | ❌ Closed | ✅ Fork it, extend it, ship it |
OmniVoice Studio gives you professional-grade AI tools without the subscription or the cloud.
Professional-grade voice AI, minus the subscription and the cloud.
<div align="center">
<br/>
@@ -219,125 +262,271 @@ OmniVoice Studio gives you professional-grade AI tools without the subscription
---
## System Requirements
## 🖥️ System Requirements
| | **Minimum** | **Recommended** |
|---|---|---|
| **OS** | Windows 10, macOS 12+, Ubuntu 20.04+ | Any modern 64-bit OS |
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
| **RAM** | 8 GB | 16 GB+ |
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
| **Python** | 3.10+ (managed by `uv`) | 3.113.12 |
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm |
| **GPU** | Optional — CPU works | NVIDIA CUDA · Apple Silicon MPS · AMD ROCm (Linux only) |
> [!TIP]
> On GPUs with **≤8 GB VRAM**, OmniVoice automatically offloads TTS to CPU during transcription — no config needed. A dedicated GPU is not required; the entire pipeline runs on CPU (just slower).
### TTS Engines
> [!NOTE]
> **AMD GPUs:** ROCm acceleration is **Linux-only and opt-in** — pick **"AMD GPU (ROCm)"** on the first-run setup screen or set `OMNIVOICE_TORCH_VARIANT=rocm` ([docs/install/linux.md](docs/install/linux.md#amd-gpu-rocm)). **On Windows, AMD GPUs (incl. Ryzen AI iGPUs) run CPU-only**: PyTorch has no Windows ROCm wheels, so Windows GPU acceleration is NVIDIA/CUDA-only ([docs/install/windows.md](docs/install/windows.md#gpu-support)).
OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is always available; additional engines are opt-in and auto-detected. Switch engines in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
> [!IMPORTANT]
> **macOS Intel (x86_64) is unsupported for the local backend:** the app UI installs, but the Python backend cannot run because PyTorch no longer ships Intel-Mac wheels ([#889](https://github.com/debpalash/OmniVoice-Studio/issues/889)). Intel-Mac users can still point the UI at a remote backend on another machine — see [docs/install/macos.md](docs/install/macos.md).
<a id="tts-engines"></a>
### 🗣️ TTS Engines
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var — the selection applies everywhere synthesis happens: single-clip generation, Voice Cloning, Video Dubbing, and Batch TTS.
<details>
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
<br/>
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|--------|:---------:|:-----:|:--------:|:-----:|:---------:|:-------:|:-------:|
| **OmniVoice** (default) | 600+ | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Built-in |
| **CosyVoice 3** | 9 + 18 dialects | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | | ✅ Native | ❌ | Varies |
| **GPT-SoVITS** | 5 | ✅ | — | ✅ CUDA/CPU | | ✅ CUDA/CPU | MIT |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | | | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MOSS-TTS-Nano** | 20 | ✅ | | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **KittenTTS** | English | | | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
| **MLX-Audio** (Kokoro, Qwen3-TTS, CSM, Dia, …) | Multi | Varies | Varies | ❌ | ✅ Native | ❌ | Varies |
| **Sherpa-ONNX** | 20+ | — | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **IndexTTS 2** ⚡ | Multi | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
| **OmniVoice GGUF** ⚡ | 600+ | ✅ | ✅ | ✅ CPU | ✅ CPU | ✅ CPU | Built-in |
| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ (8B) | 31 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ (2B) | 24 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only.
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
>
> **Clone** matters beyond single-clip generation: Video Dubbing (and any Batch job with a pinned voice) needs reference-audio cloning to preserve speaker identity, so picking a Clone-less engine (KittenTTS, Sherpa-ONNX, Supertonic 3) as the active engine fails those jobs up front with an actionable message instead of silently falling back to OmniVoice.
>
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
### ASR Engines
</details>
OmniVoice ships a multi-engine ASR (speech-to-text) backend that powers dictation, video dubbing, and subtitle generation — all fully local. **WhisperX** is the cross-platform default; the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
<a id="asr-engines"></a>
### 🎧 ASR Engines
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines** (the ASR Engines table — same picker TTS has), or pin one with the `OMNIVOICE_ASR_BACKEND` env var (the env var wins over the Settings pick). Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
<details>
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
<br/>
| Engine | `OMNIVOICE_ASR_BACKEND` | Languages | Best for |
|--------|-------------------------|:---------:|----------|
| **WhisperX** (default) | `whisperx` | ~100 | Dubbing & subtitles — word-level timing via wav2vec2 forced alignment |
| **Faster-Whisper** | `faster-whisper` | ~100 | Fast transcription on Linux / macOS / Windows (CTranslate2) |
| **Faster-Whisper (isolated)** | `faster-whisper-isolated` | ~100 | Same as Faster-Whisper but crash-isolated in a subprocess — an ASR crash won't take down the app |
| **MLX Whisper** | `mlx-whisper` | ~100 | Native Apple Silicon speed (Apple MLX / Metal) |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers |
| **Parakeet TDT** | `nemo-parakeet` | English + 25 EU | SOTA English accuracy, auto language detection (NVIDIA NeMo, GPU only) |
| **PyTorch Whisper** | `pytorch-whisper` | ~100 | CUDA / CPU fallback via 🤗 Transformers (no cuDNN 8 needed) |
| **Parakeet TDT** | `nemo-parakeet` | English + 25 EU | SOTA accuracy at ~10× realtime even on CPU, auto language detection (NVIDIA NeMo, CUDA/CPU) |
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. Every engine runs on-device — no API keys, no cloud.
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
> **GPU without efficient float16?** On older NVIDIA GPUs (Maxwell/Pascal, GTX 16xx) or after a CTranslate2/cuDNN mismatch, the CTranslate2 ASR engines (WhisperX, Faster-Whisper) can't run `float16` and OmniVoice automatically retries on `int8` — no config needed. If transcription still fails, pin the compute type with the `ASR_COMPUTE_TYPE` env var (escape hatch): `ASR_COMPUTE_TYPE=int8` (or `float32` for CPU). Set it to `int8` and restart the backend.
</details>
---
## Architecture
## 🏗️ Architecture
```
┌─────────────────────────────────────────────────┐
│ Frontend (React) │
│ DubTab · VoicePreview · BatchQueue · Gallery │
├─────────────────────────────────────────────────┤
│ Backend (FastAPI) │
97 API endpoints · SSE streaming · SQLite
├──────────┬──────────┬──────────┬────────────────┤
│ WhisperX │ Demucs │OmniVoice │ Pyannote │
ASR │ Source TTS │ Diarization
│ Sep.
└──────────┴──────────┴──────────┴────────────────┘
CUDA / MPS / ROCm / CPU (auto-detected)
┌─────────────────────────────────────────────────────────────
Frontend (React)
│ DubTab · VoiceConsole · Stories · Audiobook · Gallery
│ Dictation · BatchQueue · Diagnostics · MCP Client │
├─────────────────────────────────────────────────────────────┤
Backend (FastAPI)
│ 100+ API endpoints · SSE+WSS streaming · SQLite │
├──────────┬──────────┬──────────┬──────────┬────────────────┤
WhisperX │ Demucs │OmniVoice │ Pyannote │ Engine Routing
(+7 ASR │ Source(+10Diariz- │ ↳ GPU preflight
│ engines) │ Sep. │ TTS) │ ation │ ↳ No silent CPU │
└──────────┴──────────┴──────────┴──────────┴────────────────┘
CUDA / MPS / ROCm / CPU (auto-detected + routed)
```
<a id="openai-api"></a>
## 🔌 OpenAI-compatible API
Already have a script, agent, or tool that speaks OpenAI's audio API? Point it at `http://localhost:3900/v1` — no key needed, no code changes. The backend ships a drop-in surface for the audio endpoints, wired to whichever TTS/ASR engine you have active (and yes, `voice` accepts your cloned voice-profile IDs).
| Endpoint | What it does |
|---|---|
| `POST /v1/audio/speech` | TTS — text in; `mp3` / `wav` / `flac` / `opus` / `pcm` out. `tts-1` / `tts-1-hd` map to your active engine; OpenAI voice names (`alloy`, …) are accepted. |
| `POST /v1/audio/transcriptions` | STT — audio file in; `json`, `text`, `verbose_json`, `srt`, or `vtt` out. `whisper-1` maps to your active ASR engine. |
| `GET /v1/audio/voices` | OmniVoice extension — lists every voice profile and engine, so clients can discover your clones. |
```sh
curl http://localhost:3900/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model": "tts-1", "voice": "alloy", "input": "Generated on my own hardware.", "response_format": "wav"}' \
--output speech.wav
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3900/v1", api_key="none") # any string works — nothing checks it
result = client.audio.transcriptions.create(model="whisper-1", file=open("clip.wav", "rb"))
print(result.text)
```
Want the whole surface (100+ endpoints)? The full REST API reference is embedded in the app — **Settings → OpenAPI Reference** (Scalar-powered), or the `{}` button in the footer.
### 📓 Run on Google Colab (community)
No local GPU? A community member ([@shakib30](https://github.com/shakib30)) maintains a working Colab notebook: [shakib30/OmniVoice-Studio-google-colab](https://github.com/shakib30/OmniVoice-Studio-google-colab). Community-maintained — issues with the notebook go there; issues with OmniVoice itself come here.
### 🤝 Agent Skills
Teach your AI agent (Claude Code, Cursor, Codex, …) to use OmniVoice with one command:
```sh
npx skills add debpalash/omnivoice-studio
```
Ships two [skills](https://skills.sh): **`omnivoice`** — speak and transcribe through your local install (including your cloned voices) from any agent, free and offline; and **`oss-maintainer`** — the maintainer methodology this project is run with, for anyone running their own OSS project with an agent.
---
## Roadmap
### ✅ Shipped
| Category | Features |
|----------|----------|
| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS |
| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags |
| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export |
| **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
| **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading |
| **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system |
| **Real-time Events** | WebSocket event bus — instant sidebar refresh on data mutations, exponential backoff reconnect |
| **State Management** | Zustand store migration — `uiSlice`, `pillSlice`, `dubSlice`, `generateSlice`, `prefsSlice`, `glossarySlice` |
| **Desktop** | Cross-platform Tauri installers (macOS DMG, Windows MSI, Linux deb/AppImage), auto-update infrastructure |
| **Windows Hardening** | Cross-platform log paths, Triton workaround, HF symlink bypass, 300s health check timeout |
| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste |
| **Batch Pipeline** | Full batch TTS: extract → transcribe → translate → generate → mix → export, with live progress tracking |
## 🗺️ Roadmap
### 🔜 Up Next
- 🎬 **Lip-sync v2** — visual speech timing with wav2lip
- 📖 **Audiobook Editor** — chapter-aware long-form narration
- 🌐 **Hosted Demo** — try OmniVoice without installing anything
- 🔌 **Plugin Marketplace** — community-contributed TTS engines and effects
- 🎵 **Real-time Voice Changer** — live microphone transformation during calls
<details>
<summary><b>✅ Everything shipped so far</b> — the receipts, by category</summary>
<br/>
| Category | Features |
|----------|----------|
| **Longform** | Audiobook editor (text/EPUB/PDF → chaptered .m4b), Stories multi-voice editor, two-pass loudnorm mastering, crash-resume for interrupted renders, pronunciation control + SSML-lite prosody |
| **Dubbing** | Full pipeline (transcribe→translate→synthesize→mux), scene-aware splitting, lip-sync scoring, streaming TTS, per-speaker voice assignment, Smart Fit timing + second-pass QC, dedicated Dub home |
| **Voice** | Zero-shot cloning, voice design, A/B comparison, voice preview widget, gallery with favorites/tags, portable persona bundles (`.ovsvoice`), voice console workspace |
| **Audio** | Demucs vocal isolation, per-segment gain, selective track export, stem/SRT/VTT/MP3 export, unlimited-length TTS via sentence-chunked generation |
| **Multi-Lang** | Multi-language batch picker, batch dubbing queue with sequential GPU execution |
| **Diarization** | Pyannote ML diarization, auto speaker clone extraction, per-speaker voice assignment |
| **ASR** | 9 engines (WhisperX, Faster-Whisper, isolated Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet TDT, Moonshine, FunASR/SenseVoice, sherpa-onnx live dictation), crash-isolated subprocess backend |
| **TTS** | 14 engines (OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, Sherpa-ONNX, + lazy: IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS), engine routing with GPU preflight |
| **Infra** | Docker deployment, CUDA/MPS/ROCm auto-detect, cuDNN 8 compat, VRAM-aware model offloading, engine routing (no silent CPU fallback), diagnostics suite & error journal, restricted-network mirror support |
| **AI Provenance** | AudioSeal invisible watermarking (SynthID-like), video logo overlay, watermark detection API |
| **UX** | Undo/redo, keyboard shortcuts, drag-and-drop, session persistence, glassmorphism design system, UI scale fix for Linux/WebKitGTK |
| **Real-time Events** | WebSocket event bus — instant sidebar refresh on data mutations, exponential backoff reconnect |
| **State Management** | Zustand store migration — `uiSlice`, `pillSlice`, `dubSlice`, `generateSlice`, `prefsSlice`, `glossarySlice` |
| **Desktop** | Cross-platform Tauri installers (macOS DMG — Apple Silicon; Intel unsupported for the local backend, #889 — Windows MSI, Linux deb/AppImage), auto-update infrastructure, single-instance enforcement, close-to-tray, macOS Gatekeeper fix |
| **Dictation** | Global system-wide hotkey (`⌘+⇧+Space`), frameless floating widget, streaming ASR via WebSocket, auto-paste, customizable hotkey, local-LLM transcript refinement |
| **Batch Pipeline** | Full batch TTS: extract → transcribe → translate → generate → mix → export, with live progress tracking |
| **MCP Server** | OmniVoice as a local TTS/STT provider for Claude, Cursor, and any MCP client |
| **Remote Backend** | Point the desktop UI at a remote backend URL with bearer auth (Tailscale-documented) |
| **Reliability** | Stall watchdog on bootstrap splash, per-engine GPU compatibility matrix, actionable errors for non-executable engine binaries, setuptools auto-repair |
</details>
---
## Community
<a id="sponsor--donate"></a>
## 💜 Sponsor / Donate
OmniVoice Studio is built by one developer using Claude Code and AI agents — and the agent bills are real. Over the last three months I've spent thousands of dollars on Claude subscriptions to keep the features shipping, the bugs fixed, and your issues answered. If OmniVoice has created value for you, helping cover those bills means I can keep developing full-time.
<div align="center">
**This month's agent bill fund**
<img src="https://img.shields.io/badge/raised_%2410_of_%24200-5%25-EAB308?style=for-the-badge" alt="$10 / $200 raised" />
<br/><br/>
<a href="https://ko-fi.com/debpalash"><img src="https://img.shields.io/badge/Ko--fi-Support_❤️-FF5E5B?style=for-the-badge&logo=ko-fi&logoColor=white" alt="Ko-fi" /></a>
&nbsp;&nbsp;
<a href="https://paypal.me/palashCoder"><img src="https://img.shields.io/badge/PayPal-Donate-00457C?style=for-the-badge&logo=paypal&logoColor=white" alt="PayPal" /></a>
<br/>
<sub>Every dollar goes directly to agent bills — keeping OmniVoice development continuous.</sub>
</div>
<a id="sponsors"></a>
### 🌟 Sponsors
OmniVoice is **free** and **AGPL-3.0** — no paid tier, no SaaS revenue. Sponsors keep development going, and in return get a logo slot here, in the app, and (for top tiers) on the project website. It's a thank-you, never a paywall. **[See tiers & become a sponsor →](SPONSORS.md)**
<div align="center">
<!-- SPONSORS:START — logo slots are filled here as sponsors come aboard; see SPONSORS.md -->
**Your logo here** — [become a sponsor](SPONSORS.md)
<!-- SPONSORS:END -->
</div>
<sub>💡 GitHub also shows a **Sponsor** button at the top of this repo, wired to the same links via <a href=".github/FUNDING.yml"><code>.github/FUNDING.yml</code></a>.</sub>
---
## 💬 Community
<div align="center">
<a href="https://discord.gg/bzQavDfVV9"><img src="https://img.shields.io/badge/💬_Discord-Join_Community-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord" /></a>
<br/>
<sub>We respond to setup questions within hours, not days.</sub>
</div>
<details>
<summary><b>What happens in there</b></summary>
<br/>
| Channel | What happens there |
|---------|--------------------|
| `#showcase` | Members share their dubs, clones, and voice designs |
| `#help` | Setup issues, GPU troubleshooting, model questions |
| `#feature-requests` | Vote on what gets built next |
| `#dev` | Architecture discussions, PR reviews, engine integrations |
| `#announcements` | Release notes, breaking changes, early access |
| `#announcements` | Release news and the big moments — new versions land here first |
| `#releases` + `#changelog` | Every build and exactly what's inside it |
| `#issues` | Bug reports as forum posts — triaged straight into GitHub issues |
| `#ideas` | Feature requests, discussed and voted on |
| `#discuss-ideas` | Design talk before things get built |
| `#general` | Setup help, GPU troubleshooting, and showing off your dubs |
**[→ Join the Discord](https://discord.gg/bzQavDfVV9)** — we respond to setup questions within hours, not days.
</details>
---
## Contributing
<a id="contributing"></a>
We welcome contributions of all kinds — bug fixes, new TTS engine adapters, UI improvements, docs, and translations.
## 🤝 Contributing
Yes please — bug fixes, new TTS engine adapters, UI improvements, docs, translations. All of it.
- 📖 Read the **[Contributing Guide](CONTRIBUTING.md)** for setup, code style, and PR workflow
- 🐛 Browse [good first issues](https://github.com/debpalash/OmniVoice-Studio/labels/good%20first%20issue)
@@ -345,18 +534,26 @@ We welcome contributions of all kinds — bug fixes, new TTS engine adapters, UI
---
## FAQ
## FAQ
<details>
<summary><b>Is this really as good as ElevenLabs?</b></summary>
<br/>
For voice cloning and dubbing, yes — OmniVoice uses a state-of-the-art diffusion TTS model with 646 languages (ElevenLabs supports 32). Quality is comparable for most use cases. Where ElevenLabs wins is in their polished cloud API and pre-made voice library. OmniVoice wins on privacy, cost, language coverage, and customizability.
Honest answer: <b>it depends on what you're doing.</b>
<b>Where OmniVoice is genuinely competitive:</b> voice cloning from a clean reference clip (state-of-the-art open diffusion TTS), language coverage (646 languages vs. their 32), and everything structural — no per-character billing, no usage caps, no audio leaving your machine, full pipeline customizability (10 TTS engines, 10 ASR engines, your choice of translation).
<b>Where ElevenLabs still wins:</b> out-of-the-box consistency and polish, especially for English TTS. Their one model is heavily tuned; our quality depends on which engine you pick, your hardware, and — for cloning — the reference audio (a dry, close-mic clip clones dramatically better than a noisy or echoey one).
<b>For dubbing specifically:</b> a dub is a chain — transcription → translation → cloning → synthesis — and the output is only as good as its weakest link on <i>your</i> source material. Noisy or accented source audio degrades transcription, which degrades everything downstream; some language pairs translate better than others. If parts of a dub come out incoherent, check the segment table's <i>original</i> text first: if the transcription is already wrong there, switch the ASR engine (Settings → Engines) or use cleaner source audio — that's usually the fix, not the voice.
Try it on your real material — it's free and takes one download. Many users find it replaces ElevenLabs outright; some keep both for different jobs. Both outcomes are fine with us.
</details>
<details>
<summary><b>Does it work on Apple Silicon (M1/M2/M3/M4)?</b></summary>
<br/>
Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are available for faster transcription on Apple hardware.
Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are available for faster transcription on Apple hardware. <b>Intel Macs are not supported</b>: the app UI installs, but the local Python backend cannot run because PyTorch no longer ships Intel-Mac wheels (<a href="https://github.com/debpalash/OmniVoice-Studio/issues/889">#889</a>) — an Intel Mac can only be used with a remote backend.
</details>
<details>
@@ -380,12 +577,14 @@ Yes. MPS acceleration is auto-detected. MLX-optimized Whisper models are availab
<details>
<summary><b>Can I add my own TTS engine?</b></summary>
<br/>
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary at the bottom. Six engines are built in: OmniVoice, CosyVoice, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, and KittenTTS. See the <a href="#tts-engines">TTS Engines</a> section for details.
Yes. OmniVoice uses a <b>built-in backend registry</b>. To add an engine in ~50 lines, subclass <code>TTSBackend</code> in <code>backend/services/tts_backend.py</code> and add it to the <code>_REGISTRY</code> dictionary. Fourteen engines are built in: OmniVoice, CosyVoice 3, GPT-SoVITS, MLX-Audio (14+ sub-engines), VoxCPM2, MOSS-TTS-Nano, KittenTTS, Sherpa-ONNX, plus lazy-registered IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, and Confucius4-TTS. See the <a href="#tts-engines">TTS Engines</a> section for details.
</details>
---
## License
<a id="license"></a>
## 📜 License
OmniVoice Studio is free and open-source software under the [**GNU Affero General Public License v3.0 (AGPL-3.0)**](https://www.gnu.org/licenses/agpl-3.0.html).
@@ -397,7 +596,7 @@ The bundled `omnivoice/` TTS model by Han Zhu remains Apache-2.0 upstream. See [
---
## Acknowledgments
## 🙏 Acknowledgments
OmniVoice Studio is built on the shoulders of exceptional open-source work:
@@ -410,6 +609,20 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
| [**CTranslate2**](https://github.com/OpenNMT/CTranslate2) | Optimized Transformer inference on CPU and GPU |
| [**AudioSeal (Meta)**](https://github.com/facebookresearch/audioseal) | Invisible neural audio watermarking for AI provenance |
| [**Tauri**](https://tauri.app) | Native desktop app framework |
| [**Supertone / Supertonic 3**](https://huggingface.co/Supertone/supertonic-3) | ONNX TTS engine — 31 languages, CPU-efficient |
| [**Sherpa-ONNX**](https://github.com/k2-fsa/sherpa-onnx) | WASM-ready universal TTS/ASR runtime |
| [**GPT-SoVITS**](https://github.com/RVC-Boss/GPT-SoVITS) | Zero-shot TTS engine — 5 languages, RTF 0.014 |
---
## 🧰 More local open-source from the maker
Like the local-first philosophy? It runs in the family:
| Project | What it is |
|---------|------------|
| [**Opal**](https://github.com/debpalash/Opal) 💠 | **Play everything.** The evolved media player for the next decades of entertainment — video, anime, comics, torrents, Jellyfin/Plex, with local AI built in. |
| [**memxt**](https://github.com/debpalash/memxt) 🧠 | **The fastest benchmarked open-source AI memory system.** 100% local memory for AI agents, with MCP support. |
---
@@ -419,7 +632,8 @@ OmniVoice Studio is built on the shoulders of exceptional open-source work:
If you read this far, you're our kind of person.<br/>
**[⭐ Star this repo](https://github.com/debpalash/OmniVoice-Studio)** so others can find it too.<br/>
**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.
**[💬 Join the Discord](https://discord.gg/bzQavDfVV9)** to share what you build.<br/>
**[❤️ Support development](https://ko-fi.com/debpalash)** — fund the AI agent bills that keep OmniVoice shipping.
<br/>
+4
View File
@@ -372,9 +372,13 @@ OmniVoice 配备多引擎 TTS 后端。默认引擎(OmniVoice)始终可用
| **MLX-Audio**Kokoro, Qwen3-TTS, CSM, Dia 等) | 多语言 | 因引擎而异 | 因引擎而异 | ❌ | ✅ 原生 | ❌ | 因引擎而异 |
| **VoxCPM2** | 30 | ✅ | ✅ | ✅ CUDA/CPU | ✅ MPS | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-Nano** | 20 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **MOSS-TTS-v1.5**(8B,可选装) | 31 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts**(2B,可选装) | 24 | ✅ | ❌ | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **KittenTTS** | 英语 | ❌ | ❌ | ✅ CPU | ✅ CPU | ✅ CPU | MIT |
> **CUDA** = GPU 加速 · **MPS** = Apple Silicon Metal · **CPU** = 随处可运行,大模型较慢 · KittenTTS 和 MOSS-TTS-Nano 可在 CPU 上实时运行 · MLX-Audio 仅限 Apple Silicon。
>
> **MOSS-TTS-v1.5**8B,约 16 GB 权重)和 **dots.tts**2B,约 9 GB 权重)是重量级可选引擎,从本地克隆在独立 venv 中运行——参见 [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) 和 [dots.tts](docs/engines/dots-tts.md)。两者均不支持 Apple Silicon **MPS**(上游仅支持 CUDA/CPU;在 Mac 上以 CPU 运行)。dots.tts 上游仅支持 Linux/macOS——无 Windows 路径。
---
+119
View File
@@ -0,0 +1,119 @@
<div align="center">
<img src="docs/logo.png" alt="OmniVoice Logo" width="96" />
<h1>Sponsor OmniVoice Studio</h1>
<p><b>Keep the open-source ElevenLabs alternative free, local, and shipping.</b></p>
</div>
---
## Why sponsor?
OmniVoice Studio is built by one developer, in the open, using Claude Code and AI agents — and the agent bills are real. Over the last few months I've spent thousands of dollars on Claude subscriptions to keep features shipping, bugs fixed, and your issues answered.
OmniVoice is **free**, **fully local**, and **AGPL-3.0**. There's no paid tier, no accounts, no cloud, and no SaaS revenue — nothing runs on a server we bill you for, because nothing runs on a server at all. That's the whole point, and it's also why there's no recurring revenue to fund development. Sponsorship is what makes continued full-time work possible.
If OmniVoice has created value for you or your company, sponsoring means the next release keeps coming — and you get a thank-you (and, at most tiers, a logo slot) in return.
### Where your money goes
Every dollar goes to the cost of building OmniVoice — chiefly the **AI agent bills that keep it shipping** (Claude subscriptions and API usage), plus the occasional signing certificate, test hardware, and model-hosting costs. It is not a salary top-up; it's what keeps the lights on for continuous development.
---
## Sponsorship tiers
Tiers are about **visibility and gratitude** — what you get is placement, not gated features (see [Not a paywall](#not-a-paywall)). Higher tiers include everything in the tiers below them.
| Tier | Suggested monthly | What you get |
|------|-------------------|--------------|
| **🥉 Backer** | _set by owner_ <!-- OWNER: set amounts --> | Your name or handle listed in the **Backers** section of this file, with a link of your choice. |
| **🟫 Bronze** | _set by owner_ <!-- OWNER: set amounts --> | Everything above, **plus** a small logo in `SPONSORS.md` **and** in the README [Sponsors section](README.md#sponsors). |
| **🥈 Silver** | _set by owner_ <!-- OWNER: set amounts --> | Everything above, **plus** your logo in the **README** and in the app's **in-app Sponsors page footer** (as that page ships). |
| **🥇 Gold** | _set by owner_ <!-- OWNER: set amounts --> | Everything above, **plus** a **prominent logo slot** and link on the project **website / landing page**. |
> **Amounts are set by the maintainer** — look for the `<!-- OWNER: set amounts -->` markers in this file's source. If you don't see a price that fits, say so in your inquiry; custom and annual arrangements are welcome.
Placements marked "as that page ships" (the in-app Sponsors page and the project website) are on the near-term roadmap. Until they exist, Silver/Gold logos live in `SPONSORS.md` and the README, and are added to the app and site the moment those land — no re-application needed.
---
## How to become a sponsor
**1. Open a sponsorship inquiry (recommended).** This opens a short GitHub form (name/org, logo, tier, contact) so we can get you set up:
> **[→ Open a sponsorship inquiry](https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml)**
**2. Or start recurring support directly:**
- **Ko-fi (recurring or one-time):** [ko-fi.com/debpalash](https://ko-fi.com/debpalash)
- **PayPal (one-time):** [paypal.me/palashCoder](https://paypal.me/palashCoder)
If you sponsor via Ko-fi/PayPal and want a logo slot, still open an inquiry (or drop a note there) so we know who to credit and where to link.
**3. Prefer to talk first?** Reach out directly:
- Email: <!-- OWNER: add your sponsor contact email here if you want one public -->
- Or ask in the `#dev` / `#announcements` channels on [Discord](https://discord.gg/bzQavDfVV9).
---
## Logo & asset guidelines
To make your logo look sharp everywhere (README on GitHub, the in-app page, the website), please send:
- **Format:** **SVG preferred** (scales cleanly); otherwise **PNG at 2× resolution**.
- **Background:** **transparent** — no baked-in white/black box.
- **Contrast:** send a variant that stays legible on **both light and dark** backgrounds, or one light-mode and one dark-mode file (GitHub and the app both render in either theme).
- **Dimensions:** legible at **~40px tall**; keep the wordmark within roughly **480px wide**. Landscape/wordmark shapes work best in the README row.
- **File size:** keep SVGs under ~50 KB and PNGs under ~100 KB.
- **Link target:** the destination URL you want the logo to point to (usually your homepage).
**How your logo gets added:**
- **Easiest:** attach the asset and link in your [sponsorship inquiry](https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml) — the maintainer places it.
- **Or open a PR:** add your asset under `docs/sponsors/` and an entry to the tables in this file. Silver/Gold logos are also wired into the app's in-app Sponsors page (via the `sponsors.js` manifest) and the project website as those surfaces ship.
By sponsoring you confirm you have the right to use the submitted logo and grant OmniVoice permission to display it in the contexts above. We won't alter your logo beyond scaling, and we'll remove it promptly on request.
---
## Current sponsors
OmniVoice doesn't have any sponsors yet — **you could be the first.** These slots fill in as sponsors come aboard.
### 🥇 Gold
_Be the first Gold sponsor — [claim this slot](#how-to-become-a-sponsor)._
### 🥈 Silver
_Open — [become a Silver sponsor](#how-to-become-a-sponsor)._
### 🟫 Bronze
_Open — [become a Bronze sponsor](#how-to-become-a-sponsor)._
### 🥉 Backers
_Open — [become a Backer](#how-to-become-a-sponsor)._
<!-- When a sponsor joins, add them to the matching section above:
- Logo tiers (Bronze+): <a href="https://sponsor.example"><img src="docs/sponsors/name.svg" alt="Name" height="48" /></a>
- Backers: - [Name / handle](https://link) -->
---
## Not a paywall
Sponsorship is a **thank-you, never a paywall.**
Every feature of OmniVoice Studio is and will remain **free** and **open-source under [AGPL-3.0](LICENSE)**. Sponsors do **not** get private builds, gated features, license exceptions, or anything that degrades the experience for people who don't (or can't) pay. What sponsors get is **visibility and our gratitude** — and the knowledge that they're directly funding the next release.
OmniVoice stays local-first and fully functional with zero dollars spent. Sponsoring just helps it keep getting better, faster.
---
<div align="center">
<sub>Thank you for keeping local-first voice AI alive and free. ❤️</sub><br/>
<sub>Questions? <a href="https://github.com/debpalash/OmniVoice-Studio/issues/new?template=sponsor.yml">Open an inquiry</a> · <a href="https://discord.gg/bzQavDfVV9">Discord</a></sub>
</div>
+15 -1
View File
@@ -13,12 +13,18 @@
# Run: uv run pyinstaller backend.spec --noconfirm --clean
import platform
import sys
from PyInstaller.utils.hooks import collect_data_files, collect_all, collect_submodules
from PyInstaller.utils.hooks import collect_data_files, collect_all, collect_submodules, copy_metadata
IS_MAC_ARM = sys.platform == "darwin" and platform.machine() == "arm64"
datas = []
binaries = []
# Bundle the omnivoice package's .dist-info so importlib.metadata.version()
# resolves inside the frozen build. Without it the backend can't read its own
# version and falls back to the literal in backend/core/version.py — which is
# how a 0.3.6 desktop build shipped reporting "0.3.5" in About + bug reports.
datas += copy_metadata('omnivoice')
hiddenimports = [
# Web stack
'uvicorn', 'uvicorn.logging', 'uvicorn.loops', 'uvicorn.loops.auto',
@@ -27,6 +33,14 @@ hiddenimports = [
'uvicorn.lifespan', 'uvicorn.lifespan.on',
'fastapi', 'fastapi.responses', 'starlette',
'multipart',
# SOCKS proxy support (#959). httpx imports socksio lazily inside a
# try/except (only when a socks5:// proxy env var is set), so
# PyInstaller's static tracer never sees it — without this entry the
# frozen installers keep raising "Using SOCKS proxy, but the 'socksio'
# package is not installed" on every model load under a SOCKS proxy,
# even though pyproject.toml ships the package. Guarded by
# tests/test_socks_proxy.py.
'socksio',
# Core
'uuid', 'asyncio',
+7 -6
View File
@@ -18,7 +18,6 @@ Design notes
"""
from __future__ import annotations
import asyncio
import hashlib
import logging
import os
@@ -137,7 +136,7 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
from api.routers.generation import ( # noqa: WPS433 — intentional lazy import
get_model,
_run_inference,
_gpu_pool,
run_on_gpu_pool_guarded,
_safe_torchaudio_save,
)
@@ -147,8 +146,6 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
language = None
text = (a.get("sample_script") or "").strip() or _FALLBACK_SCRIPT
loop = asyncio.get_running_loop()
def _infer(seed: int):
return _run_inference(
model, # _model
@@ -171,14 +168,18 @@ async def _render_archetype_wav(a: dict, out_path: Path) -> None:
"broadcast", # effect_preset
)
audio_tensor = await loop.run_in_executor(_gpu_pool, _infer, _PREVIEW_SEED)
# Bounded + pool-reset on hang so a wedged preview render can't starve the
# GPU pool and brick the backend (#730 class).
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _infer(_PREVIEW_SEED), what="Archetype preview generate")
if _is_unusable_audio(audio_tensor):
# Blank OR a degenerate tonal buzz — retry once on a different seed to
# step off the bad diffusion trajectory. Static message only: the
# archetype id is request-derived (CodeQL log-injection); the seed is a
# module constant, safe to log.
logger.warning("Archetype rendered unusable at seed %d — retrying once", _PREVIEW_SEED)
audio_tensor = await loop.run_in_executor(_gpu_pool, _infer, _PREVIEW_SEED + 1)
audio_tensor = await run_on_gpu_pool_guarded(
lambda: _infer(_PREVIEW_SEED + 1), what="Archetype preview generate")
if _is_unusable_audio(audio_tensor):
raise RuntimeError("the voice engine returned no audible audio for this archetype")
+60 -13
View File
@@ -164,6 +164,7 @@ async def audiobook_cover(cover: UploadFile = File(...)) -> dict:
class AudiobookRequest(BaseModel):
text: str
default_voice: str | None = None # voice profile id; None = engine default
language: str | None = None # None/"Auto" → profile language, else autodetect (#505)
bitrate: str = "128k"
format: str = "m4b" # "m4b" | "mp3"
loudness: str | None = None # None/"off" | "acx" | "podcast" (opt-in)
@@ -215,7 +216,35 @@ def _resolve_voice(profile_id: str | None) -> dict:
return out
def _build_synth(default_voice: str | None) -> dict:
def _resolve_default_language(language: str | None, default_voice: str | None) -> str | None:
"""Pick the language to thread into the longform synth callable.
Priority (mirrors the single-shot /generate path, #533): an explicit
non-Auto request ``language`` wins; otherwise the selected profile's stored
language drives it; otherwise ``None`` (genuine Auto — the engine
autodetects, exactly as before). Hardcoding ``None`` here (#505 B2) let the
engine re-autodetect per chunk, so a non-English clone flipped to the wrong
language on short/ambiguous chapters.
"""
if language and language != "Auto":
return language
if default_voice:
from core.db import db_conn
with db_conn() as conn:
row = conn.execute(
"SELECT language FROM voice_profiles WHERE id=?", (default_voice,)
).fetchone()
if row:
try:
prof_lang = row["language"]
except (KeyError, IndexError):
prof_lang = None
if prof_lang and prof_lang != "Auto":
return prof_lang
return None
def _build_synth(default_voice: str | None, language: str | None = None) -> dict:
"""Describe how to synthesize for the active TTS engine.
Returns a dict with ``mode``, ``resolve`` (voice-id → resolved refs, cached
@@ -223,6 +252,11 @@ def _build_synth(default_voice: str | None) -> dict:
``get_model``; other engines carry a ready ``synth`` + ``sample_rate``.
:func:`_prepare_synth` turns this into a uniform ``(synth, sr, resolve,
engine_id)`` once the (async) model is in hand.
``language`` (already resolved by :func:`_resolve_default_language`) is
threaded into every chunk's ``generate`` so a non-English clone stays in its
language instead of re-autodetecting per chunk (#505 B2). ``None`` keeps the
engine's autodetect behavior unchanged.
"""
from services.tts_backend import OmniVoiceBackend, active_backend_id, get_backend_class
@@ -239,14 +273,14 @@ def _build_synth(default_voice: str | None) -> dict:
if cls is OmniVoiceBackend:
from services.model_manager import get_model
return {"mode": "omnivoice", "resolve": resolve,
"engine_id": engine_id, "get_model": get_model}
"engine_id": engine_id, "get_model": get_model, "language": language}
backend = cls()
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
return backend.generate(
text, language=None, ref_audio=v["ref_audio"],
text, language=language, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0,
)
@@ -254,20 +288,22 @@ def _build_synth(default_voice: str | None) -> dict:
"synth": synth, "sample_rate": backend.sample_rate}
async def _prepare_synth(default_voice: str | None):
async def _prepare_synth(default_voice: str | None, language: str | None = None):
"""Resolve :func:`_build_synth` into ``(synth, sample_rate, resolve,
engine_id)`` — awaiting the OmniVoice model load when needed. Shared by the
full job and the per-chapter preview."""
info = _build_synth(default_voice)
full job and the per-chapter preview. ``language`` is threaded into every
chunk so a non-English clone holds its language (#505 B2)."""
info = _build_synth(default_voice, language=language)
resolve, engine_id = info["resolve"], info["engine_id"]
if info["mode"] == "omnivoice":
lang = info["language"]
model = await info["get_model"]()
sr = getattr(model, "sampling_rate", 24000)
def synth(text, voice_id, speed=None):
v = resolve(voice_id)
return model.generate(
text=text, language=None, ref_audio=v["ref_audio"],
text=text, language=lang, ref_audio=v["ref_audio"],
ref_text=v["ref_text"], instruct=v["instruct"], duration=None,
speed=float(speed) if speed else 1.0,
)[0]
@@ -323,6 +359,7 @@ class AudiobookPreviewRequest(BaseModel):
text: str
chapter_index: int = 0
default_voice: str | None = None
language: str | None = None # None/"Auto" → profile language, else autodetect
lexicon: dict | None = None
@@ -346,7 +383,10 @@ async def audiobook_preview(req: AudiobookPreviewRequest) -> dict:
chapter = plan.chapters[req.chapter_index]
cache_dir = os.path.join(OUTPUTS_DIR, "longform_cache") # shared with _render_longform_sse
os.makedirs(cache_dir, exist_ok=True)
synth, sr, resolve, engine_id = await _prepare_synth(req.default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
req.default_voice,
language=_resolve_default_language(req.language, req.default_voice),
)
loop = asyncio.get_running_loop()
wav_path, dur, was_cached = await loop.run_in_executor(
_gpu_pool, _render_chapter_cached, chapter, synth, sr, engine_id, resolve, cache_dir,
@@ -364,6 +404,7 @@ async def _render_longform_sse(
plan,
*,
default_voice: str | None,
language: str | None = None,
fmt: str = "m4b",
bitrate: str = "128k",
loudness: str | None = None,
@@ -412,7 +453,8 @@ async def _render_longform_sse(
for c in plan.chapters
],
params={
"default_voice": default_voice, "fmt": fmt, "bitrate": bitrate,
"default_voice": default_voice, "language": language,
"fmt": fmt, "bitrate": bitrate,
"loudness": loudness, "cover_path": cover_path,
"metadata": metadata, "lexicon": lexicon,
},
@@ -453,7 +495,9 @@ async def _render_longform_sse(
loop = asyncio.get_running_loop()
try:
synth, sr, resolve, engine_id = await _prepare_synth(default_voice)
synth, sr, resolve, engine_id = await _prepare_synth(
default_voice, language=_resolve_default_language(language, default_voice)
)
total = len(plan.chapters)
chapter_files: list[str] = []
@@ -557,7 +601,8 @@ async def audiobook_synthesize(req: AudiobookRequest):
plan = parse_audiobook_script(req.text, default_voice=req.default_voice)
return StreamingResponse(
_render_longform_sse(
plan, default_voice=req.default_voice, fmt=req.format, bitrate=req.bitrate,
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
lexicon=req.lexicon, job_type="audiobook",
),
@@ -582,6 +627,7 @@ class LongformChapter(BaseModel):
class LongformRenderRequest(BaseModel):
chapters: list[LongformChapter] = []
default_voice: str | None = None
language: str | None = None # None/"Auto" → profile language, else autodetect (#505)
bitrate: str = "128k"
format: str = "m4b"
loudness: str | None = None
@@ -612,7 +658,8 @@ async def longform_render(req: LongformRenderRequest):
plan = AudiobookPlan(chapters=chapters)
return StreamingResponse(
_render_longform_sse(
plan, default_voice=req.default_voice, fmt=req.format, bitrate=req.bitrate,
plan, default_voice=req.default_voice, language=req.language,
fmt=req.format, bitrate=req.bitrate,
loudness=req.loudness, cover_path=req.cover_path, metadata=req.metadata,
lexicon=req.lexicon, job_type="story",
),
@@ -702,7 +749,7 @@ async def resume_longform(job_id: str):
# never names a work dir / output file (defence-in-depth path-injection).
return StreamingResponse(
_render_longform_sse(
plan, default_voice=p.get("default_voice"),
plan, default_voice=p.get("default_voice"), language=p.get("language"),
fmt=p.get("fmt", "m4b"), bitrate=p.get("bitrate", "128k"),
loudness=p.get("loudness"), cover_path=p.get("cover_path"),
metadata=p.get("metadata"), lexicon=p.get("lexicon"),
+29 -16
View File
@@ -142,7 +142,7 @@ async def _run_batch_pipeline(job_id: str, job: dict):
_set_progress(job, "transcribe", 0)
from services.asr_backend import get_active_asr_backend
from services.model_manager import _gpu_pool, _cpu_pool
from services.model_manager import _gpu_pool, _cpu_pool, run_on_gpu_pool_guarded
from services.segmentation import (
segment_transcript, assign_speakers_heuristic,
)
@@ -162,7 +162,12 @@ async def _run_batch_pipeline(job_id: str, job: dict):
pass
return segments, detected_lang
segments, source_lang = await loop.run_in_executor(_gpu_pool, _transcribe)
# Bound the batch transcribe (#730) so a wedged whisperx/CTranslate2 call
# can't hold its GPU-pool worker forever and starve the rest of the backend
# ("can't reach backend"); run_transcribe_guarded also resets the pool on
# timeout to restore capacity.
from services.asr_backend import run_transcribe_guarded
segments, source_lang = await run_transcribe_guarded(_gpu_pool, _transcribe, what="Batch")
source_lang = (source_lang or "en").split("_")[0][:2].lower()
job["segments"] = segments
job["source_lang"] = source_lang
@@ -174,6 +179,21 @@ async def _run_batch_pipeline(job_id: str, job: dict):
job["status"] = "failed"
return
# ── Engine resolution (issue #312 class) ────────────────────────────
# Batch used to hardcode OmniVoice via get_model() regardless of the
# engine selected in Settings → Engines. require_cloning only when a
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
# any active engine. Resolved ONCE for the whole job (every language
# below shares the same active engine); an uncaught ValueError here
# propagates to _worker()'s existing except-Exception handling, which
# already records a structured job failure via core.failure.build_failure.
from services.tts_backend import resolve_generation_backend
backend = await resolve_generation_backend(
require_cloning=bool(job.get("voice_id")),
cloning_purpose="this batch job's pinned voice",
)
sr = backend.sample_rate
# ── 3. Translate + Generate per language ───────────────────────────
total_langs = len(langs)
outputs = {}
@@ -238,13 +258,10 @@ async def _run_batch_pipeline(job_id: str, job: dict):
total_segments=len(translated_segments),
)
from services.model_manager import get_model
from services.audio_dsp import apply_mastering, normalize_audio
from services.audio_io import atomic_save_wav
import torch
_model = await get_model()
sr = _model.sampling_rate
total_samples = int(duration * sr)
full_audio = torch.zeros(1, total_samples)
total_segs = len(translated_segments)
@@ -290,28 +307,24 @@ async def _run_batch_pipeline(job_id: str, job: dict):
ref_text = row.get("ref_text")
try:
audios = _model.generate(
audio_out = backend.generate(
text=text, language=lang,
ref_audio=ref_audio, ref_text=ref_text,
duration=dur, num_step=16,
guidance_scale=2.0, speed=1.0,
denoise=True, postprocess_output=True,
)
audio_out = audios[0]
# TODO(#312): this route runs the OmniVoice model directly (not the active
# backend), so VoxCPM2 never reaches it. When these routes become
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
mastered = apply_mastering(
audio_out,
sample_rate=sr,
)
return normalize_audio(mastered, target_dBFS=-2.0)
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=sr)
return normalize_audio(audio_out, target_dBFS=-2.0)
except Exception as e:
logger.warning("TTS failed for seg %d (lang=%s): %s", i, lang, e)
return torch.zeros(1, int(dur * sr))
try:
audio_tensor = await loop.run_in_executor(_gpu_pool, _gen)
# Bounded + pool-reset on hang so a wedged batch segment can't
# starve the GPU pool and brick the backend (#730 class).
audio_tensor = await run_on_gpu_pool_guarded(_gen, what="Batch generate")
# Fit to slot
target_samples_seg = int(seg_duration * sr)
+26 -5
View File
@@ -18,7 +18,7 @@ import os
import tempfile
import time
from fastapi import APIRouter, File, Form, UploadFile
from fastapi import APIRouter, File, Form, HTTPException, UploadFile
from typing import Optional
router = APIRouter()
@@ -96,9 +96,17 @@ async def transcribe_audio(
return result, backend.id
from services.model_manager import _gpu_pool
loop = asyncio.get_running_loop()
from services.asr_backend import ASRTimeoutError, run_transcribe_guarded
t0 = time.perf_counter()
result, engine_id = await loop.run_in_executor(_gpu_pool, _run)
try:
result, engine_id = await run_transcribe_guarded(
_gpu_pool, _run, what="Dictation",
)
except ASRTimeoutError as e:
# Backend is alive — ASR couldn't finish. 504 with guidance, not a
# silent hang the UI reads as "can't reach the local backend".
logger.warning("Capture transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
elapsed = round(time.perf_counter() - t0, 2)
# Normalize result shape
@@ -112,6 +120,15 @@ async def transcribe_audio(
from services.refinement import collapse_repetitive_artifacts
full_text = collapse_repetitive_artifacts(full_text)
# Cross-transport parity: deterministically polish the final text
# (leading capital + terminal punctuation) exactly like the live
# dictation socket (capture_ws) does, so the widget's POST fallback and
# MCP/CLI callers get the same typed-looking result the WS returns —
# not the raw "...test" the REST path used to leak. Segments stay raw
# (their timings/verbatim recognition are the contract).
from services.text_polish import polish_text
full_text = polish_text(full_text)
# Calculate audio duration from segments if available
duration = 0.0
if segments:
@@ -127,8 +144,12 @@ async def transcribe_audio(
if _truthy(refine) and full_text:
from services.refinement import maybe_refine
refined = await asyncio.to_thread(maybe_refine, full_text)
if refined and refined != full_text:
refined_text = refined
if refined:
# Polish the refined text too, so both surfaced strings read as
# typed text (mirrors the raw-vs-refined contract of the WS).
refined = polish_text(refined)
if refined != full_text:
refined_text = refined
logger.info(
"Capture transcription done: engine=%s, elapsed=%.2fs, duration=%.1fs, mode=%s, refined=%s",
+485 -15
View File
@@ -19,7 +19,15 @@ Protocol:
"segments": [...], "language": "en",
"duration_s": 4.2, "transcription_time_s": 0.8,
"engine": "mlx-whisper"}
{"type": "error", "detail": "..."} error
{"type": "status", "stage": "downloading"|"loading"|"ready"}
model cold-start
{"type": "error", "message": "...", "kind": "...",
"detail": "..."} error ("detail"
kept for legacy)
Every ``final`` text is normalised by services.text_polish (leading
capital for Latin scripts, terminal punctuation, single-spaced) so the
pasted result reads like typed text. Partials are raw.
"""
from __future__ import annotations
@@ -32,6 +40,7 @@ import time
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
from api.dependencies import _LOOPBACK_HOSTS, ws_remote_authorized
from services.text_polish import polish_text
router = APIRouter()
logger = logging.getLogger("omnivoice.capture_ws")
@@ -101,6 +110,30 @@ def _pcm16_to_wav(pcm: bytes, sample_rate: int) -> str | None:
return None
def _select_sherpa_spec(websocket: WebSocket):
"""Resolve the sherpa dictation model for this WS session, or None.
A ``?model=<id>`` query param wins (the frontend can pin a model per
session); otherwise the persisted ``dictation.model_id`` pref is used (only
when dictation is enabled). Returns the :class:`SherpaModelSpec` or None
(None the legacy Whisper/WebM path runs unchanged).
"""
try:
from services import sherpa_dictation as sd
except Exception:
return None
requested = websocket.query_params.get("model")
if requested:
return sd.get_spec(requested) # explicit selection (may be None if bad)
# Fall back to the persisted dictation pref.
try:
from services.asr_backend import dictation_model_id
mid = dictation_model_id()
except Exception:
mid = None
return sd.get_spec(mid) if mid else None
@router.websocket("/ws/transcribe")
async def ws_transcribe(websocket: WebSocket):
"""Stream audio in, get partial + final transcription out."""
@@ -119,6 +152,24 @@ async def ws_transcribe(websocket: WebSocket):
await websocket.accept()
# Live-dictation engine selection. When a sherpa-onnx model is selected
# (via ?model= or the dictation.model_id pref) AND sherpa is installed,
# run the dedicated low-latency handler. Otherwise fall through to the
# legacy Whisper/WebM path, byte-for-byte unchanged.
spec = _select_sherpa_spec(websocket)
if spec is not None:
from services.asr_backend import SherpaDictationBackend
ok, _reason = SherpaDictationBackend.is_available()
if ok:
if spec.streaming:
await _run_sherpa_streaming(websocket, spec)
else:
await _run_sherpa_offline(websocket, spec)
return
# sherpa not installed → fall through to the legacy path so the user
# still gets dictation (just not live partials).
logger.info("sherpa dictation selected but unavailable — legacy path")
# Opt-in dictate-over-playback AEC (parity Action 8b). Default OFF →
# identical legacy behaviour. When on, frames are 1-byte-tagged raw PCM
# and the cleaned mic stream is muxed via stdlib wave (not ffmpeg).
@@ -260,20 +311,29 @@ async def ws_transcribe(websocket: WebSocket):
if total_bytes > MIN_FINAL_BUFFER_BYTES:
try:
result = await _transcribe_buffer_full(audio_chunks, pcm_sr=pcm_sr)
# Dictation v2: deterministic polish so the pasted final reads
# like typed text (leading capital, terminal punctuation).
result["text"] = polish_text(result.get("text", ""))
# Wave 2.1: optional local-LLM refinement of the final text.
# Off-thread (network call, not GPU); pass-through on any
# failure or when no LLM backend is configured. The raw text
# always ships too — clients paste refined_text ?? text.
# HARD-BOUNDED (maybe_refine_async, ~4s OMNIVOICE_REFINE_TIMEOUT_S):
# a slow/dead LLM can never delay this `final` beyond the budget —
# it falls back to the unrefined (but polished) text. Best-effort:
# never let refinement turn a good final into an error. The raw
# text always ships too — clients paste refined_text ?? text.
if result.get("text"):
from services.refinement import maybe_refine
refined = await asyncio.to_thread(maybe_refine, result["text"])
if refined and refined != result["text"]:
result["refined_text"] = refined
try:
from services.refinement import maybe_refine_async
refined = await maybe_refine_async(result["text"])
if refined and refined != result["text"]:
result["refined_text"] = refined
except Exception as e: # noqa: BLE001
logger.debug("Dictation refinement skipped: %s", e)
if not await _safe_send({"type": "final", **result}):
logger.debug("Skipped final send — client already disconnected")
except Exception as e:
logger.error("Final transcription failed: %s", e)
await _safe_send({"type": "error", "detail": str(e)})
await _safe_send({"type": "error", "message": str(e),
"kind": "transcribe", "detail": str(e)})
else:
await _safe_send({
"type": "final",
@@ -292,6 +352,413 @@ async def ws_transcribe(websocket: WebSocket):
pass
# ── sherpa-onnx live dictation handlers ─────────────────────────────────────
#
# Both handlers read raw int16 mono PCM frames (reusing the AEC framing: an
# opt-in 1-byte type prefix when ?aec=1, else bare PCM) at ?sr= (default 16000).
# This is the low-latency transport — no WebM/ffmpeg in the hot path.
# How often the offline-kind handler re-decodes the live window for a partial
# (streaming-kind decodes every frame, no cadence needed).
SHERPA_OFFLINE_PARTIAL_S = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_PARTIAL", "0.8"))
# Utterance gate for the offline-kind handler: once the trailing this-many
# seconds of the live buffer fall below the RMS floor, the utterance is
# COMMITTED — decoded, flushed as a `final`, and dropped from the buffer. Each
# decode is thereby bounded by one utterance instead of the whole session
# (the old full-buffer re-decode was O(n²)), and a sentence commits ~0.6s
# after the user stops speaking instead of only at EOF.
SHERPA_OFFLINE_SILENCE_S = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_SILENCE", "0.6"))
SHERPA_OFFLINE_RMS_FLOOR = float(os.environ.get("OMNIVOICE_SHERPA_OFFLINE_RMS", "0.01"))
def _pcm16_to_f32(pcm: bytes):
"""int16 little-endian mono PCM bytes → float32 numpy in [-1, 1]."""
import numpy as np
if not pcm:
return np.zeros(0, dtype=np.float32)
# Guard against an odd trailing byte from a split frame.
if len(pcm) % 2:
pcm = pcm[:-1]
return np.frombuffer(pcm, dtype=np.int16).astype(np.float32) / 32768.0
async def _sherpa_session(websocket: WebSocket):
"""Shared WS receive setup for the sherpa handlers.
Returns ``(get_frame, state)`` where ``get_frame`` is an async callable
that yields the next near-end (mic) PCM bytes, ``b""`` for a keepalive/ref
frame, or ``None`` on EOF/disconnect. ``state`` carries sample rate, AEC,
and the disconnect flag for the caller's finaliser.
"""
pcm_sr = 16000
try:
pcm_sr = int(websocket.query_params.get("sr", "16000"))
except (TypeError, ValueError):
pcm_sr = 16000
aec = None
if websocket.query_params.get("aec") in ("1", "true", "on"):
try:
from services.aec import NlmsEchoCanceller
aec = NlmsEchoCanceller(sample_rate=pcm_sr)
except Exception as e:
logger.warning("AEC requested but disabled (sherpa): %s", e)
aec = None
return pcm_sr, aec
async def _recv_pcm_frame(websocket: WebSocket, aec):
"""Receive one frame; return (kind, pcm_bytes).
kind {"near","eof","skip"}. Demuxes AEC-tagged frames when ``aec`` is on
and feeds the playback reference into the canceller. A text "EOF" or an
empty/closed socket yields kind "eof".
"""
msg = await websocket.receive()
mtype = msg.get("type")
if mtype == "websocket.disconnect":
return "eof", b""
if mtype != "websocket.receive":
return "skip", b""
data = msg.get("bytes")
if data is not None:
if len(data) == 0:
return "eof", b""
if aec is not None:
kind, payload = _demux_aec_frame(data)
if kind == "far":
aec.push_far_end(payload)
return "skip", b""
if not payload:
return "skip", b""
return "near", aec.process_near_end(payload)
return "near", data
if msg.get("text") == "EOF":
return "eof", b""
return "skip", b""
async def _sherpa_load_with_status(websocket: WebSocket, backend, spec) -> bool:
"""Build the recognizer off the event loop, narrating cold-start progress.
Sends ``{"type":"status","stage":"downloading"|"loading"}`` before the
load ("downloading" when the pinned assets aren't in the HF cache yet;
stage-only HF's per-file progress isn't worth a callback plumb-through)
and ``{"type":"status","stage":"ready"}`` after, so the widget can show
*why* the first dictation takes a moment. Returns False when the load
failed (the error frame is sent and the socket closed here).
"""
try:
from services import sherpa_dictation as _sd
stage = "loading" if _sd.is_installed(spec) else "downloading"
except Exception:
stage = "loading"
try:
await websocket.send_json({"type": "status", "stage": stage})
except Exception:
pass
try:
await asyncio.to_thread(backend.ensure_loaded)
except Exception as e:
logger.error("sherpa dictation load failed (%s): %s", spec.id, e)
try:
await websocket.send_json({"type": "error", "message": str(e),
"kind": "load", "detail": str(e)})
await websocket.close()
except Exception:
pass
return False
try:
await websocket.send_json({"type": "status", "stage": "ready"})
except Exception:
pass
return True
async def _run_sherpa_streaming(websocket: WebSocket, spec):
"""True streaming: feed the OnlineRecognizer frame-by-frame, emit `partial`
every time the decoded text grows, and `final` on sherpa's endpoint (silence)
detection and on EOF. <300ms perceived latency on CPU for the tiny models.
"""
import numpy as np
from services.asr_backend import get_sherpa_dictation_backend
pcm_sr, aec = await _sherpa_session(websocket)
logger.info("sherpa streaming dictation: model=%s sr=%d aec=%s",
spec.id, pcm_sr, bool(aec))
# Reuse the shared, per-model warm backend (#888): the recognizer is built
# once and shared across sessions instead of rebuilt (1.32.5s) per connect,
# so the first dictation is instant when the preload warmed it. Each session
# still gets its own decode stream below.
backend = get_sherpa_dictation_backend(spec.id)
# Build the recognizer off the event loop if it isn't warm yet
# (download-on-first-use + ONNX session init can take a moment); status
# frames keep the widget honest.
if not await _sherpa_load_with_status(websocket, backend, spec):
return
rec = backend._rec
stream = rec.create_stream()
last_partial = ""
committed: list[str] = [] # finalized utterances this session
client_disconnected = False
async def _send(payload) -> bool:
nonlocal client_disconnected
if client_disconnected:
return False
try:
await websocket.send_json(payload)
return True
except Exception:
client_disconnected = True
return False
def _decode_after_feed(pcm: bytes):
"""Blocking: feed one PCM frame, decode, return (text, is_endpoint).
Runs in a thread so the ONNX work never blocks the event loop."""
samples = _pcm16_to_f32(pcm)
if len(samples):
stream.accept_waveform(pcm_sr, samples)
while rec.is_ready(stream):
rec.decode_stream(stream)
endpoint = rec.is_endpoint(stream)
text = (rec.get_result(stream) or "").strip()
return text, endpoint
def _flush_final():
"""Blocking: pad + drain the stream for the trailing utterance."""
tail = np.zeros(int(0.5 * pcm_sr), dtype=np.float32)
stream.accept_waveform(pcm_sr, tail)
stream.input_finished()
while rec.is_ready(stream):
rec.decode_stream(stream)
return (rec.get_result(stream) or "").strip()
try:
while True:
kind, pcm = await _recv_pcm_frame(websocket, aec)
if kind == "eof":
break
if kind == "skip":
continue
text, endpoint = await asyncio.to_thread(_decode_after_feed, pcm)
if endpoint:
# Commit this utterance (polished — it gets pasted); reset
# for the next one.
text = polish_text(text)
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
rec.reset(stream)
last_partial = ""
elif text and text != last_partial:
last_partial = text
await _send({"type": "partial", "text": text})
except WebSocketDisconnect:
client_disconnected = True
except Exception as e:
logger.warning("sherpa streaming loop ended: %s", e)
client_disconnected = True
# Drain the trailing (un-endpointed) utterance on EOF.
try:
tail_text = await asyncio.to_thread(_flush_final)
except Exception as e:
logger.debug("sherpa streaming flush failed: %s", e)
tail_text = ""
tail_text = polish_text(tail_text)
if tail_text and tail_text != (committed[-1] if committed else None):
committed.append(tail_text)
# Pieces are already polished; the join is too (polish is idempotent).
full = " ".join(t for t in committed if t).strip()
segments = [{"start": 0.0, "end": None, "text": t} for t in committed if t]
if not client_disconnected:
if full:
# Hard-bounded refinement (~4s): never delays this summary `final`
# beyond OMNIVOICE_REFINE_TIMEOUT_S even with a dead LLM endpoint.
try:
from services.refinement import maybe_refine_async
refined = await maybe_refine_async(full)
except Exception:
refined = None
payload = {"type": "final", "text": full, "segments": segments,
"language": "auto", "engine": backend.id}
if refined and refined != full:
payload["refined_text"] = refined
await _send(payload)
else:
await _send({"type": "final", "text": "", "segments": [],
"language": "auto", "engine": backend.id})
try:
await websocket.close()
except Exception:
pass
async def _run_sherpa_offline(websocket: WebSocket, spec):
"""Offline-kind sherpa model with live partials, utterance-windowed.
Raw PCM accumulates in a *live* buffer holding only the current
(uncommitted) utterance. Every ~800ms the live window is re-decoded for a
``partial``; when the trailing ~0.6s of it fall below the RMS floor the
utterance is committed decoded once more, flushed as a ``final``, and
its samples dropped so per-partial cost is bounded by one utterance
(not the whole session) and sentences commit as the user pauses instead
of only at EOF."""
from services.asr_backend import get_sherpa_dictation_backend
pcm_sr, aec = await _sherpa_session(websocket)
logger.info("sherpa offline dictation: model=%s sr=%d aec=%s",
spec.id, pcm_sr, bool(aec))
# Shared, per-model warm backend (#888) — built once, reused per session.
backend = get_sherpa_dictation_backend(spec.id)
if not await _sherpa_load_with_status(websocket, backend, spec):
return
buf = bytearray() # live (uncommitted) PCM only
committed: list[str] = [] # polished utterances already flushed
last_partial = ""
running = True
client_disconnected = False
last_audio = time.monotonic()
# Trailing-silence gate window, in bytes of int16 mono PCM.
sil_bytes = max(2, int(SHERPA_OFFLINE_SILENCE_S * pcm_sr) * 2)
async def _send(payload) -> bool:
nonlocal client_disconnected
if client_disconnected:
return False
try:
await websocket.send_json(payload)
return True
except Exception:
client_disconnected = True
return False
def _rms(pcm: bytes) -> float:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return 0.0
return float((samples * samples).mean() ** 0.5)
def _decode_window(pcm: bytes) -> str:
samples = _pcm16_to_f32(pcm)
if not len(samples):
return ""
return backend._decode_offline(samples, pcm_sr)
async def receive():
nonlocal running, client_disconnected, last_audio
try:
while running:
kind, pcm = await _recv_pcm_frame(websocket, aec)
if kind == "eof":
running = False
break
if kind == "skip":
continue
buf.extend(pcm)
last_audio = time.monotonic()
except WebSocketDisconnect:
client_disconnected = True
running = False
except Exception as e:
logger.debug("sherpa offline receive ended: %s", e)
running = False
async def _commit(snapshot: bytes):
"""Finalize one utterance: decode it off-thread, flush a polished
`final`, drop its samples from the live buffer. `receive()` may
append while we decode only the snapshot's prefix is dropped."""
nonlocal last_partial
try:
text = await asyncio.to_thread(_decode_window, snapshot)
except Exception as e:
logger.debug("sherpa offline commit decode failed: %s", e)
return
del buf[:len(snapshot)]
last_partial = ""
text = polish_text(text)
if text:
committed.append(text)
await _send({"type": "final", "text": text,
"segments": [{"start": 0.0, "end": None, "text": text}],
"language": "auto", "engine": backend.id})
async def partials():
nonlocal last_partial, running
while running:
await asyncio.sleep(SHERPA_OFFLINE_PARTIAL_S)
if not running or len(buf) < 2000:
continue
snapshot = bytes(buf)
if len(snapshot) > sil_bytes and \
_rms(snapshot[-sil_bytes:]) < SHERPA_OFFLINE_RMS_FLOOR:
if _rms(snapshot[:-sil_bytes]) >= SHERPA_OFFLINE_RMS_FLOOR:
await _commit(snapshot)
else:
# Pure silence — drop it (keep the gate window for
# continuity) so a long pause can't grow the buffer.
del buf[:len(snapshot) - sil_bytes]
continue
try:
text = await asyncio.to_thread(_decode_window, snapshot)
except Exception as e:
logger.debug("sherpa offline partial failed: %s", e)
continue
if text and text != last_partial:
last_partial = text
await _send({"type": "partial", "text": text})
recv_task = asyncio.create_task(receive())
part_task = asyncio.create_task(partials())
await asyncio.wait([recv_task, part_task], return_when=asyncio.FIRST_COMPLETED)
running = False
for t in (recv_task, part_task):
if not t.done():
t.cancel()
try:
await t
except (asyncio.CancelledError, Exception):
pass
# Drain the trailing (un-committed) utterance on EOF.
try:
tail = await asyncio.to_thread(_decode_window, bytes(buf))
except Exception as e:
logger.error("sherpa offline final failed: %s", e)
tail = ""
tail = polish_text(tail)
if tail:
committed.append(tail)
# Pieces are already polished; the join is too (polish is idempotent).
full = " ".join(committed).strip()
segments = [{"start": 0.0, "end": None, "text": t} for t in committed]
if not client_disconnected:
payload = {"type": "final", "text": full, "segments": segments,
"language": "auto", "engine": backend.id}
if full:
# Hard-bounded refinement (~4s) — never delays the `final`.
try:
from services.refinement import maybe_refine_async
refined = await maybe_refine_async(full)
if refined and refined != full:
payload["refined_text"] = refined
except Exception:
pass
await _send(payload)
try:
await websocket.close()
except Exception:
pass
async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None) -> str:
"""Quick partial transcription of the current audio buffer."""
@@ -301,15 +768,17 @@ async def _transcribe_buffer(chunks: list[bytes], *, pcm_sr: int | None = None)
try:
from services.model_manager import _gpu_pool
from services.asr_backend import get_capture_asr_backend
from services.asr_backend import get_capture_asr_backend, run_transcribe_guarded
def _run():
backend = get_capture_asr_backend()
result = backend.transcribe(tmp, word_timestamps=False)
return result.get("text", "")
loop = asyncio.get_running_loop()
text = await loop.run_in_executor(_gpu_pool, _run)
# Bound dictation transcribes (#730): a wedged whisperx/CTranslate2 call
# must not hold its GPU-pool worker forever and starve TTS / other ASR
# into a "can't reach backend"; on timeout the pool is reset to recover.
text = await run_transcribe_guarded(_gpu_pool, _run, what="Dictation")
return text.strip()
finally:
try:
@@ -327,7 +796,7 @@ async def _transcribe_buffer_full(chunks: list[bytes], *, pcm_sr: int | None = N
try:
from services.model_manager import _gpu_pool
from services.asr_backend import get_capture_asr_backend
from services.asr_backend import get_capture_asr_backend, run_transcribe_guarded
def _run():
backend = get_capture_asr_backend()
@@ -362,8 +831,9 @@ async def _transcribe_buffer_full(chunks: list[bytes], *, pcm_sr: int | None = N
"engine": backend.id,
}
loop = asyncio.get_running_loop()
return await loop.run_in_executor(_gpu_pool, _run)
# Bounded + pool-resetting on timeout (#730), same rationale as the
# partial path above.
return await run_transcribe_guarded(_gpu_pool, _run, what="Dictation")
finally:
try:
os.unlink(tmp)
+127
View File
@@ -0,0 +1,127 @@
"""
Dictation router sherpa-onnx live-dictation engine.
Exposes the seven sherpa-onnx dictation models and the dictation prefs the
frontend dictation UI binds to.
GET /dictation/models the 7 models + install state (frontend model list)
GET /dictation/prefs { enabled, mode, model_id }
POST /dictation/prefs persist any subset of those prefs
Install state reuses the same HF-cache check the model store uses, so a model
shown "installed" here is the same snapshot the backend will load.
Prefs are stored in the shared ``prefs.json`` store under the ``dictation.*``
namespace (``dictation.enabled``, ``dictation.mode``, ``dictation.model_id``),
mirroring how the ASR/TTS engine picks persist.
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from typing import Optional
from api.dependencies import require_loopback
from core import prefs
from services import sherpa_dictation as sd
router = APIRouter()
logger = logging.getLogger("omnivoice.dictation")
# Pref keys (the binding contract — the frontend writes exactly these).
PREF_ENABLED = "dictation.enabled"
PREF_MODE = "dictation.mode"
PREF_MODEL_ID = "dictation.model_id"
_DEFAULT_ENABLED = True
_DEFAULT_MODE = "toggle"
_VALID_MODES = ("toggle", "hold")
def _read_prefs() -> dict:
mid = prefs.get(PREF_MODEL_ID, sd.DEFAULT_MODEL_ID)
if not sd.is_sherpa_model(mid):
mid = sd.DEFAULT_MODEL_ID
mode = prefs.get(PREF_MODE, _DEFAULT_MODE)
if mode not in _VALID_MODES:
mode = _DEFAULT_MODE
return {
"enabled": bool(prefs.get(PREF_ENABLED, _DEFAULT_ENABLED)),
"mode": mode,
"model_id": mid,
}
@router.get("/dictation/models", dependencies=[Depends(require_loopback)])
def list_dictation_models():
"""The seven sherpa-onnx dictation models + install state.
Each entry: id, repo_id, label, tag ("offline"|"streaming"), recommended,
size_gb, languages, kind, and install state (installed/installing). The
``installed`` flag is computed from the same HF cache the model store reads,
so it matches the model-store row state.
"""
available, reason = sd.sherpa_available()
out = []
for spec in sd.list_specs():
out.append({
"id": spec.id,
"repo_id": spec.repo_id,
"label": spec.label,
"tag": spec.tag,
"recommended": spec.recommended,
"size_gb": spec.size_gb,
"languages": spec.languages,
"kind": spec.kind,
"installed": sd.is_installed(spec),
})
return {
"models": out,
"engine_available": available,
"engine_reason": None if available else reason,
"default_model_id": sd.DEFAULT_MODEL_ID,
}
@router.get("/dictation/prefs", dependencies=[Depends(require_loopback)])
def get_dictation_prefs():
return _read_prefs()
class DictationPrefsUpdate(BaseModel):
enabled: Optional[bool] = None
mode: Optional[str] = None
model_id: Optional[str] = None
@router.post("/dictation/prefs", dependencies=[Depends(require_loopback)])
def set_dictation_prefs(req: DictationPrefsUpdate):
"""Persist any subset of the dictation prefs. Validates ``mode`` and
``model_id`` so a bad value can't wedge the capture engine."""
if req.mode is not None:
if req.mode not in _VALID_MODES:
raise HTTPException(
status_code=400,
detail=f"mode must be one of {_VALID_MODES}",
)
prefs.set_(PREF_MODE, req.mode)
if req.model_id is not None:
if not sd.is_sherpa_model(req.model_id):
raise HTTPException(
status_code=400,
detail=f"unknown dictation model_id {req.model_id!r}",
)
# Normalise to the canonical dictation id (accept repo_id too).
prefs.set_(PREF_MODEL_ID, sd.get_spec(req.model_id).id)
if req.enabled is not None:
prefs.set_(PREF_ENABLED, bool(req.enabled))
# Rebuild the cached capture singleton so the change takes effect at once.
try:
from services import asr_backend
asr_backend._capture_backend = None
asr_backend._capture_backend_key = None
except Exception:
pass
return _read_prefs()
+395 -88
View File
@@ -16,6 +16,7 @@ from core.tasks import task_manager
from core import event_bus
from schemas.requests import DubIngestUrlRequest
from services.model_manager import get_model, _gpu_pool, _cpu_pool, get_diarization_pipeline, offload_tts_for_asr, restore_tts_after_asr
from services.asr_backend import ASRTimeoutError, reset_pool_after_wedge, run_transcribe_guarded
from services.audio_io import _safe_soundfile_write
from services.ffmpeg_utils import find_ffmpeg
from services.segmentation import (
@@ -23,6 +24,9 @@ from services.segmentation import (
assign_speakers_from_diarization,
assign_speakers_from_turns,
assign_speakers_heuristic,
resplit_segments_by_diarization,
resplit_segments_by_turns,
_words_from_whisper,
clean_up_segments,
)
from services.onset_align import snap_segment_starts
@@ -31,6 +35,7 @@ from services import dub_pipeline
router = APIRouter()
logger = logging.getLogger("omnivoice.api")
# ── Legacy-name aliases to services/dub_pipeline.py ────────────────────────
# Phase 2.4 moved the business logic into a service. Other routers
# (dub_generate, dub_translate, dub_export) + internal call sites below still
@@ -360,11 +365,41 @@ async def dub_ingest_url(req: DubIngestUrlRequest):
TRANSCRIBE_CHUNK_S = float(os.environ.get("OMNIVOICE_TRANSCRIBE_CHUNK_S", "30.0"))
TRANSCRIBE_CHUNK_TIMEOUT_S = float(os.environ.get("OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S", "120.0"))
#: How many times to attempt each transcribe chunk before giving up on it. A
#: transient wedge (esp. the first chunk, where whisperx cold-loads its model)
#: shouldn't silently drop that whole window — retry once on a fresh pool so the
#: transcript doesn't come back "missing the beginning".
_CHUNK_TRANSCRIBE_ATTEMPTS = max(1, int(os.environ.get("OMNIVOICE_TRANSCRIBE_CHUNK_ATTEMPTS", "2")))
_sse_event = dub_pipeline.sse_event
_prep_event_helper = dub_pipeline.prep_event # alias; we keep the module-local _prep_event below for the inline one-liner shape
#: User-facing warning emitted when auto voice cloning is skipped because the
#: speaker labels came from the silence-gap heuristic (see _diarize /
#: extract_speaker_clones — gap-based labels routinely mix two people's audio
#: into one reference, which is how "made up" clone voices happen).
CLONE_SKIP_HEURISTIC_MSG = (
"auto voice cloning skipped: speaker labels are gap-based estimates — "
"set up diarization (Settings → Models → pyannote) for per-speaker clones"
)
def _clamp_num_speakers(value) -> Optional[int]:
"""Clamp the user's speaker-count hint to a sane 120 range.
Shared by the SSE and legacy transcribe endpoints so the two can't drift.
None / non-int / out-of-range None (auto-detect), so a bad query string
can never break a diarization call.
"""
if value is None:
return None
try:
value = int(value)
except (TypeError, ValueError):
return None
return value if 1 <= value <= 20 else None
@router.get("/dub/transcribe-stream/{job_id}")
async def dub_transcribe_stream(
@@ -383,15 +418,14 @@ async def dub_transcribe_stream(
pyannote auto-detects the count but its auto-detect can collapse a
multi-speaker clip to a single speaker (issue #274). When the user knows
the exact count, supplying it forces pyannote to return that many speakers.
On paths that can't honor the hint exactly (inline ASR turns, the
silence-gap heuristic) it is never silently dropped: the heuristic cycles
the requested count and a `warning` SSE event tells the user how far the
labels can be trusted.
"""
# Clamp to a sane range; ignore anything non-positive / absurd so a bad
# query string can never break the diarization call. None → auto-detect.
if num_speakers is not None:
try:
num_speakers = int(num_speakers)
num_speakers = num_speakers if 1 <= num_speakers <= 20 else None
except (TypeError, ValueError):
num_speakers = None
num_speakers = _clamp_num_speakers(num_speakers)
job = _get_job(job_id)
@@ -418,6 +452,11 @@ async def dub_transcribe_stream(
asr_audio_target = job.get("vocals_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
asr_audio_target = job.get("audio_path")
# #963: onset snapping is only trustworthy on the Demucs vocals
# track. When separation failed/was skipped, dub_pipeline sets
# vocals_path to the mixed audio_path — so compare paths instead
# of trusting the key's presence.
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
preflight_error = "No audio available for transcription."
else:
@@ -425,10 +464,22 @@ async def dub_transcribe_stream(
try:
# The PyTorch-Whisper backend lazily builds its own pipeline
# when no preloaded `_asr_pipe` is present (issue #255), so it
# no longer needs OMNIVOICE_PRELOAD_TTS_ASR=1 — don't reject it
# here; any load failure surfaces per-chunk with a real cause.
# no longer needs OMNIVOICE_PRELOAD_TTS_ASR=1.
_asr_backend = get_active_asr_backend(asr_pipe=getattr(_model, "_asr_pipe", None))
# Eagerly load the model HERE so a real load failure (e.g.
# WhisperX: missing weights, CTranslate2/cuDNN mismatch, the
# torch-2.6 weights-only VAD regression) surfaces once, with
# its actual cause, as a clean preflight `error` event —
# instead of being buried in N cryptic per-chunk failures
# and retried on every chunk (#578). Run in a thread so the
# (blocking) load doesn't stall the event loop.
_ensure_loaded = getattr(_asr_backend, "ensure_loaded", None)
if callable(_ensure_loaded):
await asyncio.get_running_loop().run_in_executor(
_gpu_pool, _ensure_loaded
)
except Exception as e:
logger.exception("transcribe preflight: ASR load failed (job=%s)", job_id)
from core.failure import build_failure
f = build_failure(e, stage="transcribe-preflight", include_diagnostic=False)
preflight_error = "ASR backend initialization failed: " + f["reason"] + (
@@ -436,9 +487,16 @@ async def dub_transcribe_stream(
)
scene_cuts = job.get("scene_cuts") or []
async def gen():
async def _gen_body():
if preflight_error:
yield _sse_event("error", {"detail": preflight_error})
# Always follow a terminal `error` with `done` so the stream closes
# via a named event, not a raw connection drop. A bare error+close
# races the browser's native EventSource error (which carries no
# `data`); if that native error wins, the client falls back to the
# misleading generic "stream dropped … ASR backend failed" message
# and the real cause (in `detail`) is lost (#578).
yield _sse_event("error", {"detail": preflight_error, "retryable": True})
yield _sse_event("done", {})
return
import math
import tempfile
@@ -453,7 +511,9 @@ async def dub_transcribe_stream(
try:
audio_np, sr = await loop.run_in_executor(_cpu_pool, _load)
except Exception as e:
yield _sse_event("error", {"detail": f"audio load failed: {e}"})
# Terminal error → always emit `done` (see preflight note, #578).
yield _sse_event("error", {"detail": f"audio load failed: {e}", "retryable": True})
yield _sse_event("done", {})
return
total = float(len(audio_np)) / float(sr) if sr else 0.0
@@ -470,6 +530,9 @@ async def dub_transcribe_stream(
logger.warning("offload_tts_for_asr failed (continuing): %s", e)
all_segments: list[dict] = []
# Words (global-timeline) retained so diarization can re-split a segment
# that spans two speakers' turns at the word boundary (#486).
all_words: list = []
detected_lang = None
next_seg_id = 0
chunk_errors: list[str] = []
@@ -521,31 +584,59 @@ async def dub_transcribe_stream(
logger.exception("chunk transcribe failed (backend=%s)", _asr_backend.id)
return {"chunks": [], "language": None, "error": str(e)}
try:
# wait_for in a loop to yield pings so the EventSource connection doesn't drop
fut = loop.run_in_executor(_gpu_pool, _transcribe_chunk)
waited = 0.0
part = None
# Retry a failed/timed-out chunk once on a fresh pool before giving
# up. Otherwise a transient wedge on the FIRST chunk (whisperx often
# cold-loads its model there, the #730 hang) drops that whole window
# and the transcript is "missing the beginning, only middle+end".
# The retry reuses the same audio window, so a recovered chunk fills
# the hole instead of leaving silent gaps.
part = None
for _attempt in range(1, _CHUNK_TRANSCRIBE_ATTEMPTS + 1):
# A wedged chunk gets the SAME guarded-timeout + pool-reset
# semantics as the whole-file paths (#730/#851):
# run_transcribe_guarded bounds the call, abandons the poisoned
# pool so the retry (and any concurrent TTS work) gets a fresh
# worker, and raises the actionable ASRTimeoutError. Run it as
# a task and poll so we can keep yielding pings — the
# EventSource connection drops without them.
pool_reset_by_guard = False
task = asyncio.ensure_future(run_transcribe_guarded(
_gpu_pool, _transcribe_chunk,
what=f"Dub chunk {i + 1}/{chunks_n}",
timeout=TRANSCRIBE_CHUNK_TIMEOUT_S,
timeout_env="OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S",
))
while True:
done, pending = await asyncio.wait([fut], timeout=5.0)
done, _pending = await asyncio.wait({task}, timeout=5.0)
if done:
part = done.pop().result()
break
yield _sse_event("ping", {})
waited += 5.0
if waited >= TRANSCRIBE_CHUNK_TIMEOUT_S:
# Re-raise TimeoutError if we exceed the overall limit
raise asyncio.TimeoutError()
except asyncio.TimeoutError:
logger.error(
"Transcribe chunk %d/%d timed out after %.0fs (job=%s)",
i + 1, chunks_n, TRANSCRIBE_CHUNK_TIMEOUT_S, job_id,
)
part = {
"chunks": [], "language": None,
"error": f"Chunk {i+1} timed out after {TRANSCRIBE_CHUNK_TIMEOUT_S:.0f}s — "
f"ASR backend may be stuck. Try restarting the server.",
}
try:
part = task.result()
except ASRTimeoutError as e:
# The guard already reset the pool; keep the actionable
# message (it names the durable fixes, and — after repeated
# timeouts — the crash-isolated engine escape hatch).
pool_reset_by_guard = True
logger.error(
"Transcribe chunk %d/%d timed out after %.0fs (attempt %d/%d, job=%s)",
i + 1, chunks_n, TRANSCRIBE_CHUNK_TIMEOUT_S, _attempt,
_CHUNK_TRANSCRIBE_ATTEMPTS, job_id,
)
part = {"chunks": [], "language": None, "error": str(e)}
# Success → keep it. Failure/timeout → retry once on a fresh
# worker (the internal _transcribe_chunk except returns an
# error-part; the timeout path already reset the pool).
if part is not None and not part.get("error"):
break
if _attempt < _CHUNK_TRANSCRIBE_ATTEMPTS:
logger.warning(
"Retrying transcribe chunk %d/%d after failure/timeout (next attempt %d/%d, job=%s)",
i + 1, chunks_n, _attempt + 1, _CHUNK_TRANSCRIBE_ATTEMPTS, job_id,
)
if not pool_reset_by_guard:
reset_pool_after_wedge(
_gpu_pool, what=f"Dub chunk {i + 1}/{chunks_n}")
if part.get("error"):
chunk_errors.append(part["error"])
logger.warning("Chunk %d/%d error: %s", i + 1, chunks_n, part["error"])
@@ -553,16 +644,29 @@ async def dub_transcribe_stream(
detected_lang = part["language"]
asr_speaker_turns.extend(part.get("speaker_turns") or [])
chunk_segs = segment_transcript(part, duration=t1, scene_cuts=scene_cuts)
# Same word source segment_transcript used (already global-timeline),
# kept for the post-diarization speaker re-split (#486).
try:
all_words.extend(_words_from_whisper(part))
except Exception:
pass
# #280: Whisper often stretches a segment's start back over
# leading music/silence (classic case: speech begins at 0:03,
# transcript says 0.0 → the dub plays 3 s early). Snap starts
# forward to the actual speech onset. `audio_np` is the same
# track ASR ran on — vocals.wav when Demucs succeeded.
# track ASR ran on — vocals.wav when Demucs succeeded. #963:
# when it didn't (mixed audio), snapping is disabled — every
# footstep/sigh/score cue is a false onset candidate there.
try:
snap_segment_starts(chunk_segs, audio_np, sr)
snap_segment_starts(chunk_segs, audio_np, sr,
separated_vocals=asr_on_vocals)
except Exception as e:
logger.warning("onset alignment skipped for chunk %d: %s", i, e)
chunk_segs = assign_speakers_heuristic(chunk_segs)
# Provisional per-chunk labels for the streaming UI only — the
# final diarization pass below overwrites them. Honor the user's
# speaker-count hint here too so the interim view doesn't flip
# between 2 and N speakers.
chunk_segs = assign_speakers_heuristic(chunk_segs, num_speakers)
for s in chunk_segs:
s["id"] = f"s{next_seg_id:05x}"
s["text_original"] = s.get("text", "")
@@ -616,30 +720,110 @@ async def dub_transcribe_stream(
return
def _diarize():
"""Returns (segments, warning_payload_or_None).
"""Returns (segments, warning_payload_or_None, labels_source).
`labels_source` records where the speaker labels came from
`"pyannote"` | `"turns"` | `"heuristic"` so downstream
auto-clone extraction can refuse to cut reference audio from
gap-based estimates (a mixed-speaker reference is how "made up"
clone voices happen).
`warning_payload` is a structured dict
`{detail, error_class, docs_url}` whenever we silently fell back
to the silence-gap heuristic (no HF_TOKEN, model unavailable,
license not accepted, or pyannote raised). The heuristic only
detects speaker turns from >1.2s silences, so a rapid-fire
manwoman exchange will read as one speaker. Issue #78 — we
attach an `error_class` so the front-end's errorDocsMap can
render a "See docs" deeplink instead of a dead-end toast.
license not accepted, or pyannote raised) or whenever the
user's `num_speakers` hint could not be honored exactly. The
heuristic only detects speaker turns from >1.2s silences, so a
rapid-fire manwoman exchange will read as one speaker. Issue
#78 — we attach an `error_class` so the front-end's errorDocsMap
can render a "See docs" deeplink instead of a dead-end toast.
"""
# The active ASR backend already diarized inline (FunASR cam++):
# use its speaker turns directly and skip pyannote entirely (#182).
if asr_speaker_turns:
logger.info("Using inline ASR diarization (%d turns); skipping pyannote.", len(asr_speaker_turns))
return assign_speakers_from_turns(all_segments, asr_speaker_turns), None
from services.model_manager import (
DIARIZATION_ERR_LICENSE,
DIARIZATION_ERR_NO_TOKEN,
)
from core import error_docs_map
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
def _hint_suffix() -> str:
"""Honest caveat appended to heuristic-fallback warnings when a
multi-speaker hint is set: the count is now honored, but the
heuristic can't attribute voices. (A hint of 1 IS fully
honored one label so it needs no caveat.)"""
if not num_speakers or num_speakers < 2:
return ""
return (
f" Your speaker-count setting ({num_speakers}) is only "
f"approximately honored: the heuristic cycles "
f"{num_speakers} speaker labels on silence gaps instead "
f"of recognizing voices, so lines may be attributed to "
f"the wrong speaker."
)
def _use_turns(crash: Exception | None = None, err_sentinel=None):
"""Label from the ASR backend's inline speaker turns; warn when
that means the user's explicit count can't be enforced."""
logger.info(
"Using inline ASR diarization (%d turns)%s.",
len(asr_speaker_turns),
"" if crash else "; skipping pyannote",
)
assigned = assign_speakers_from_turns(all_segments, asr_speaker_turns)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
resplit = resplit_segments_by_turns(assigned, all_words, asr_speaker_turns)
if not num_speakers:
return resplit, None, "turns"
error_class = (
"HF_AUTH_FAILED"
if err_sentinel == DIARIZATION_ERR_NO_TOKEN
else "PYANNOTE_LICENSE_REQUIRED"
)
if crash:
detail = (
f"Speaker diarization crashed mid-run "
f"({type(crash).__name__}); falling back to the ASR "
f"engine's built-in speaker turns. Speaker-count hint "
f"ignored: the detected count may differ from the "
f"{num_speakers} you set."
)
else:
detail = (
f"Speaker-count hint ignored: pyannote diarization is "
f"unavailable, so the ASR engine's built-in speaker "
f"turns were used and the detected count may differ "
f"from the {num_speakers} you set. Set up diarization "
f"(Settings → Models → pyannote) to enforce an exact "
f"speaker count."
)
return resplit, {
"detail": detail,
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
"speaker_hint": {"requested": num_speakers, "status": "ignored"},
}, "turns"
# The active ASR backend already diarized inline (FunASR cam++):
# its turns are the fast path and skip pyannote entirely (#182) —
# but ONLY when the user didn't set an explicit speaker count.
# Inline turns are labeled per-30s-chunk and can't be forced to N
# speakers, so a set num_speakers prefers pyannote — the one
# engine that honors an exact count. When pyannote can't load,
# the turns are still the best labels available; use them and say
# so instead of silently eating the hint.
diar_pipe = None
err_sentinel = None
if asr_speaker_turns:
if num_speakers:
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
if not diar_pipe:
return _use_turns(err_sentinel=err_sentinel)
logger.info(
"num_speakers=%d set: preferring pyannote over %d inline "
"ASR turns (only pyannote honors an exact count).",
num_speakers, len(asr_speaker_turns),
)
else:
diar_pipe, err_sentinel = get_diarization_pipeline(return_error=True)
if not diar_pipe:
# Phase 1 AUTH-01: ask the resolver (App → Env → HF-CLI),
# not just the env var. This is the #35 fix — users who
@@ -690,13 +874,20 @@ async def dub_transcribe_stream(
f"heuristic; rapid speaker turns may be merged."
)
error_class = "PYANNOTE_LICENSE_REQUIRED"
warning = {
"detail": detail + _hint_suffix(),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
}
if num_speakers:
warning["speaker_hint"] = {
"requested": num_speakers,
"status": "approximate" if num_speakers > 1 else "honored",
}
return (
assign_speakers_heuristic(all_segments),
{
"detail": detail,
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
},
assign_speakers_heuristic(all_segments, num_speakers),
warning,
"heuristic",
)
try:
# Pass the user's speaker-count hint through to pyannote when
@@ -708,9 +899,17 @@ async def dub_transcribe_stream(
diar = diar_pipe(asr_audio_target, num_speakers=num_speakers)
else:
diar = diar_pipe(asr_audio_target)
return assign_speakers_from_diarization(all_segments, diar), None
assigned = assign_speakers_from_diarization(all_segments, diar)
# #486: split any segment that spans two speakers' turns at the
# word boundary (single-speaker segments pass through unchanged).
return resplit_segments_by_diarization(assigned, all_words, diar), None, "pyannote"
except Exception as e:
logger.error(f"Diarization failed: {e}")
# Inline ASR turns beat the silence-gap heuristic as a crash
# fallback (this path is reachable with turns present since a
# set num_speakers routes turns-jobs through pyannote).
if asr_speaker_turns:
return _use_turns(crash=e)
# Mid-run failure — classify against the same sentinels so a
# post-load 401 (rare but possible after a token rotation)
# still gets the right docs deeplink.
@@ -721,36 +920,50 @@ async def dub_transcribe_stream(
if err_class_post == DIARIZATION_ERR_LICENSE
else "PYANNOTE_LICENSE_REQUIRED" # LOAD failures land here too
)
warning = {
"detail": (
f"Speaker diarization crashed mid-run "
f"({type(e).__name__}); falling back to a silence-gap "
f"heuristic. Rapid speaker turns may be merged."
+ _hint_suffix()
),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
}
if num_speakers:
warning["speaker_hint"] = {
"requested": num_speakers,
"status": "approximate" if num_speakers > 1 else "honored",
}
return (
assign_speakers_heuristic(all_segments),
{
"detail": (
f"Speaker diarization crashed mid-run "
f"({type(e).__name__}); falling back to a silence-gap "
f"heuristic. Rapid speaker turns may be merged."
),
"error_class": error_class,
"docs_url": error_docs_map.lookup(error_class),
},
assign_speakers_heuristic(all_segments, num_speakers),
warning,
"heuristic",
)
fut_diar = loop.run_in_executor(_gpu_pool, _diarize)
final_segs = None
diar_warning = None
labels_source = "heuristic"
while True:
done, pending = await asyncio.wait([fut_diar], timeout=5.0)
if done:
final_segs, diar_warning = done.pop().result()
final_segs, diar_warning, labels_source = done.pop().result()
break
yield _sse_event("ping", {})
if diar_warning:
logger.warning("diarization fallback: %s", diar_warning.get("detail"))
yield _sse_event("warning", {
payload = {
"detail": diar_warning.get("detail"),
"source": "diarization",
"error_class": diar_warning.get("error_class"),
"docs_url": diar_warning.get("docs_url"),
})
}
# Machine-readable trail of what happened to the user's
# speaker-count hint (the `detail` text carries the human story).
if diar_warning.get("speaker_hint"):
payload["speaker_hint"] = diar_warning["speaker_hint"]
yield _sse_event("warning", payload)
job["segments"] = final_segs
@@ -762,17 +975,55 @@ async def dub_transcribe_stream(
try:
from services.speaker_clone import extract_speaker_clones, auto_profile_id
vocals_for_clone = job.get("vocals_path") or asr_audio_target
fut_clones = loop.run_in_executor(
_cpu_pool, extract_speaker_clones,
vocals_for_clone, final_segs, os.path.dirname(vocals_for_clone),
)
clones = None
while True:
done, pending = await asyncio.wait([fut_clones], timeout=5.0)
if done:
clones = done.pop().result()
break
yield _sse_event("ping", {})
clones = {}
if labels_source == "heuristic":
# Clone-purity guard: heuristic labels are silence-gap
# estimates, not voice identity — a per-speaker reference cut
# from them routinely concatenates two people's audio and the
# clone sounds "made up". Skip auto-clones and say so instead
# of shipping bad ones. (extract_speaker_clones enforces the
# same guard internally; this branch exists to surface the
# warning to the user.)
logger.info(
"auto speaker clones skipped (labels_source=heuristic, job=%s)",
job_id,
)
yield _sse_event("warning", {
"detail": CLONE_SKIP_HEURISTIC_MSG,
"source": "speaker_clone",
})
else:
fut_clones = loop.run_in_executor(
_cpu_pool, lambda: extract_speaker_clones(
vocals_for_clone, final_segs,
os.path.dirname(vocals_for_clone),
labels_source=labels_source,
),
)
while True:
done, pending = await asyncio.wait([fut_clones], timeout=5.0)
if done:
clones = done.pop().result()
break
yield _sse_event("ping", {})
if clones:
from services.speaker_clone import refine_ref_texts
# Bound the re-transcribe like every other ASR dispatch in
# this file (#730): a wedged transcribe would otherwise hold
# the GPU-pool worker forever and starve later work into a
# "can't reach backend". On timeout the guard resets the pool
# and raises — keep the original (unrefined) clones, matching
# refine_ref_text's own "failure is a strict no-op" fallback.
try:
clones = await run_transcribe_guarded(
_gpu_pool,
lambda: refine_ref_texts(clones, _asr_backend),
what="Dub clone ref-text refine",
)
except ASRTimeoutError as e:
logger.warning(
"clone ref-text refine timed out; keeping original ref_text: %s", e
)
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
# own reference from the vocals so the dub of each line matches the
# prosody of its source line. Short lines fall back to the
@@ -792,6 +1043,19 @@ async def dub_transcribe_stream(
),
)
if seg_clones:
from services.speaker_clone import refine_ref_texts
# Same guard as the per-speaker refine above (#730):
# keep the original seg_clones on a wedge/timeout.
try:
seg_clones = await run_transcribe_guarded(
_gpu_pool,
lambda: refine_ref_texts(seg_clones, _asr_backend),
what="Dub segment ref-text refine",
)
except ASRTimeoutError as e:
logger.warning(
"segment ref-text refine timed out; keeping original ref_text: %s", e
)
job["segment_clones"] = seg_clones
except Exception as e:
logger.warning("per-segment clone refs skipped: %s", e)
@@ -856,6 +1120,25 @@ async def dub_transcribe_stream(
})
yield _sse_event("done", {})
async def gen():
# Terminal-event guard (#516): the SSE stream must NEVER close without a
# terminal event. Any unanticipated exception in the body (e.g. an ASR
# load that escapes the per-chunk handler) previously dropped the
# connection, which the frontend can only report as "stream dropped,
# likely ASR failed" — hiding the real cause. Emit a structured `error`
# (with the actionable hint from build_failure) then `done`, so the user
# sees the real failure + a Retry instead of a silent disconnect.
try:
async for ev in _gen_body():
yield ev
except Exception as e: # noqa: BLE001 — last-resort stream finalizer
logger.exception("transcribe stream crashed (job=%s)", job_id)
from core.failure import build_failure
f = build_failure(e, stage="transcribe", include_diagnostic=False)
detail = f["reason"] + (f"{f['hint']}" if f.get("hint") else "")
yield _sse_event("error", {"detail": detail, "retryable": True})
yield _sse_event("done", {})
return StreamingResponse(
gen(),
media_type="text/event-stream",
@@ -867,18 +1150,28 @@ async def dub_transcribe_stream(
@router.post("/dub/transcribe/{job_id}")
async def dub_transcribe(job_id: str):
async def dub_transcribe(job_id: str, num_speakers: Optional[int] = None):
"""Legacy synchronous transcribe (kept for the headless CLI).
`num_speakers` mirrors the SSE endpoint's query param (same 120 clamp):
an exact speaker count forwarded to pyannote, or cycled by the silence-gap
heuristic when pyannote is unavailable. None auto-detect.
"""
num_speakers = _clamp_num_speakers(num_speakers)
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
_model = await get_model()
def _transcribe():
asr_audio_target = job.get("vocals_path")
if not asr_audio_target or not os.path.exists(asr_audio_target):
asr_audio_target = job.get("audio_path")
# #963: same source-awareness as the SSE endpoint — vocals_path
# falls back to the mixed audio_path when Demucs failed/skipped.
asr_on_vocals = bool(asr_audio_target) and asr_audio_target != job.get("audio_path")
import torch
detected_lang = None
@@ -922,10 +1215,13 @@ async def dub_transcribe(job_id: str):
segments = segment_transcript(result, duration=job.get("duration", 0.0), scene_cuts=scene_cuts)
# #280: snap segment starts forward to the actual speech onset so the
# dub doesn't begin seconds before the original speaker does.
# dub doesn't begin seconds before the original speaker does. #963:
# only on the separated vocals track — on mixed audio every ambient
# sound is a false onset candidate, so snapping is disabled.
try:
audio_for_onset, onset_sr = sf.read(asr_audio_target, dtype="float32")
snap_segment_starts(segments, audio_for_onset, onset_sr)
snap_segment_starts(segments, audio_for_onset, onset_sr,
separated_vocals=asr_on_vocals)
except Exception as e:
logger.warning("onset alignment skipped: %s", e)
@@ -933,13 +1229,20 @@ async def dub_transcribe(job_id: str):
if diar_pipe:
try:
diar_target = job.get("vocals_path") or job.get("audio_path")
diarization = diar_pipe(diar_target)
# Same hint pass-through as the SSE endpoint (#274): omit the
# kwarg entirely when unset so we don't depend on it existing
# in every pyannote build.
if num_speakers:
logger.info("Diarizing with num_speakers=%d (user hint)", num_speakers)
diarization = diar_pipe(diar_target, num_speakers=num_speakers)
else:
diarization = diar_pipe(diar_target)
segments = assign_speakers_from_diarization(segments, diarization)
except Exception as e:
logger.error(f"Pyannote diarization failed during inference: {e}. Falling back to heuristic.")
segments = assign_speakers_heuristic(segments)
segments = assign_speakers_heuristic(segments, num_speakers)
else:
segments = assign_speakers_heuristic(segments)
segments = assign_speakers_heuristic(segments, num_speakers)
# Previously ran `segment_for_subtitles(segments)` here. Removed 2026-04-21 —
# that splitter enforces Netflix's 17 CPS reading-speed ceiling which
@@ -959,7 +1262,11 @@ async def dub_transcribe(job_id: str):
try:
loop = asyncio.get_running_loop()
try:
segments_result = await loop.run_in_executor(_gpu_pool, _transcribe)
# Bound the whole-file transcribe (#730): a wedged whisperx/CTranslate2
# call would otherwise hold its GPU-pool worker forever and starve
# every other request into a "can't reach backend". run_transcribe_guarded
# also resets the pool on timeout so capacity is restored.
segments_result = await run_transcribe_guarded(_gpu_pool, _transcribe, what="Dub")
except asyncio.CancelledError:
job["aborted"] = True
raise
+86 -17
View File
@@ -170,8 +170,42 @@ async def dub_list_tracks(job_id: str):
return {"tracks": job.get("dubbed_tracks", {})}
def _segments_for_lang(job: dict, lang: "str | None") -> list:
"""Job segments with `text` overlaid from ``job["segments_i18n"][lang]``.
P1.2 ``job["segments"]`` is single-slot: it holds whichever language was
generated LAST, so exporting subtitles for track A after generating track B
emitted B's text under A's language label (the "N identical subtitle
files" class). ``segments_i18n`` ({lang: {segKey: text}}, written by
``dub_generate._sync_job_segments``) preserves each generated track's text;
this overlays it non-destructively when present.
Back-compat: no lang requested, no ``segments_i18n`` on the job (predates
the field), no entry for this lang, or no text for a given segment each
falls back to the segment as-is, i.e. exactly today's behaviour.
Segment keys are the stable id (str) with the list index (str) as the
legacy fallback, mirroring how the map is written.
"""
segments = job.get("segments", [])
if not lang:
return segments
i18n = job.get("segments_i18n")
lang_texts = i18n.get(lang) if isinstance(i18n, dict) else None
if not isinstance(lang_texts, dict) or not lang_texts:
return segments
out = []
for i, seg in enumerate(segments):
key = str(seg.get("id")) if seg.get("id") is not None else str(i)
txt = lang_texts.get(key)
if txt is None:
txt = lang_texts.get(str(i))
out.append(dict(seg, text=txt) if isinstance(txt, str) and txt.strip() else seg)
return out
def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
fitted_segments: "list[dict] | None" = None) -> str | None:
fitted_segments: "list[dict] | None" = None,
lang: "str | None" = None) -> str | None:
"""Build a temp SRT from job segments for use with ffmpeg's subtitles filter.
Returned path is already ffmpeg-filter-safe (plain ASCII basename under exports_dir).
@@ -181,8 +215,11 @@ def _write_burn_srt(job: dict, exports_dir: str, stamp: str, dual: bool,
fitted timeline when provided, cue times come from there instead of
the original ``job["segments"]`` timings, so burned subs track the
retimed video / fitted audio rather than the source timeline.
``lang`` (P1.2): burn the named track's text (see ``_segments_for_lang``)
instead of whatever language generated last.
"""
segments = job.get("segments", [])
segments = _segments_for_lang(job, lang)
if not segments:
return None
if fitted_segments:
@@ -486,7 +523,9 @@ async def dub_download(
# Smart Fit: cue times come from the fitted timeline — that's where the
# dubbed audio actually sits, whether or not the video retime succeeds.
fitted_segments = _fitted_segments_for(job, default_track) if default_track and default_track != "original" else None
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments) if burn_subs else None
# Burn the DEFAULT track's text (P1.2) — it's the audio the viewer hears.
_burn_lang = default_track if default_track and default_track != "original" else None
sub_path = _write_burn_srt(job, exports_dir, stamp, dual, fitted_segments=fitted_segments, lang=_burn_lang) if burn_subs else None
# ── Smart Fit video retime (two-tier) ─────────────────────────────────
# Tier 1 (≤48 chunks): single filter_complex graph inlined into the mux
@@ -1125,20 +1164,40 @@ async def dub_get_audio(job_id: str):
raise HTTPException(status_code=404, detail="Audio file not found")
return FileResponse(audio, media_type="audio/wav")
def _seg_wav_candidates(job: dict, lang: "str | None", seg_keys: tuple) -> list:
"""Per-segment WAV name candidates, language-keyed first (P1.3).
Generation writes ``seg_{lang}_{id}.wav`` now; ``lang`` defaults to the
job's last-generated track. Legacy un-keyed names (``seg_{id}.wav`` /
``seg_{index}.wav``) stay as fallbacks so jobs rendered by previous
builds keep serving their audio these read-only endpoints keep the
permissive fallback that matches their historic behaviour (the strict
single-track gate lives on the generate splice path, where a wrong-
language read would be baked into a track).
"""
lang = lang or job.get("language_code")
keys = []
if lang:
keys.extend(f"{lang}_{k}" for k in seg_keys)
keys.extend(seg_keys)
return keys
@router.get("/dub/preview/{job_id}/{segment_index}")
async def dub_preview_segment(job_id: str, segment_index: int):
async def dub_preview_segment(job_id: str, segment_index: int, lang: str = Query(None)):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Resolve the stable-id-named WAV via the render manifest; fall back to the
# legacy index name for jobs rendered before id-based naming (#185). Each
# candidate is realpath-normalised and containment-checked BEFORE any
# filesystem access, so the guard dominates every path sink.
# Resolve the stable-id-named WAV via the render manifest — language-keyed
# name first (P1.3), then the legacy id/index names for jobs rendered
# before per-language (and before id-based, #185) naming. Each candidate
# is realpath-normalised and containment-checked BEFORE any filesystem
# access, so the guard dominates every path sink.
order = job.get("seg_order") or []
seg_id = order[segment_index] if 0 <= segment_index < len(order) else segment_index
base = os.path.realpath(DUB_DIR)
seg_path = None
for _sid in (seg_id, segment_index):
for _sid in _seg_wav_candidates(job, lang, (seg_id, segment_index)):
cand = os.path.realpath(dub_seg_path(job_id, _sid))
if cand.startswith(base + os.sep) and os.path.exists(cand):
seg_path = cand
@@ -1187,8 +1246,14 @@ async def dub_qc_pass(job_id: str, lang: str = Query(None), drift_threshold: flo
try:
from services.model_manager import _get_gpu_pool
loop = asyncio.get_running_loop()
recognized, engine_id = await loop.run_in_executor(_get_gpu_pool(), _recognize)
from services.asr_backend import ASRTimeoutError, run_transcribe_guarded
recognized, engine_id = await run_transcribe_guarded(
_get_gpu_pool(), _recognize, what="QC",
)
except ASRTimeoutError as e:
# Backend is alive; ASR just couldn't finish in time. 504, not 500/connection.
logger.warning("dub QC ASR pass timed out for %s: %s", job_id, e)
raise HTTPException(status_code=504, detail=str(e))
except Exception as e:
logger.exception("dub QC ASR pass failed for %s", job_id)
raise HTTPException(status_code=500, detail=f"QC transcription failed: {e}")
@@ -1336,13 +1401,16 @@ def _fitted_cue_times(job: dict, lang: str | None) -> list | None:
async def dub_export_srt(
job_id: str,
dual: bool = False,
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
segments = job.get("segments", [])
# P1.2 — text follows the REQUESTED track, not whichever language was
# generated last (job["segments"] is single-slot). Legacy jobs without
# segments_i18n fall back to today's behaviour.
segments = _segments_for_lang(job, lang)
if not segments:
raise HTTPException(status_code=400, detail="No transcript segments available")
@@ -1385,13 +1453,14 @@ def _format_vtt_time(seconds):
async def dub_export_vtt(
job_id: str,
dual: bool = False,
lang: str = Query(None, description="Track language code. When that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
lang: str = Query(None, description="Track language code. Emits that track's text (segments_i18n) when the job carries it; when that track was generated under Smart Fit or stretch_video, cue times come from the fitted timeline."),
):
job = _get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
segments = job.get("segments", [])
# Same per-track text resolution as /dub/srt (see comment there, P1.2).
segments = _segments_for_lang(job, lang)
if not segments:
raise HTTPException(status_code=400, detail="No transcript segments available")
@@ -1421,7 +1490,7 @@ async def dub_export_vtt(
@router.get("/dub/export-segments/{job_id}")
async def dub_export_segments_zip(job_id: str):
async def dub_export_segments_zip(job_id: str, lang: str = Query(None)):
import zipfile
job = _get_job(job_id)
if not job:
@@ -1439,7 +1508,7 @@ async def dub_export_segments_zip(job_id: str):
seg_id = order[i] if i < len(order) else i
# realpath + containment guard before any filesystem access.
seg_path = None
for _sid in (seg_id, i):
for _sid in _seg_wav_candidates(job, lang, (seg_id, i)):
cand = os.path.realpath(dub_seg_path(job_id, _sid))
if cand.startswith(base + os.sep) and os.path.exists(cand):
seg_path = cand
File diff suppressed because it is too large Load Diff
+219 -80
View File
@@ -8,7 +8,7 @@ from fastapi.responses import JSONResponse
from schemas.requests import TranslateRequest
from services.model_manager import _cpu_pool, _gpu_pool
from services.translator import cinematic_available, cinematic_refine_many
from services.translator import cinematic_available, cinematic_refine_many, _cinematic_budget
from api.routers.dub_core import _get_job
router = APIRouter()
@@ -302,15 +302,69 @@ async def dub_translate(req: TranslateRequest):
translated = await loop.run_in_executor(_gpu_pool, _translate_nllb)
if os.environ.get("OMNIVOICE_UNLOAD_NLLB", "1") == "1":
_unload_nllb()
return {"translated": translated, "target_lang": req.target_lang, "source_lang": src_lang,
**_dialect_flags(req, applied=False)}
# Cinematic/Autofit refine + rate-ratio badges must run for NLLB too
# (previously this returned before _maybe_cinematic, so a Cinematic
# pick on NLLB silently produced plain Fast output). Unloading NLLB
# first is fine — the refine LLM is a separate network provider.
return await _maybe_cinematic(translated, req, src_lang, loop)
# OpenAI / Ollama Local LLM Translation
# LLM translation — resolves through the LLM Skills registry: per-skill
# "Dub translation" override → global active provider (Settings → LLM
# Providers). The keys users configure + test in the app now actually
# power this engine; the raw TRANSLATE_* env vars stay working as a
# power-user override so pre-skills setups see zero behavior change.
if provider == "openai":
base_url = os.environ.get("TRANSLATE_BASE_URL")
model_name = os.environ.get("TRANSLATE_MODEL", "gpt-3.5-turbo")
from openai import OpenAI
client = OpenAI(base_url=base_url, api_key=api_key or "local")
from services import llm_skills
llm_timeout = llm_skills._default_timeout()
handle = None
try:
handle = llm_skills.resolve_skill_client("dub_translation")
except Exception: # noqa: BLE001 — resolution must never 500 a translate
logger.exception("dub_translation skill resolution failed; trying env fallback")
if handle is not None:
client = handle.client
model_name = handle.model
llm_timeout = handle.timeout
# The provider-store key never touches env; resolve it so the
# error scrubber below can redact it if a provider echoes it.
try:
from services import llm_providers
api_key = llm_providers.resolve_api_key(
llm_skills.effective_provider("dub_translation")) or api_key
except Exception: # noqa: BLE001 — scrub-key resolution is best-effort
pass
elif os.environ.get("TRANSLATE_BASE_URL") or api_key:
# Legacy env-only setup (no provider configured in-app).
from openai import OpenAI
# max_retries=0: a 429 + long Retry-After must not let one segment's
# SDK call sleep+retry and blow the overall translate wall time.
client = OpenAI(base_url=os.environ.get("TRANSLATE_BASE_URL"),
api_key=api_key or "local", max_retries=0)
model_name = os.environ.get("TRANSLATE_MODEL", "gpt-4o-mini")
else:
# Nothing configured anywhere — name the exact next step instead
# of letting an empty key surface as a raw 401 per segment.
try:
reason = llm_skills.resolve_skill("dub_translation").reason
except Exception: # noqa: BLE001
reason = None
if reason == "disabled":
friendly = (
"The LLM translation engine is turned off — enable the "
"'Dub translation' skill in Settings → LLM Skills, or "
"pick another engine in the Engine dropdown."
)
else:
friendly = (
"The LLM translation engine has no provider configured. "
"Add and test one in Settings → LLM Providers (it powers "
"this engine; route it per-skill in Settings → LLM "
"Skills), or set TRANSLATE_BASE_URL + TRANSLATE_API_KEY "
"+ TRANSLATE_MODEL. Or pick another engine in the "
"Engine dropdown."
)
return JSONResponse(status_code=400, content={"error": friendly})
def _build_prompt(src_code: str, tgt_code: str) -> str:
"""Build a system prompt that resists hallucinations on small
@@ -372,6 +426,7 @@ async def dub_translate(req: TranslateRequest):
res = client.chat.completions.create(
model=model_name,
temperature=0.2, # less drift than default 1.0
timeout=llm_timeout, # bound per call (OMNIVOICE_LLM_TIMEOUT, 45s default)
messages=[
{"role": "system", "content": sys_for_attempt},
{"role": "user", "content": seg.text},
@@ -399,25 +454,38 @@ async def dub_translate(req: TranslateRequest):
seg.id, attempt + 1, e,
)
# Both attempts failed — keep source text + flag error so the
# frontend can surface "fallback to literal" warning.
return {"id": seg.id, "text": seg.text, "error": last_err or "llm-failed"}
# frontend can surface "fallback to literal" warning. Scrub the
# provider error: some OpenAI-compatible providers echo the key
# or a user_id in the body, which must not reach the UI verbatim.
from core.scrub import scrub_provider_error
return {"id": seg.id, "text": seg.text,
"error": scrub_provider_error(last_err, api_key) or "llm-failed"}
tasks = [loop.run_in_executor(_cpu_pool, _translate_llm, seg) for seg in req.segments]
translated = await asyncio.gather(*tasks)
translated.sort(key=lambda x: str(x["id"]))
return {"translated": translated, "target_lang": req.target_lang, "source_lang": src_lang,
**_dialect_flags(req, applied=True)}
# provider="openai" is already an LLM translation — _maybe_cinematic
# skips the reflect/adapt re-refine (already_llm) but still stamps
# rate-ratio badges and runs the bounded Autofit fit pass. Before
# this it returned here, so Cinematic/Autofit on the LLM engine did
# nothing.
return await _maybe_cinematic(translated, req, src_lang, loop, already_llm=True)
# Offline Argos Translate
if provider == "argos" or provider == "libretranslate":
try:
import argostranslate # noqa: F401
except ImportError:
# Single-source the install command from the engine registry so
# this 400 and the proactive Install button in the Engine
# selector can never drift (see translation_engines.install_command).
from services.translation_engines import install_command
cmd = install_command("argos") or "uv pip install argostranslate"
friendly = (
f"The '{provider}' translation engine needs the optional "
f"`argostranslate` Python package, which isn't installed in "
f"this backend. Install it with `uv pip install argostranslate` "
f"(or `pip install argostranslate`) and restart the server, or "
f"this backend. Install it with `{cmd}` "
f"and restart the server, or "
f"switch the Engine dropdown to another provider."
)
return JSONResponse(status_code=400, content={"error": friendly})
@@ -460,8 +528,11 @@ async def dub_translate(req: TranslateRequest):
return results
translated = await loop.run_in_executor(_cpu_pool, _translate_argos)
return {"translated": translated, "target_lang": req.target_lang, "source_lang": src_lang,
**_dialect_flags(req, applied=False)}
# Argos is the DEFAULT engine — routing it through _maybe_cinematic is
# the headline fix: a user who picks Cinematic/Autofit on Argos now
# gets the LLM refine + fit pass (and rate-ratio badges in Fast mode)
# instead of silent plain-Fast output.
return await _maybe_cinematic(translated, req, src_lang, loop)
# Legacy / API Deep_Translator logic.
# Preflight the optional `deep_translator` dep once so we fail with a
@@ -470,11 +541,16 @@ async def dub_translate(req: TranslateRequest):
try:
import deep_translator # noqa: F401
except ImportError:
# Same single-source install command as the Engine selector's Install
# button (translation_engines.install_command) — google/deepl/
# microsoft/mymemory all share the deep_translator package.
from services.translation_engines import install_command
cmd = install_command(provider) or "uv pip install deep_translator"
friendly = (
f"The '{provider}' translation engine needs the optional "
f"`deep_translator` Python package, which isn't installed in "
f"this backend. Install it with `uv pip install deep_translator` "
f"(or `pip install deep_translator`) and restart the server, or "
f"this backend. Install it with `{cmd}` "
f"and restart the server, or "
f"switch the Engine dropdown to Argos (local, bundled), NLLB "
f"(local, heavier), or OpenAI (LLM)."
)
@@ -530,7 +606,11 @@ async def dub_translate(req: TranslateRequest):
)
time.sleep(0.25 * (attempt + 1))
logger.error("translate %s -> %s gave up (provider=%s): %s", src_arg, seg_lc, provider, last_err)
return {"id": seg.id, "text": seg.text, "error": last_err or "unknown"}
# Scrub before it reaches the UI — DeepL/Microsoft errors can echo
# the API key (same class as the OpenAI user_id leak).
from core.scrub import scrub_provider_error
return {"id": seg.id, "text": seg.text,
"error": scrub_provider_error(last_err, _deepl_key or _msft_key or api_key) or "unknown"}
tasks = [loop.run_in_executor(_cpu_pool, _translate_single, seg) for seg in req.segments]
translated = await asyncio.gather(*tasks)
@@ -544,24 +624,19 @@ async def dub_translate(req: TranslateRequest):
return JSONResponse(status_code=500, content={"error": str(e)})
async def _maybe_cinematic(translated, req, src_lang, loop):
"""If quality=cinematic and a usable LLM is configured, run REFLECT+ADAPT.
Otherwise return Fast-mode shape unchanged.
def _stamp_predicted_rate_ratio(translated, req) -> None:
"""Stamp a predicted ``rate_ratio`` on every row that has a known slot.
No LLM needed just the per-language CPS table from ``services/speech_rate``.
The UI's ``seg-rate-badge`` reads it (Fast mode included) to show which
segments will compress hard at generation time, so users can edit text or
pick a heavier quality. Mutates ``translated`` in place; never raises.
"""
quality = (getattr(req, "quality", None) or "fast").lower()
# Stamp the predicted rate_ratio on every translated row that has a
# known slot. Works for Fast mode too — no LLM needed; just the CPS
# table from services/speech_rate. The UI's `seg-rate-badge` reads
# this value and shows users which segments will compress hard at
# generation time, so they can edit text or pick Cinematic quality.
try:
from services.speech_rate import rate_ratio as _predict_rate_ratio
slots = {str(s.id): getattr(s, "slot_seconds", None) for s in req.segments}
for row in translated:
seg_ref = next(
(s for s in req.segments if str(s.id) == str(row["id"])),
None,
)
slot = getattr(seg_ref, "slot_seconds", None) if seg_ref else None
slot = slots.get(str(row["id"]))
text = (row.get("text") or "").strip()
if slot and text and not row.get("error"):
row["rate_ratio"] = round(
@@ -570,19 +645,119 @@ async def _maybe_cinematic(translated, req, src_lang, loop):
except Exception as e:
logger.debug("non-LLM rate_ratio prediction skipped: %s", e)
base = {"translated": translated, "target_lang": req.target_lang, "source_lang": src_lang,
"quality_used": "fast", **_dialect_flags(req, applied=False)}
if quality != "cinematic":
async def _apply_fit_pass(rows, req, slots_by_id, source_by_id, quality, loop, deadline) -> None:
"""Run the Autofit slot-fit pass over ``rows`` concurrently, in place.
Bounded by ``deadline`` (shared with the cinematic refine) so a slow /
rate-limited LLM can't spin the fit pass per-segment unbounded — the old
behavior, which ran one blocking ``adjust_for_slot`` per segment in the
merge loop, outside any budget. Segments still running at the deadline keep
their current text and get ``rate_error='fit-budget'``. Only rows with a
slot + text + no prior error participate.
"""
strict = (quality == "autofit")
items = []
for row in rows:
seg_id = str(row["id"])
slot = slots_by_id.get(seg_id)
text = row.get("text") or ""
if slot and text and not row.get("error"):
items.append((seg_id, text, float(slot), req.target_lang,
source_by_id.get(seg_id), strict))
if not items:
return
try:
from services.speech_rate import adjust_for_slot_many
fits = await adjust_for_slot_many(
items, executor=_cpu_pool, deadline=deadline, loop=loop,
)
except Exception as e:
logger.warning("rate-fit pass skipped: %s", e)
return
for row in rows:
f = fits.get(str(row["id"]))
if not f:
continue
if f.get("text"):
row["text"] = f["text"]
if f.get("rate_ratio") is not None:
row["rate_ratio"] = f["rate_ratio"]
if f.get("error"):
row["rate_error"] = f["error"]
async def _maybe_cinematic(translated, req, src_lang, loop, *, already_llm=False):
"""Post-process a literal translation into Cinematic/Autofit output.
Runs for EVERY provider now (Argos/NLLB/Google//OpenAI). The three
LLM-independent branches (nllb/argos) and the openai branch used to return
*before* reaching this, so a Cinematic/Autofit pick on them including the
DEFAULT Argos engine silently produced plain Fast output with a success
toast. Fast mode still returns the plain translation (plus rate-ratio badges).
``already_llm`` (provider="openai"): the translation was itself produced by
an LLM, so the REFLECT+ADAPT *re*-refine is skipped, but the bounded Autofit
fit pass + rate-ratio stamping still run, and the dialect the translate
prompt already baked in is reported as applied.
"""
quality = (getattr(req, "quality", None) or "fast").lower()
_stamp_predicted_rate_ratio(translated, req)
# #280 item 2 — regional dialect hint, guarded against a stale dialect from
# another language. For already_llm the initial translate prompt already
# applied it, so it's reported applied in the Fast-shape base too.
dialect_hint = ""
_dialect = getattr(req, "dialect", None)
if _dialect and str(_dialect).lower().startswith(str(req.target_lang).lower()[:2]):
dialect_hint = dialect_clause(_dialect)
base = {"translated": translated, "target_lang": req.target_lang, "source_lang": src_lang,
"quality_used": "fast",
**_dialect_flags(req, applied=(already_llm and bool(dialect_hint)))}
# Fast (and anything unrecognised) returns the plain translation unchanged.
if quality not in ("cinematic", "autofit"):
return base
source_by_id: dict[str, str] = {str(s.id): s.text for s in req.segments}
slots_by_id = {
str(s.id): getattr(s, "slot_seconds", None)
for s in req.segments
if getattr(s, "slot_seconds", None)
}
# One wall-clock deadline shared by the whole LLM phase (refine + fit), so a
# slow/rate-limited provider can't run either pass unbounded. <=0 disables.
budget = _cinematic_budget()
deadline = (loop.time() + budget) if budget and budget > 0 else None
# provider="openai": already an LLM translation → skip REFLECT+ADAPT, keep
# the rate-ratio badges, still run the bounded fit pass.
if already_llm:
merged = []
for row in translated:
out = {"id": row["id"],
"text": row.get("text", "") or "",
"literal": row.get("text", "") or ""}
if row.get("error"):
out["error"] = row["error"]
if "rate_ratio" in row:
out["rate_ratio"] = row["rate_ratio"]
merged.append(out)
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
return {"translated": merged, "target_lang": req.target_lang,
"source_lang": src_lang, "quality_used": quality,
**_dialect_flags(req, applied=bool(dialect_hint))}
# Non-LLM provider → the reflect/adapt refine needs a separately-configured
# LLM (Settings → LLM Providers). Without one, degrade to Fast with a flag.
if not cinematic_available():
logger.warning("cinematic requested but no LLM configured — returning Fast result.")
logger.warning("%s requested but no LLM configured — returning Fast result.", quality)
base["cinematic_skipped"] = "no-llm-configured"
return base
# Build a map from id → original segment (to fetch source text + direction).
source_by_id: dict[str, str] = {str(s.id): s.text for s in req.segments}
directions: dict[str, str] = {
str(s.id): s.direction
for s in req.segments
@@ -590,7 +765,7 @@ async def _maybe_cinematic(translated, req, src_lang, loop):
}
pairs = []
passthrough_index = {}
for i, row in enumerate(translated):
for row in translated:
seg_id = str(row["id"])
literal = row.get("text", "") or ""
if row.get("error") or not literal.strip():
@@ -601,12 +776,6 @@ async def _maybe_cinematic(translated, req, src_lang, loop):
if not pairs:
return base
# #280 item 2: thread the regional-dialect hint into the reflect/adapt
# prompts. Guard against a stale dialect from another language.
dialect_hint = ""
if req.dialect and str(req.dialect).lower().startswith(str(req.target_lang).lower()[:2]):
dialect_hint = dialect_clause(req.dialect)
refined = await cinematic_refine_many(
pairs,
source_lang=src_lang,
@@ -618,16 +787,6 @@ async def _maybe_cinematic(translated, req, src_lang, loop):
)
refined_by_id = {r["id"]: r for r in refined}
# Phase 4.4 — speech-rate fit pass. Segment boundaries aren't in the
# translate request (by design — translator is boundary-agnostic), so we
# only run it when the caller supplied `slot_seconds` on each segment.
# The frontend populates this for Cinematic calls from the edit view.
slots_by_id = {
str(s.id): getattr(s, "slot_seconds", None)
for s in req.segments
if getattr(s, "slot_seconds", None)
}
merged = []
for row in translated:
seg_id = str(row["id"])
@@ -646,35 +805,15 @@ async def _maybe_cinematic(translated, req, src_lang, loop):
}
if r.get("error"):
out["error"] = r["error"]
# Optional slot-fit pass — only when the caller asked for cinematic
# *and* provided a slot. Runs best-effort; no-LLM or mid-loop failure
# just leaves the cinematic text untouched.
slot = slots_by_id.get(seg_id)
if slot and out["text"]:
try:
from services.speech_rate import adjust_for_slot
fit = await asyncio.to_thread(
adjust_for_slot,
out["text"],
slot_seconds=float(slot),
target_lang=req.target_lang,
source_text=source_by_id.get(seg_id),
)
if fit.get("text"):
out["text"] = fit["text"]
out["rate_ratio"] = fit.get("rate_ratio")
if fit.get("error"):
out["rate_error"] = fit["error"]
except Exception as e:
logger.warning("rate-fit skipped for %s: %s", seg_id, e)
merged.append(out)
# Phase 4.4 speech-rate fit pass — now concurrent + bounded (see helper).
await _apply_fit_pass(merged, req, slots_by_id, source_by_id, quality, loop, deadline)
return {
"translated": merged,
"target_lang": req.target_lang,
"source_lang": src_lang,
"quality_used": "cinematic",
"quality_used": quality,
**_dialect_flags(req, applied=bool(dialect_hint)),
}
+189
View File
@@ -15,6 +15,9 @@ Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`,
`OMNIVOICE_LLM_BACKEND`) still win over the UI choice so power-users can pin
a backend without Settings silently undoing it.
"""
import os
import re
import threading
from time import perf_counter
from fastapi import APIRouter, Depends, HTTPException
@@ -261,9 +264,177 @@ def engine_health(engine_id: str):
}
# ── Real-synthesis self-test (in-process TTS engines) ──────────────────────
#
# ``/health`` above is a liveness/import probe — for an in-process backend it
# only calls ``is_available()`` and the UI labels the result "deps OK". This
# route goes one step further: for an AVAILABLE, IN-PROCESS TTS engine it runs
# a *tiny real synthesis* from a fixed short phrase and reports duration +
# sample-rate + sample count, proving the engine actually emits audio rather
# than merely importing. The Compat Matrix's "Self-test" button calls it.
#
# Guardrails (kept identical across macOS/Windows/Linux per the default-feature
# rule — the phrase, timeout and gating don't branch on OS):
# * TTS family + available + in-process only. Subprocess engines keep their
# spawn-and-ping ``health_check`` (a real synth there is a sidecar
# cold-start — out of scope for a click-to-test affordance).
# * Bounded wall-clock timeout (``OMNIVOICE_SELFTEST_TIMEOUT_S``, default 90s):
# a runaway synth returns ``ok=False`` / ``timed_out=True`` instead of
# hanging the Settings panel. The orphaned worker is best-effort daemon.
# * A process-wide lock serialises self-tests so a click-storm can't stack
# concurrent model loads.
# * Only ever on user click (POST) — never on Settings load. Loopback-gated.
# Deliberately short + ASCII so the synth stays CPU-cheap and the phrase never
# trips the no-hardcoded-CJK guard.
_SELFTEST_PHRASE = "OmniVoice engine self test."
_SELFTEST_LOCK = threading.Lock()
def _selftest_timeout_s() -> float:
try:
return max(1.0, float(os.environ.get("OMNIVOICE_SELFTEST_TIMEOUT_S", "90")))
except (TypeError, ValueError):
return 90.0
def _sample_count(audio) -> int:
"""Total sample count of an engine's ``generate()`` return, tolerant of
torch.Tensor / numpy.ndarray / list shapes. 0 when it can't be measured."""
try:
shape = getattr(audio, "shape", None)
if shape is not None and len(shape) > 0:
return int(shape[-1])
return int(len(audio))
except Exception:
return 0
def _run_synth_bounded(backend, timeout_s: float) -> dict | None:
"""Run one tiny synthesis in a daemon thread, bounded by ``timeout_s``.
Returns ``{"audio": .., "duration_ms": ..}`` on success, ``{"error": exc}``
on a synth exception, or ``None`` when the timeout elapsed (worker left
running best-effort Python threads can't be force-killed)."""
box: dict = {}
def _worker():
t0 = perf_counter()
try:
audio = backend.generate(_SELFTEST_PHRASE, language="en", num_step=8)
box["audio"] = audio
except Exception as exc: # noqa: BLE001 — surfaced to the caller as ok=False
box["error"] = exc
finally:
box["duration_ms"] = (perf_counter() - t0) * 1000.0
th = threading.Thread(target=_worker, name="engine-selftest", daemon=True)
th.start()
th.join(timeout_s)
if th.is_alive():
return None
return box
class SelfTestResponse(BaseModel):
id: str
ok: bool
message: str
duration_ms: float
sample_rate: int | None = None
num_samples: int | None = None
audio_seconds: float | None = None
timed_out: bool = False
@router.post(
"/engines/{engine_id}/selftest",
response_model=SelfTestResponse,
dependencies=[Depends(require_loopback)],
)
def engine_selftest(engine_id: str):
"""Run a bounded, real synthesis on an available in-process TTS engine.
404 for an unknown TTS id; 400 when the engine is subprocess-isolated or
not currently available (a real synth on either is meaningless). Never
raises through to a 500 on a synth failure the exception is captured into
``ok=False`` / ``message`` so the panel renders a per-row failure."""
if engine_id not in tts_backend._REGISTRY:
raise HTTPException(
status_code=404,
detail=f"unknown TTS engine id: {engine_id!r}",
)
cls = tts_backend._REGISTRY[engine_id]
if getattr(cls, "_is_subprocess_isolated", False):
raise HTTPException(
status_code=400,
detail=(
f"{engine_id} is subprocess-isolated — self-test runs real "
"synthesis for in-process engines only. Use Test engine "
"(spawn-and-ping) for subprocess engines."
),
)
try:
ok, msg = cls.is_available()
except Exception as exc: # noqa: BLE001
ok, msg = False, f"{type(exc).__name__}: {exc}"
if not ok:
raise HTTPException(
status_code=400,
detail=(
f"{engine_id} is not available: {tts_backend._mask_hf_tokens(msg)}. "
"Install/enable the engine, then self-test."
),
)
timeout_s = _selftest_timeout_s()
# Serialise so a click-storm can't stack concurrent model loads.
with _SELFTEST_LOCK:
backend = _get_engine_instance(cls)
res = _run_synth_bounded(backend, timeout_s)
if res is None:
return SelfTestResponse(
id=engine_id,
ok=False,
message=f"timed out after {timeout_s:.0f}s (model still loading?)",
duration_ms=timeout_s * 1000.0,
timed_out=True,
)
if "error" in res:
exc = res["error"]
return SelfTestResponse(
id=engine_id,
ok=False,
message=tts_backend._mask_hf_tokens(f"{type(exc).__name__}: {exc}"),
duration_ms=res.get("duration_ms", 0.0),
)
n = _sample_count(res.get("audio"))
try:
sr = int(getattr(backend, "sample_rate", 0) or 0) or None
except Exception:
sr = None
secs = round(n / sr, 3) if (sr and n) else None
return SelfTestResponse(
id=engine_id,
ok=n > 0,
message="synthesized" if n > 0 else "engine returned no audio",
duration_ms=res["duration_ms"],
sample_rate=sr,
num_samples=n or None,
audio_seconds=secs,
)
class SelectEngineRequest(BaseModel):
family: str # "tts" | "asr" | "llm"
backend_id: str
# Only meaningful for family="tts", backend_id="mlx-audio" (#981) — picks
# which of mlx-audio's curated models is actually loaded. A curated key
# ("kokoro") or a raw HF repo id ("mlx-community/Kokoro-82M-bf16") — the
# same tolerance MLXAudioBackend.__init__ already has. Ignored otherwise.
model_id: str | None = None
class SelectEngineResponse(BaseModel):
@@ -306,6 +477,24 @@ def select_engine(req: SelectEngineRequest):
f"Backend {req.backend_id} can't run on this machine: {why}. "
f"Pick an engine with a CPU path, or one that supports this host's GPU.",
)
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
# persist the model pick alongside the backend id so the UI can actually
# select which curated model gets loaded (previously it always defaulted
# to Kokoro no matter what the user downloaded in Settings → Models).
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
# tolerance MLXAudioBackend.__init__ already has for power users.
# Anything else (typo'd key, malformed id) is rejected outright
# rather than silently persisted as a "custom repo" that then fails
# to resolve at load time.
if req.model_id not in known_keys and not re.fullmatch(r"[\w.-]+/[\w.-]+", req.model_id):
raise HTTPException(
400,
f"Unknown mlx-audio model: {req.model_id!r}. Expected one of "
f"{sorted(known_keys)} or a HF repo id like 'owner/name'.",
)
prefs.set_("mlx_audio_model_id", req.model_id)
prefs.set_(pref_key, req.backend_id)
return {
"family": req.family,
+521 -34
View File
@@ -1,7 +1,9 @@
import os
import io
import re
import uuid
import time
import random
import asyncio
import tempfile
import contextlib
@@ -11,16 +13,36 @@ from typing import Optional
from fastapi import APIRouter, File, Form, UploadFile, HTTPException
from fastapi.responses import StreamingResponse
from core.db import db_conn
import sqlite3
from core.db import db_conn, ensure_schema
from core.config import OUTPUTS_DIR, VOICES_DIR
from services.model_manager import get_model, _gpu_pool
import functools
from services.model_manager import (
get_model, _gpu_pool, run_on_gpu_pool_guarded, GpuJobTimeoutError,
)
from services.audio_io import _safe_torchaudio_save
from core import event_bus
from omnivoice.utils.voice_design import heal_design_instruct
router = APIRouter()
logger = logging.getLogger("omnivoice.generate")
def _profile_instruct(row):
"""Validator-safe instruct for a stored profile row.
Sanitizes the persisted instruct (dropping the ``"[object Object]"``
sentinel / freeform prose that older builds saved) and, for a design row,
rebuilds the tags from ``vd_states`` when the stored value is unusable so
a poisoned/legacy profile never 400-s generation (#550 #571 #594 #596).
"""
try:
vd = row["vd_states"]
except (KeyError, IndexError):
vd = None
return heal_design_instruct(row["instruct"], vd)
def _render_with_pauses(gen_span, segments, sample_rate):
"""Synthesize ``[(text, pause_ms), ...]`` spans and stitch silence between
them (issue #276).
@@ -60,14 +82,31 @@ def _render_with_pauses(gen_span, segments, sample_rate):
return torch.cat(parts, dim=-1)
def _sanitize_audio(audio_out):
"""Replace non-finite samples (NaN / ±inf) with silence so a model glitch
can't produce an unreadable WAV (#629). Returns the input unchanged when it's
already finite or isn't a tensor. Never raises."""
try:
import torch
if torch.is_tensor(audio_out) and not bool(torch.isfinite(audio_out).all()):
logger.warning(
"Generated audio contained non-finite samples (NaN/inf) — "
"sanitizing to silence to keep the WAV decodable (#629)."
)
return torch.nan_to_num(audio_out, nan=0.0, posinf=0.0, neginf=0.0)
except Exception:
pass
return audio_out
def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering=False):
"""Shared post-DSP for /generate: preset validation → mastering →
effect chain loudness normalization.
``skip_mastering`` honors a backend's ``applies_own_mastering`` flag
(issue #312): studio engines (e.g. VoxCPM2's native 48 kHz output)
opt out of the broadcast Compressor + Reverb chain that's tuned for
OmniVoice's 24 kHz clone output. Loudness normalization still runs —
opt out of the broadcast highpass + Compressor pre-stage that's tuned
for OmniVoice's 24 kHz clone output. Loudness normalization still runs —
it's a benign peak scale. Mirrors ``_run_tts`` in openai_compat.py.
"""
from services.audio_dsp import (
@@ -75,6 +114,14 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
apply_effects_chain, get_effect_chain,
)
# #629: a numerical glitch in the model (observed on MPS) can leave NaN/±inf
# samples, which write an unreadable WAV that then fails decoding with an
# opaque "ffmpeg returned error code: 183 / Invalid data" — surfaced to the
# user as a misleading "ran out of memory". Replace non-finite samples with
# silence here, before any DSP/encode touches the audio, so the output is
# always a valid WAV. Covers the raw path too (it returns just below).
audio_out = _sanitize_audio(audio_out)
preset = effect_preset or "broadcast"
if preset not in EFFECT_PRESETS:
raise ValueError(
@@ -96,6 +143,158 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
return normalize_audio(audio_out, target_dBFS=-2.0)
def _safe_exc_text(e: BaseException) -> str:
"""``f"{type(e).__name__}: {e}"`` — the house style used for
unrecognized-error formatting throughout the backend (grep
``type(e).__name__`` in settings.py / asr_backend.py / model_manager.py
/ engines.py) with a guard against leaking a raw container repr.
#977: an AssertionError raised deep inside a vendored dependency
(mlx-audio's Kokoro pipeline) had ``.args`` shaped like
``('du', {'a': 'American English', ...})`` a tuple containing a dict.
``str(e)`` on that renders the WHOLE table straight into the user-facing
message. Any engine's ``generate()`` can raise something shaped like
this (not just Kokoro), so guard generically: if any element of
``e.args`` is a container rather than a plain string, don't interpolate
``str(e)`` at all name the exception type and point at the log
instead.
"""
args = getattr(e, "args", ())
if any(isinstance(a, (dict, list, tuple, set, frozenset)) for a in args):
return f"{type(e).__name__} — see Settings → Logs → Backend for details"
return f"{type(e).__name__}: {e}"
def _exception_chain(e):
"""Yield ``e`` plus every ``__cause__``/``__context__`` beneath it
(cycle-safe). Engines and hub libraries routinely wrap the original
transport/allocator error, so classification must look at the whole
chain, not just the outermost message."""
seen = set()
stack = [e]
while stack:
exc = stack.pop()
if exc is None or id(exc) in seen:
continue
seen.add(id(exc))
yield exc
stack.append(exc.__cause__)
stack.append(exc.__context__)
# #880: transport-level exception type names from httpx (huggingface_hub ≥1.x
# downloads over it) and requests/urllib3 (older engine deps). Any of these
# anywhere in the exception chain means the network — not memory — killed the
# generation.
_NETWORK_EXC_NAMES = frozenset({
# httpx
"ConnectError", "ConnectTimeout", "ReadTimeout", "ReadError",
"WriteError", "WriteTimeout", "PoolTimeout", "NetworkError",
"TransportError", "RemoteProtocolError", "ProxyError", "CloseError",
# requests / urllib3
"ConnectionError", "ChunkedEncodingError", "MaxRetryError",
"NewConnectionError", "ProtocolError",
# stdlib socket-level drops mid-download
"ConnectionResetError", "ConnectionAbortedError", "ConnectionRefusedError",
# huggingface_hub: failed first-use download with nothing in the disk cache
"LocalEntryNotFoundError",
})
# Same class, but the transport error was stringified into a wrapper message
# (so the type name is gone). All lowercase; matched against .lower().
_NETWORK_MSG_SIGNATURES = (
"client has been closed", # httpx closed-client lifecycle error (#880)
"cannot send a request", # httpx: same error, message head
"connection error", # requests / huggingface_hub wording
"connection reset", # ECONNRESET mid-download
"read timed out", # requests/urllib3 timeout wording
"max retries exceeded", # urllib3 retry exhaustion
"temporary failure in name resolution", # DNS down (glibc)
"name or service not known", # DNS down (glibc)
"getaddrinfo failed", # DNS down (Windows)
)
def _is_network_failure(e) -> bool:
"""True iff the failure (anywhere in its chain) is an HTTP-client
lifecycle / network-transport error e.g. a first-use model download
from the HF Hub dying mid-generation (#880)."""
for exc in _exception_chain(e):
if type(exc).__name__ in _NETWORK_EXC_NAMES:
return True
low = str(exc).lower()
if any(sig in low for sig in _NETWORK_MSG_SIGNATURES):
return True
return False
# Signatures of an *actual* out-of-memory condition. All lowercase.
_OOM_MSG_SIGNATURES = (
"out of memory", # CUDA / MPS / generic torch wording
"not enough memory", # torch CPU DefaultCPUAllocator
"cannot allocate memory", # OS-level ENOMEM
"std::bad_alloc", # C++ allocator failure
"cublas_status_alloc_failed", # cuBLAS workspace allocation
"cuda_error_out_of_memory", # raw CUDA driver error name
"paging file is too small", # Windows [WinError 1455] mapping DLLs
)
def _is_oom_failure(e) -> bool:
"""True iff the failure (anywhere in its chain) actually looks like an
out-of-memory condition the only case where the Flush hint is honest."""
for exc in _exception_chain(e):
if isinstance(exc, MemoryError):
return True
# torch.cuda.OutOfMemoryError subclasses RuntimeError; match by name
# so this needs no torch import (and covers other frameworks' twins).
if type(exc).__name__ == "OutOfMemoryError":
return True
low = str(exc).lower()
if any(sig in low for sig in _OOM_MSG_SIGNATURES):
return True
return False
# #919: an engine that requires a model path / env var which isn't set (or is
# set to a directory missing its model files) fails with a *configuration*
# error, not a runtime one. The reporting user selected sherpa-onnx and hit
# "OMNIVOICE_SHERPA_MODEL not set. Point it to a sherpa-onnx TTS model
# directory …" — a pure setup problem — yet the OOM catch-all told them (on a
# 63 GB-RAM box) to press Flush for memory they never ran out of. Classify the
# whole CLASS of "engine not configured / required env var not set" errors so
# any current or future opt-in engine (sherpa/Confucius4/dots/MOSS …) surfaces
# actionable setup guidance instead of the memory hint. All lowercase; matched
# over the whole exception chain (engines wrap the original error).
_CONFIG_MSG_SIGNATURES = (
"not set. point it to", # sherpa: OMNIVOICE_SHERPA_MODEL not set
"no model.onnx found in", # sherpa: dir set but the model file is missing
"not configured", # generic "engine not configured" wording
"venv not found. set", # confucius4/dots/MOSS dedicated-venv opt-ins
"unavailable: omnivoice_", # is_available() reason wrapped by _ensure_loaded
)
# An OMNIVOICE_* engine env var named alongside "not set" / "point it to" /
# "set omnivoice_…" is the strongest config-missing signal and generalizes to
# any engine gated on such a var (issue #919 class).
_CONFIG_ENV_RE = re.compile(r"omnivoice_[a-z0-9_]+")
def _is_config_failure(e) -> bool:
"""True iff the failure is a *configuration* problem — a required engine
model path / env var that isn't set (or points nowhere) — rather than a
runtime fault. The remedy is to set the value, never to Flush VRAM."""
for exc in _exception_chain(e):
low = str(exc).lower()
if any(sig in low for sig in _CONFIG_MSG_SIGNATURES):
return True
if _CONFIG_ENV_RE.search(low) and (
"not set" in low or "point it to" in low or "set omnivoice_" in low
):
return True
return False
def _oom_friendly_reraise(e):
"""Best-effort cache flush + the user-facing OOM hint shared by both
inference paths."""
@@ -127,10 +326,116 @@ def _oom_friendly_reraise(e):
f"or run `chmod +x` on the engine binary named in the error. "
f"Underlying error: {e}"
) from e
# #629: a decode/ffmpeg failure on the rendered audio is NOT out of memory —
# it's unreadable audio (usually a transient numerical glitch). Say so rather
# than sending the user down the OOM path.
if "ffmpeg returned error" in es or "Decoding failed" in es or "Invalid data found" in es:
raise RuntimeError(
f"The engine produced unreadable audio (a decode step failed) — this is "
f"usually a transient glitch. Use the Flush button to reload the model, "
f"then regenerate. Underlying error: {e}"
) from e
# #664: a bad voice-design instruct (free-form prose, mixed EN/ZH, or
# conflicting tags) raises "Unsupported instruct items …" / "Cannot mix …
# in a single instruct" / "Conflicting instruct items …" from omnivoice's
# _resolve_instruct. That's a USER-INPUT validation error, not an OOM. Match
# on the message signature (NOT the type — a lower layer can wrap the original
# ValueError, which is why the route's `except ValueError` guard misses it)
# and re-raise as a clean ValueError so the route returns a 400 with the
# instruct guidance, instead of a 500 telling the user to Flush for memory
# they never ran out of. (Complements the client-side guard in #658/#612.)
_low = es.lower()
if ("unsupported instruct items" in _low
or "conflicting instruct items" in _low
or "in a single instruct" in _low):
raise ValueError(es) from e
# #705: a corrupt or wrong-architecture native component (a .dll / .pyd / .exe
# — torch, ffmpeg, or a bundled engine binary) fails to load/spawn on Windows
# with "[WinError 193] %1 is not a valid Win32 application". That is NOT OOM,
# and Flush won't help — reinstalling/repairing the component is the real fix.
if "[winerror 193]" in _low or "is not a valid win32 application" in _low:
raise RuntimeError(
f"A native component (a DLL / .pyd / .exe — e.g. torch, ffmpeg, or an "
f"engine binary) is corrupt or built for the wrong architecture "
f"([WinError 193]). Reinstall or repair that component — the Flush "
f"button won't help here. Underlying error: {e}"
) from e
# #715: a "[Errno 32] Broken pipe" (BrokenPipeError) surfacing from
# generation is NOT out of memory — it means the backend's stdout/stderr
# pipe to the desktop shell that launched it closed mid-render (an orphaned
# backend whose parent shell exited or relaunched). main.py wraps
# sys.stdout/stderr to swallow EPIPE, but a C-level write inside the native
# engine/torch can still raise one past that guard. Flush won't help —
# relaunching the app re-parents the backend to a live shell.
# #756: the GPU's compute capability isn't in this PyTorch build's arch list,
# so CUDA can't launch kernels ("no kernel image is available for execution").
# NOT OOM. get_best_device() now falls back to CPU up front, but classify the
# raw error too in case CUDA was forced (OMNIVOICE_FORCE_CUDA) or a sub-path
# still ran on the GPU — point at the real fix, not the Flush button.
if "no kernel image is available" in _low:
raise RuntimeError(
f"Your GPU isn't supported by the installed PyTorch build (CUDA can't "
f"launch kernels for its compute capability). Switch the compute device "
f"to CPU in Settings, or install a matching PyTorch (e.g. a cu128 build "
f"for newer GPUs). The Flush button won't help. Underlying error: {e}"
) from e
if isinstance(e, BrokenPipeError) or "broken pipe" in _low or "errno 32" in _low:
raise RuntimeError(
f"The backend lost its output pipe mid-generation — the desktop app "
f"that launched it closed or relaunched ([Errno 32] Broken pipe). "
f"Restart the app and try again; the Flush button won't help here. "
f"Underlying error: {e}"
) from e
# #880: an httpx/requests transport failure surfacing from generation —
# most commonly a first-use model download from the HF Hub dying with
# httpx's "Cannot send a request, as the client has been closed" (the
# shared client got closed mid-lifecycle), a connect/read timeout, or a
# dropped connection — is NOT out of memory. The model never finished
# loading, so Flush is the wrong remedy; retrying is. Matched over the
# whole exception chain (type names + stringified signatures) because
# engines wrap the original transport error.
if _is_network_failure(e):
raise RuntimeError(
f"A model download or network call failed mid-generation (usually "
f"the engine fetching its model files on first use). This is a "
f"network problem, not a memory problem — flushing VRAM won't "
f"help. Retry the generation; if it keeps failing, check your "
f"internet connection and any HF_ENDPOINT/mirror setting. "
f"Underlying error: {e}"
) from e
# #919: a required engine model path / env var that isn't set is a pure
# CONFIGURATION problem, not a runtime one. sherpa-onnx's
# "OMNIVOICE_SHERPA_MODEL not set. Point it to …" used to fall through to
# the OOM catch-all, telling a user with 63 GB of RAM to press Flush. Point
# at the real fix — set the variable — and never mention memory or Flush.
# The underlying error already names the exact variable + what to point it
# at (and Settings → Engines shows a copy-paste setup line), so keep it
# front-and-center. Checked before the OOM branch so a config error can
# never be mislabeled as memory.
if _is_config_failure(e):
raise RuntimeError(
f"This TTS engine isn't set up yet — it needs a model path or "
f"environment variable that isn't configured, so nothing was "
f"generated. Set it as the underlying error describes (it names the "
f"exact variable and what to point it at), then restart OmniVoice — "
f"or pick a ready engine in Settings → Engines. This is a setup "
f"problem, not a memory one. Underlying error: {e}"
) from e
# #880 (the class bug): the OOM hint used to be the catch-all fallback,
# so ANY unrecognized error told the user to press Flush for memory they
# never ran out of. Only claim OOM when something in the chain actually
# looks like one; everything else surfaces as what it is — unrecognized —
# with the real error front and center.
if _is_oom_failure(e):
raise RuntimeError(
f"TTS engine stopped mid-generation. This usually means it ran out of memory. "
f"Try the Flush button to reload the model, then regenerate. Underlying error: {e}"
) from e
raise RuntimeError(
f"TTS engine stopped mid-generation. This usually means it ran out of memory. "
f"Try the Flush button to reload the model, then regenerate. Underlying error: {e}"
)
f"TTS engine stopped mid-generation with an error OmniVoice doesn't "
f"recognize. Retry once; if it keeps failing, please report it with "
f"the full trace. Underlying error: {_safe_exc_text(e)}"
) from e
def _run_inference(
@@ -293,6 +598,32 @@ def _run_backend_inference(
_oom_friendly_reraise(e)
def _persist_profile_ref_text(profile_id: str, ref_text: str) -> None:
"""Cache an auto-transcribed reference transcript onto its profile row.
#1032 perf regression: profiles saved without a transcript re-ran a FULL
ASR model load + transcribe on every /generate (the #308 auto-transcribe
path). Persisting the first transcript makes subsequent generates read it
from the row like a user-entered one. The guarded UPDATE only ever fills
an empty column it can never overwrite a transcript the user typed or a
lock wrote and a failure is logged, never raised (best-effort, same
contract as the transcribe itself)."""
try:
with db_conn() as conn:
updated = conn.execute(
"UPDATE voice_profiles SET ref_text=? "
"WHERE id=? AND (ref_text IS NULL OR ref_text='')",
(ref_text, profile_id),
).rowcount
if updated:
event_bus.emit("profiles", {"action": "updated", "id": profile_id})
except Exception as e: # noqa: BLE001 — cache write must not break generate
logger.warning(
"could not persist auto-transcribed ref_text onto profile %s: %s",
profile_id, e,
)
@router.post("/generate")
async def generate_speech(
text: str = Form(...),
@@ -318,7 +649,21 @@ async def generate_speech(
# boundaries and crossfaded. 0 disables chunking (whole text to engine).
max_chunk_chars: int = Form(800, ge=0),
crossfade_ms: int = Form(50, ge=0, le=1000),
# Expressive-TTS Spec 01: apply the user pronunciation dictionary + inline
# [[…]] overrides to the text before synthesis. Default ON; the global
# OMNIVOICE_PRONUNCIATION pref can disable it for power users. Omitting it
# with an empty dictionary is byte-identical to legacy behavior.
pronounce: bool = Form(True),
):
# #502: NFC-normalize the input text so decomposed (NFD) diacritics — common
# in pasted Vietnamese and other Latin-with-marks text — are composed to the
# single codepoints the tokenizer/model expect, instead of base-letter +
# combining-mark sequences that render as distorted/garbled speech. NFC is a
# no-op for already-composed text; mirrors the duration estimator
# (utils/duration.py) so the estimate and the synthesis see the same text.
import unicodedata
text = unicodedata.normalize("NFC", text)
# ── Engine resolution (issue #312) ──────────────────────────────────────
# The request runs on the engine selected in Settings (POST /engines/select,
# env var OMNIVOICE_TTS_BACKEND wins), or an explicit per-request `engine`
@@ -374,11 +719,51 @@ async def generate_speech(
raise HTTPException(status_code=400, detail=_routing["routing_reason"])
_routing_notice = routing_notice(_routing) # (status, reason) or None
# ── #1033/#1037: warm the engine under the LOAD budget, not the generate
# budget. A cold adapter lazily loads (and possibly downloads multi-GB
# weights) inside generate(), so a fresh install's first request burned
# its whole OMNIVOICE_GENERATE_TIMEOUT_S window on the download and died
# with a misleading "too heavy for the available compute" 503 (#1014
# measured it: 0% GPU util for the full 300s). Model loading gets its own,
# larger budget (OMNIVOICE_MODEL_LOAD_TIMEOUT, default 1200s) — the same
# split get_model() already has for the native engine. Once warm, this is
# a no-op per request.
if _backend is not None:
from services.model_manager import _model_load_timeout
try:
await run_on_gpu_pool_guarded(
_backend.ensure_ready,
what=f"TTS engine '{engine_id}' model load",
timeout=_model_load_timeout(),
)
# Builtin TimeoutError base, not GpuJobTimeoutError — reload-proof
# class identity (see the twin catch in openai_compat.py).
except TimeoutError as exc:
logger.warning("engine load exceeded the model-load budget: %s", exc)
raise HTTPException(
status_code=503,
detail=(
f"TTS engine '{engine_id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the "
f"weight download is slow or stalled (check Settings → Models "
f"for progress), not that generation failed. Retry once the "
f"model shows as installed."
),
) from exc
ref_audio_path = None
cleanup_ref = False
used_seed = seed
resolved_profile_id = None
history_mode = None # profile.kind when a profile drives; else inferred at insert
# #1032: profile id to persist an auto-transcribed reference transcript to.
# Set only for a plain (unlocked) clone profile whose stored ref_text is
# empty — the case where every /generate re-ran a full ASR model load +
# transcribe of the same clip. Locked profiles are excluded (their ref
# audio is the locked take, and unlocking would leave a mismatched
# transcript paired with the original reference); design profiles are
# excluded (a re-render replaces the sample, stranding a stale transcript).
persist_ref_text_profile_id = None
if profile_id:
with db_conn() as conn:
@@ -400,7 +785,7 @@ async def generate_speech(
if not ref_text:
ref_text = row["ref_text"]
if not instruct:
instruct = row["instruct"]
instruct = _profile_instruct(row)
if used_seed is None and row["seed"] is not None:
used_seed = row["seed"]
elif profile_kind == "design":
@@ -410,26 +795,45 @@ async def generate_speech(
if ref_audio_path and not ref_text and row["ref_text"]:
ref_text = row["ref_text"]
if not instruct:
instruct = row["instruct"]
instruct = _profile_instruct(row)
if used_seed is None and row["seed"] is not None:
used_seed = row["seed"]
elif row["instruct"] and not row["is_locked"] and not row["ref_audio_path"]:
# Legacy design-shaped row (pre-0004 archetype materialization
# failure path): instruct-only conditioning.
if not instruct:
instruct = row["instruct"]
instruct = _profile_instruct(row)
if used_seed is None and row["seed"] is not None:
used_seed = row["seed"]
else:
ref_audio_path = os.path.join(VOICES_DIR, row["ref_audio_path"]) if row["ref_audio_path"] else None
if not ref_text and row["ref_text"]:
ref_text = row["ref_text"]
elif ref_audio_path and not ref_text:
# Empty stored transcript → the auto-transcribe below will
# run; cache its result onto the profile so it runs ONCE,
# not on every generate (#1032 perf regression).
persist_ref_text_profile_id = profile_id
if not instruct and row["instruct"]:
instruct = row["instruct"]
if used_seed is None and row["seed"] is not None:
used_seed = row["seed"]
if language == "Auto":
language = None
# #533: a profile's stored language must drive generation when the
# request didn't pin one. Without this the German (etc.) archetype
# generates with language=None and the model drifts to English —
# even though the archetype PREVIEW renders correctly (archetypes.py
# passes the language). An EXPLICIT non-Auto request language still
# wins; we only fill the gap. `row` is a sqlite3.Row, so guard the
# column lookup for pre-language DBs mid-upgrade.
if language is None:
try:
prof_lang = row["language"]
except (KeyError, IndexError):
prof_lang = None
if prof_lang and prof_lang != "Auto":
language = prof_lang
elif ref_audio is not None:
try:
with tempfile.NamedTemporaryFile(delete=False, suffix=".wav") as f:
@@ -446,32 +850,93 @@ async def generate_speech(
# fallback behaves exactly as before.
if ref_audio_path and not ref_text:
from services.asr_backend import transcribe_reference
ref_text = await asyncio.get_running_loop().run_in_executor(
_gpu_pool, transcribe_reference, ref_audio_path
)
# Same #730 hang risk as any whisperx transcribe — bound + reset the pool
# so a wedged reference transcribe can't brick the backend. This path is
# best-effort (transcribe_reference returns None on failure → the model's
# built-in ASR fallback), so a timeout degrades to None rather than
# failing the whole generate.
try:
ref_text = await run_on_gpu_pool_guarded(
functools.partial(transcribe_reference, ref_audio_path),
what="Reference transcribe",
)
except GpuJobTimeoutError as e:
logger.warning("reference transcribe hung (%s); using model ASR fallback", e)
ref_text = None
# #1032: cache the transcript onto its clone profile so the ASR model
# load + transcribe above happens once per profile, not per generate.
# Only fills an empty column — a user-entered transcript always wins.
if ref_text and persist_ref_text_profile_id:
_persist_profile_ref_text(persist_ref_text_profile_id, ref_text)
# #526: materialize a concrete seed when none was supplied (and no profile
# pinned one) so the take is reproducible and we can hand it back via the
# X-Seed header for the "keep this seed" control. An explicit request seed
# or a profile's stored seed still wins — used_seed is only filled when it
# is still None here, never overwritten.
if used_seed is None:
used_seed = random.randint(0, 2**31 - 1)
# Expressive-TTS Spec 01: apply the user pronunciation dictionary + inline
# [[…]] one-off overrides to the text, here — AFTER `language` is fully
# resolved (a profile may fill it above) so per-language entries match the
# real render language, and BEFORE the text reaches either inference path
# (native OmniVoice or a pluggable backend) and the chunk splitter. This is
# the single point user text → normalized text → model, so the transform
# covers generate for every engine. Pure text substitution → identical on
# mac/Win/Linux. A disabled pref or empty dictionary is a pass-through, so
# plain text stays byte-identical (#G5 backward-compat).
from core import prefs as _prefs
_pron_env = os.environ.get("OMNIVOICE_PRONUNCIATION")
if _pron_env is not None:
# Env wins (power-user override); "0"/"false"/"no"/"off" disable it.
_pron_enabled = _pron_env.strip().lower() not in ("0", "false", "no", "off", "")
else:
_pron_enabled = bool(_prefs.get("pronunciation_enabled", True))
if pronounce and _pron_enabled:
from services.pronunciation import apply_pronunciation, load_entries_from_db
try:
_pron_rows = load_entries_from_db()
except Exception: # noqa: BLE001 — table missing / DB locked → no-op
_pron_rows = []
text = apply_pronunciation(text, _pron_rows, language)
else:
# Even with the dictionary off, inline [[…]] overrides are an explicit,
# in-text authoring choice → always honored (and never left as literal
# double-bracket text the model would mispronounce).
from services.pronunciation import apply_inline_overrides
text = apply_inline_overrides(text)
start_time = time.time()
try:
loop = asyncio.get_running_loop()
if _backend is not None:
audio_tensor = await loop.run_in_executor(
_gpu_pool, _run_backend_inference,
_backend, text, language, ref_audio_path, ref_text, instruct,
duration, num_step, guidance_scale, speed, denoise,
postprocess_output, used_seed, effect_preset,
max_chunk_chars, crossfade_ms,
# Bounded + pool-reset on hang so a wedged generate can't starve the
# GPU pool and brick the backend ("can't reach backend", #730 class).
audio_tensor = await run_on_gpu_pool_guarded(
functools.partial(
_run_backend_inference,
_backend, text, language, ref_audio_path, ref_text, instruct,
duration, num_step, guidance_scale, speed, denoise,
postprocess_output, used_seed, effect_preset,
max_chunk_chars, crossfade_ms,
),
what="TTS generate",
)
# Read after generation: engines with lazy model loading report
# their real rate only once weights are up.
sample_rate = _backend.sample_rate
else:
audio_tensor = await loop.run_in_executor(
_gpu_pool, _run_inference,
_model, text, language, ref_audio_path, ref_text, instruct, duration,
num_step, guidance_scale, speed, t_shift, denoise,
postprocess_output, layer_penalty_factor, position_temperature,
class_temperature, used_seed, effect_preset,
max_chunk_chars, crossfade_ms,
audio_tensor = await run_on_gpu_pool_guarded(
functools.partial(
_run_inference,
_model, text, language, ref_audio_path, ref_text, instruct, duration,
num_step, guidance_scale, speed, t_shift, denoise,
postprocess_output, layer_penalty_factor, position_temperature,
class_temperature, used_seed, effect_preset,
max_chunk_chars, crossfade_ms,
),
what="TTS generate",
)
sample_rate = _model.sampling_rate
# Invisible AudioSeal provenance watermark on the final audio. Embedding
@@ -493,13 +958,29 @@ async def generate_speech(
audio_dur = round(audio_tensor.shape[-1] / sample_rate, 2)
with db_conn() as conn:
conn.execute(
"INSERT INTO generation_history (id, text, mode, language, instruct, profile_id, audio_path, duration_seconds, generation_time, seed, created_at) VALUES (?,?,?,?,?,?,?,?,?,?,?)",
(audio_id, text[:200], history_mode or ("clone" if ref_audio_path else "design"),
language or "Auto", instruct or "", resolved_profile_id,
audio_filename, audio_dur, gen_time, used_seed, time.time())
)
# #710: the clip is already generated and saved above. A history-write
# failure — e.g. "no such table: generation_history" on a DB that missed
# schema init — must NOT 500 the user's generation. Self-heal the schema
# once and retry; if it still fails, log and return the audio anyway.
def _write_history():
with db_conn() as conn:
conn.execute(
"INSERT INTO generation_history (id, text, mode, language, instruct, profile_id, audio_path, duration_seconds, generation_time, seed, created_at) VALUES (?,?,?,?,?,?,?,?,?,?,?)",
(audio_id, text[:200], history_mode or ("clone" if ref_audio_path else "design"),
language or "Auto", instruct or "", resolved_profile_id,
audio_filename, audio_dur, gen_time, used_seed, time.time())
)
try:
_write_history()
except sqlite3.OperationalError as e:
logger.warning("generation history write failed (%s); healing schema + retrying", e)
try:
ensure_schema()
_write_history()
except Exception as e2:
logger.warning("history write still failed after schema heal; returning audio anyway: %s", e2)
except Exception as e:
logger.warning("generation history write failed; returning audio anyway: %s", e)
event_bus.emit("generation_history", {"action": "created", "id": audio_id})
buffer = io.BytesIO()
@@ -535,6 +1016,12 @@ async def generate_speech(
)
except HTTPException:
raise
except GpuJobTimeoutError as e:
# A wedged GPU generate — the pool was already reset to restore capacity
# (#730 class). Report the actionable timeout instead of the misleading
# "can't reach backend" the frontend shows when the pool starves.
logger.error("Generate timed out: %s", e)
raise HTTPException(status_code=503, detail=str(e)) from e
except ValueError as e:
logger.error("Validation failed: %s", e)
raise HTTPException(status_code=400, detail=str(e)) from e
@@ -545,7 +1032,7 @@ async def generate_speech(
status_code=500,
detail=(
f"Couldn't synthesize audio. See Settings → Logs → Backend for the full trace. "
f"Underlying error: {e}"
f"Underlying error: {_safe_exc_text(e)}"
),
)
finally:
+24 -9
View File
@@ -189,15 +189,21 @@ def auto_extract(project_id: str, req: AutoExtractRequest):
Writes them as `auto=1` rows. Existing terms with the same (source,target)
are NOT duplicated. Returns the full current glossary after the pass.
"""
from services.translator import _llm_client, _llm_model, _llm_timeout # reuse same client
# Resolved through the LLM Skills registry so auto-extract can be toggled
# or routed to its own provider (Settings → LLM Skills) independently of
# the translation pipeline. None == disabled or no provider configured.
from services import llm_skills
client = _llm_client()
if client is None:
handle = llm_skills.resolve_skill_client("glossary_extract")
if handle is None:
raise HTTPException(
status_code=503,
detail=(
"Auto-extract needs an LLM. Set TRANSLATE_BASE_URL + TRANSLATE_API_KEY "
"(Ollama works locally: base_url=http://localhost:11434/v1) and try again."
"Auto-extract needs an LLM. Set one up in Settings → LLM Providers "
"(pick a provider, add its key, choose a model, Test) — or use local "
"Ollama / LM Studio for a fully offline setup — and make sure the "
"Glossary auto-extract skill is enabled in Settings → LLM Skills, "
"then try again."
),
)
@@ -220,9 +226,9 @@ def auto_extract(project_id: str, req: AutoExtractRequest):
)
try:
res = client.chat.completions.create(
model=_llm_model(),
timeout=_llm_timeout(),
res = handle.client.chat.completions.create(
model=handle.model,
timeout=handle.timeout,
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
@@ -231,9 +237,18 @@ def auto_extract(project_id: str, req: AutoExtractRequest):
body = (res.choices[0].message.content or "").strip()
except Exception as e:
logger.warning("auto-extract LLM call failed: %s", e)
# Scrub the provider error — some OpenAI-compatible providers echo the
# API key or a user_id in the body, which must not reach the UI verbatim.
from core.scrub import scrub_provider_error
from services import llm_providers
_p = llm_providers.active_provider()
_key = llm_providers.resolve_api_key(_p) if _p else None
raise HTTPException(
status_code=502,
detail=f"LLM didn't respond. Check Settings → Logs → Backend for the trace. Error: {e}",
detail=(
"LLM didn't respond. Check Settings → Logs → Backend for the trace. "
f"Error: {scrub_provider_error(e, _key)}"
),
)
# Parse: SOURCE || TARGET || note (lines are allowed to be sloppy — we're forgiving).
+75 -11
View File
@@ -22,7 +22,6 @@ from __future__ import annotations
import io
import logging
import os
import asyncio
import tempfile
from typing import Literal, Optional
@@ -30,7 +29,7 @@ from fastapi import APIRouter, File, Form, HTTPException, UploadFile
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from services.model_manager import _gpu_pool
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
logger = logging.getLogger("omnivoice.openai_compat")
@@ -109,6 +108,23 @@ class SpeechRequest(BaseModel):
ge=0,
description="OmniVoice GGUF extension: long-form internal chunk threshold.",
)
# #1014: these two were silently DISCARDED before (pydantic ignores
# undeclared fields) — a 200 OK that quietly dropped the caller's quality
# knobs. Declared now and passed through, matching the native /generate
# form fields (defaults there: num_step=16, guidance_scale=2.0; the
# model's documented "quality" preset is num_step=32).
num_step: Optional[int] = Field(
default=None,
ge=1,
le=128,
description="OmniVoice extension: iterative unmasking steps (app default 16; 32 = the model's documented quality preset).",
)
guidance_scale: Optional[float] = Field(
default=None,
gt=0,
le=20,
description="OmniVoice extension: classifier-free guidance scale (app default 2.0).",
)
class TranscriptionResponse(BaseModel):
@@ -238,10 +254,10 @@ def _run_tts(backend, text: str, kw: dict):
sr = backend.sample_rate
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
# That chain's Compressor + 8% Reverb is tuned for OmniVoice's 24 kHz clone
# output; applied to a studio engine it adds an audible level pump and a
# reverb tail that degrade the very output we want clean. Loudness
# normalisation still runs — it's a benign peak scale, not dynamics.
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
# output; applied to a studio engine it adds an audible level pump that
# degrades the very output we want clean. Loudness normalisation still
# runs — it's a benign peak scale, not dynamics.
if not getattr(backend, "applies_own_mastering", False):
wav = apply_mastering(wav, sample_rate=sr)
wav = normalize_audio(wav, target_dBFS=-2.0)
@@ -275,6 +291,10 @@ async def create_speech(req: SpeechRequest):
kw["chunk_duration"] = req.chunk_duration
if req.chunk_threshold is not None:
kw["chunk_threshold"] = req.chunk_threshold
if req.num_step is not None:
kw["num_step"] = req.num_step
if req.guidance_scale is not None:
kw["guidance_scale"] = req.guidance_scale
if req.language:
kw["language"] = req.language
if req.instruct:
@@ -312,9 +332,44 @@ async def create_speech(req: SpeechRequest):
# Not a profile ID — might be a KittenTTS preset or similar
kw["voice"] = voice
# ── #1033/#1037/#1014: warm the engine under the LOAD budget before the
# generate clock starts. The T4 verification (#1014) measured a fresh
# install's first /v1/audio/speech burning its whole 300s generate budget
# on the multi-GB checkpoint download (0% GPU util throughout) and dying
# with a misleading "too heavy for the available compute" error. Model
# loading gets OMNIVOICE_MODEL_LOAD_TIMEOUT (default 1200s); once warm
# this is a per-request no-op.
from services.model_manager import _model_load_timeout
try:
loop = asyncio.get_running_loop()
wav, sr = await loop.run_in_executor(_gpu_pool, _run_tts, backend, req.input, kw)
await run_on_gpu_pool_guarded(
backend.ensure_ready,
what=f"TTS engine '{backend.id}' model load",
timeout=_model_load_timeout(),
)
# Catch the BUILTIN TimeoutError base, not GpuJobTimeoutError by name:
# several tests reload services.model_manager mid-suite, so a class
# imported at call time can differ in identity from the one the guard
# (bound at this module's import) actually raises — the except would
# silently miss. The builtin base has one identity forever. (Caught by
# this exact test failing CI-only, in full-suite order.)
except TimeoutError as e:
logger.warning("engine load exceeded the model-load budget: %s", e)
raise HTTPException(
status_code=503,
detail=(
f"TTS engine '{backend.id}' did not finish loading within its "
f"model-load budget — on a first run this usually means the weight "
f"download is slow or stalled (check Settings → Models for "
f"progress), not that generation failed. Retry once the model "
f"shows as installed."
),
) from e
try:
# Bounded + pool-reset on hang so a wedged TTS request can't starve the
# GPU pool and brick the backend (#730 class).
wav, sr = await run_on_gpu_pool_guarded(
lambda: _run_tts(backend, req.input, kw), what="OpenAI TTS generate")
except Exception as e:
logger.exception("OpenAI TTS failed: %s", e)
raise HTTPException(status_code=500, detail=str(e))
@@ -384,12 +439,15 @@ async def create_transcription(
try:
backend = get_active_asr_backend()
# Run transcription in the thread pool to avoid blocking the event loop
loop = asyncio.get_running_loop()
# Run transcription in the thread pool to avoid blocking the event loop,
# bounded so a stuck/starved ASR returns a 504 with guidance instead of
# hanging the request forever (see run_transcribe_guarded).
from services.asr_backend import run_transcribe_guarded
word_ts = response_format == "verbose_json"
result = await loop.run_in_executor(
result = await run_transcribe_guarded(
_gpu_pool,
lambda: backend.transcribe(tmp_path, word_timestamps=word_ts),
what="OpenAI",
)
# Extract the full text from segments
@@ -456,6 +514,12 @@ async def create_transcription(
# Default: json
return TranscriptionResponse(text=full_text)
except HTTPException:
raise
except TimeoutError as e:
# ASRTimeoutError (subclass): backend alive, ASR too heavy for compute.
logger.warning("OpenAI transcription timed out: %s", e)
raise HTTPException(status_code=504, detail=str(e))
except Exception as e:
logger.exception("OpenAI transcription failed: %s", e)
raise HTTPException(status_code=500, detail=str(e))
+6 -1
View File
@@ -60,6 +60,11 @@ async def export_persona(
profile = dict(row)
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
# #693: if OMNIVOICE_MODEL is set, record the *resolved* checkpoint in the
# exported bundle so a leaked engine id (e.g. "omnivoice") can't be baked in;
# keep "" when unset (the bundle's "engine unspecified" marker).
from services.model_manager import resolve_omnivoice_checkpoint
engine_id = resolve_omnivoice_checkpoint() if os.environ.get("OMNIVOICE_MODEL", "").strip() else ""
try:
loop = asyncio.get_running_loop()
content = await loop.run_in_executor(
@@ -70,7 +75,7 @@ async def export_persona(
license_spdx=license_spdx,
tags=tag_list,
include_reference=include_reference,
engine_id=os.environ.get("OMNIVOICE_MODEL", ""),
engine_id=engine_id,
omnivoice_version=APP_VERSION,
),
)
+23
View File
@@ -12,6 +12,7 @@ from core.db import db_conn
from core.config import VOICES_DIR, OUTPUTS_DIR
from core import event_bus
from core.personalities import get_personalities
from omnivoice.utils.voice_design import heal_design_instruct, sanitize_instruct
router = APIRouter()
@@ -72,10 +73,28 @@ async def create_profile(
raise ValueError("not an object")
except ValueError:
raise HTTPException(status_code=422, detail="vd_states must be a JSON object")
# Root-cause close for #983: a design profile must never be PERSISTED
# with a partial vd_states shape, regardless of which client (older
# frontend build, hand-edited payload, third-party API caller) created
# it — a missing category key crashes DesignMethodPanel's render on
# every future client that selects this profile. CATEGORY_ORDER is the
# same single source of truth the frontend's CATEGORIES keys mirror
# (core/describe_voice.py), so this can't drift from the picker.
from core.describe_voice import CATEGORY_ORDER
for _cat in CATEGORY_ORDER:
parsed.setdefault(_cat, "Auto")
vd_states = _json.dumps(parsed)
# An all-Auto design (every category left on "Auto") yields an empty
# instruct — that's still a valid, saveable voice: synthesis falls back
# to neutral instruct-only conditioning (see generation.py design path).
# Don't gate save on a non-empty instruct.
#
# Defence-in-depth against the "[object Object]" / freeform-prose poison
# (#550 #571 #594 #596): never persist an instruct the engine validator
# would reject. Sanitize the submitted instruct and, if it's unusable,
# rebuild the tags from vd_states — so the row is always generation-safe
# regardless of which frontend build saved it.
instruct = heal_design_instruct(instruct, parsed)
profile_id = str(uuid.uuid4())[:8]
@@ -167,6 +186,10 @@ def update_profile(profile_id: str, patch: ProfileUpdate):
continue
if col == "name" and not val.strip():
raise HTTPException(status_code=400, detail="A voice profile needs a name.")
if col == "instruct":
# Never let an edit persist a validator-rejecting instruct (prose /
# "[object Object]"); keep only whitelist tags (#550 #571 #594 #596).
val = sanitize_instruct(val)
fields.append(f"{col} = ?")
params.append(val.strip() if col in ("name", "language") else val)
if not fields:
+306
View File
@@ -0,0 +1,306 @@
"""
Pronunciation dictionary router Expressive-TTS Spec 01 Phase 1.
CRUD for the DB-backed, per-language pronunciation dictionary the
``PronunciationPanel`` (Settings Pronunciation) edits, plus a model-free
``/pronunciation/test`` dry-run. Entries are applied as pure text substitution
before synthesis (see ``services/pronunciation.apply_pronunciation`` and the
generate path), so a saved entry actually changes the audio on every engine.
Endpoints (loopback-only, like the dictation router):
GET /pronunciation list every entry
POST /pronunciation create one entry
PUT /pronunciation/{entry_id} update an entry (partial)
DELETE /pronunciation/{entry_id} remove an entry
POST /pronunciation/test dry-run substitution (no model)
GET /pronunciation/export all entries as JSON (round-trips import)
POST /pronunciation/import bulk add entries from JSON
Scope: ``language='*'`` is global (applies to every request); a 2-letter code
(``'en'``, ``'de'``) applies only when the request language matches.
"""
from __future__ import annotations
import logging
import re
import time
import uuid
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from api.dependencies import require_loopback
from core.db import db_conn
from services.pronunciation import apply_pronunciation, entries_for_language
logger = logging.getLogger("omnivoice.pronunciation")
router = APIRouter()
_VALID_TYPES = ("respelling", "ipa", "cmu")
_ALL_LANG = "*"
# IPA: the input is validated as a non-empty string of Unicode letters / IPA
# extension codepoints + the usual suprasegmental marks; we reject ASCII control
# and the bracket/pipe chars that would collide with the inline grammar. This is
# a charset gate (catches obvious garbage early), not a full IPA grammar.
_IPA_BAD = re.compile(r"[\[\]\|\x00-\x1f]")
# CMU / ARPABET: space-separated phoneme tokens (letters + an optional 0-2 stress
# digit), e.g. "N AH0 V AE1 D AH0". Reject anything else.
_CMU_TOKEN = re.compile(r"^[A-Za-z]{1,3}[0-2]?$")
def _validate_type_replacement(etype: str, replacement: str) -> None:
"""Raise 400 on a phoneme replacement that's obviously malformed.
Respelling rows accept any text. IPA rows must be a non-empty string free of
bracket/pipe/control chars. CMU rows must be space-separated ARPABET tokens.
Validating on save (not at synth) means a model never sees garbage phonemes
(Spec 01 §R3 never pass unvalidated phoneme strings to a model).
"""
if etype == "respelling":
return
rep = (replacement or "").strip()
if not rep:
raise HTTPException(
status_code=400,
detail=f"A {etype.upper()} entry needs a phoneme string in 'replacement'.",
)
if etype == "ipa":
if _IPA_BAD.search(rep):
raise HTTPException(
status_code=400,
detail="That IPA string contains brackets, a pipe, or control characters. "
"Use plain IPA symbols, e.g. ˈnɛvʌdə.",
)
elif etype == "cmu":
tokens = rep.split()
if not tokens or any(not _CMU_TOKEN.match(tok) for tok in tokens):
raise HTTPException(
status_code=400,
detail="That doesn't look like CMU/ARPABET. Use space-separated tokens with "
"optional stress digits, e.g. N AH0 V AE1 D AH0.",
)
def _norm_language(language: Optional[str]) -> str:
"""Normalize a scope to '*' (global) or a lowercase 2-letter code."""
if not language:
return _ALL_LANG
s = str(language).strip()
if not s or s == _ALL_LANG or s.lower() == "auto":
return _ALL_LANG
return s.lower()[:2]
def _row_to_dict(r) -> dict:
d = dict(r)
d["enabled"] = bool(d.get("enabled"))
# ``scope`` is the UI-facing alias for ``language`` ('*' shows as Global).
d["scope"] = d.get("language") or _ALL_LANG
return d
# ── Schemas ──────────────────────────────────────────────────────────────────
class PronEntry(BaseModel):
term: str
replacement: str = ""
type: str = "respelling"
language: str = _ALL_LANG
enabled: bool = True
class PronEntryUpdate(BaseModel):
term: Optional[str] = None
replacement: Optional[str] = None
type: Optional[str] = None
language: Optional[str] = None
enabled: Optional[bool] = None
class PronTestRequest(BaseModel):
text: str
language: Optional[str] = None
class PronImportRequest(BaseModel):
entries: List[PronEntry]
replace: bool = False # True → clear existing rows first
# ── CRUD ─────────────────────────────────────────────────────────────────────
@router.get("/pronunciation", dependencies=[Depends(require_loopback)])
def list_entries():
with db_conn() as conn:
rows = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries ORDER BY created_at ASC, id ASC"
).fetchall()
return [_row_to_dict(r) for r in rows]
@router.post("/pronunciation", dependencies=[Depends(require_loopback)])
def create_entry(entry: PronEntry):
term = entry.term.strip()
if not term:
raise HTTPException(status_code=400, detail="A pronunciation entry needs a term.")
etype = (entry.type or "respelling").strip().lower()
if etype not in _VALID_TYPES:
raise HTTPException(
status_code=400,
detail=f"Unknown entry type {entry.type!r}. Use one of: {', '.join(_VALID_TYPES)}.",
)
_validate_type_replacement(etype, entry.replacement)
eid = str(uuid.uuid4())[:12]
now = time.time()
lang = _norm_language(entry.language)
with db_conn() as conn:
conn.execute(
"INSERT INTO pronunciation_entries (id, term, replacement, type, language, enabled, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?)",
(eid, term, entry.replacement, etype, lang, 1 if entry.enabled else 0, now),
)
row = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries WHERE id = ?", (eid,)
).fetchone()
return _row_to_dict(row)
@router.put("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
def update_entry(entry_id: str, patch: PronEntryUpdate):
with db_conn() as conn:
existing = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries WHERE id = ?", (entry_id,)
).fetchone()
if existing is None:
raise HTTPException(status_code=404, detail="No such pronunciation entry.")
# Resolve the post-update type + replacement so phoneme validation runs
# against the final state (e.g. switching type without changing text).
new_type = (patch.type.strip().lower() if patch.type is not None else existing["type"]) or "respelling"
if new_type not in _VALID_TYPES:
raise HTTPException(
status_code=400,
detail=f"Unknown entry type {patch.type!r}. Use one of: {', '.join(_VALID_TYPES)}.",
)
new_replacement = patch.replacement if patch.replacement is not None else existing["replacement"]
_validate_type_replacement(new_type, new_replacement)
fields, params = [], []
if patch.term is not None:
term = patch.term.strip()
if not term:
raise HTTPException(status_code=400, detail="A pronunciation entry needs a term.")
fields.append("term = ?"); params.append(term)
if patch.replacement is not None:
fields.append("replacement = ?"); params.append(patch.replacement)
if patch.type is not None:
fields.append("type = ?"); params.append(new_type)
if patch.language is not None:
fields.append("language = ?"); params.append(_norm_language(patch.language))
if patch.enabled is not None:
fields.append("enabled = ?"); params.append(1 if patch.enabled else 0)
if not fields:
raise HTTPException(
status_code=400,
detail="PUT body was empty. Include at least one field to change, or DELETE the entry.",
)
params.append(entry_id)
# nosec B608 - `fields` are fixed literal assignments ("term = ?", …) from
# the allowlist above; every user value is a bound `?` parameter, never
# interpolated. The f-string only joins constant column fragments.
conn.execute(
f"UPDATE pronunciation_entries SET {', '.join(fields)} WHERE id = ?", # nosec B608
params,
)
row = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries WHERE id = ?", (entry_id,)
).fetchone()
return _row_to_dict(row)
@router.delete("/pronunciation/{entry_id}", dependencies=[Depends(require_loopback)])
def delete_entry(entry_id: str):
with db_conn() as conn:
cur = conn.execute("DELETE FROM pronunciation_entries WHERE id = ?", (entry_id,))
return {"deleted": cur.rowcount > 0}
# ── Dry-run + import/export ───────────────────────────────────────────────────
@router.post("/pronunciation/test", dependencies=[Depends(require_loopback)])
def test_substitution(req: PronTestRequest):
"""Show the post-substitution text for ``req.text`` — no model call.
Applies the same dictionary + inline ``[[]]`` resolution the synth path
runs, so the user sees exactly what the engine will be handed.
"""
with db_conn() as conn:
rows = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries"
).fetchall()
substituted = apply_pronunciation(req.text, rows, req.language)
applied = entries_for_language(rows, req.language)
return {
"input": req.text,
"substituted": substituted,
"changed": substituted != req.text,
"applied_terms": sorted(applied.keys(), key=len, reverse=True),
}
@router.get("/pronunciation/export", dependencies=[Depends(require_loopback)])
def export_entries():
"""Every entry as a JSON-serializable list (round-trips ``/import``)."""
with db_conn() as conn:
rows = conn.execute(
"SELECT term, replacement, type, language, enabled "
"FROM pronunciation_entries ORDER BY created_at ASC, id ASC"
).fetchall()
return {"entries": [
{"term": r["term"], "replacement": r["replacement"], "type": r["type"],
"language": r["language"], "enabled": bool(r["enabled"])}
for r in rows
]}
@router.post("/pronunciation/import", dependencies=[Depends(require_loopback)])
def import_entries(req: PronImportRequest):
"""Bulk-add entries. ``replace=true`` clears the table first.
Each entry is validated like ``POST /pronunciation``; one bad row fails the
whole import (400) so the table is never left half-applied.
"""
now = time.time()
cleaned = []
for e in req.entries:
term = e.term.strip()
if not term:
continue # silently skip blank terms — they're a no-op anyway
etype = (e.type or "respelling").strip().lower()
if etype not in _VALID_TYPES:
raise HTTPException(
status_code=400,
detail=f"Entry {term!r}: unknown type {e.type!r}.",
)
_validate_type_replacement(etype, e.replacement)
cleaned.append((str(uuid.uuid4())[:12], term, e.replacement, etype,
_norm_language(e.language), 1 if e.enabled else 0, now))
with db_conn() as conn:
if req.replace:
conn.execute("DELETE FROM pronunciation_entries")
conn.executemany(
"INSERT INTO pronunciation_entries (id, term, replacement, type, language, enabled, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?)",
cleaned,
)
return {"imported": len(cleaned), "replaced": req.replace}
+392 -7
View File
@@ -12,6 +12,7 @@ The state endpoint duplicates `/system/hf-token/state` (which lives on
"""
from __future__ import annotations
import asyncio
import logging
import os
from dataclasses import asdict
@@ -134,12 +135,22 @@ class _RefinementBody(BaseModel):
def _refinement_state():
from services.refinement import get_refinement_config
from services.llm_backend import get_active_llm_backend
from services.refinement import (
_skill_llm,
get_last_refine_status,
get_refinement_config,
)
cfg = get_refinement_config()
# The UI shows whether refinement can actually run (needs an LLM).
cfg["llm_ready"] = get_active_llm_backend().id != "off"
# `llm_ready` only means "an endpoint is CONFIGURED" — a placeholder/dead
# endpoint still reads ready. It's resolved through the LLM Skills registry
# so a disabled dictation_refinement skill / per-skill provider override
# reads the same here as on the actual refine path. The honesty layer is
# `last_refine_status`: {ok, reason, at} from the most recent final, so the
# panel can flag a configured-but-failing LLM (the real safety is the hard
# refine timeout, which keeps a dead endpoint from ever stalling the final).
cfg["llm_ready"] = _skill_llm().id != "off"
cfg["last_refine_status"] = get_last_refine_status()
return cfg
@@ -234,6 +245,248 @@ def set_llm_endpoint(body: _LLMEndpointBody):
return _llm_endpoint_state()
# ── Multi-provider LLM registry (Settings → LLM Providers) ────────────────
# Keys persist ENCRYPTED via settings_store.set_secret (never .env, never
# returned). base_url/model/account overrides are non-secret. Loopback-gated
# by the router dep, so LAN peers can't read masks or write keys.
class _LLMProviderBody(BaseModel):
api_key: str | None = Field(None, description="API key; '' clears it, None leaves unchanged")
base_url: str | None = None
model: str | None = None
account_id: str | None = Field(None, description="Cloudflare account id")
make_active: bool = False
class _LLMActiveBody(BaseModel):
provider: str = Field(..., description="provider id to activate")
@router.get("/llm-providers")
def list_llm_providers():
"""All providers with resolved base_url/model + whether a key is configured.
Never returns key material only `has_key`/`key_from_env` booleans.
"""
from services import llm_providers
return {
"active": llm_providers.active_provider_id(),
"providers": [llm_providers.describe(p) for p in llm_providers.all_providers()],
}
@router.put("/llm-providers/{provider_id}")
def save_llm_provider(provider_id: str, body: _LLMProviderBody):
"""Save a provider's key (encrypted) + optional base_url/model/account.
A None field is left unchanged; an empty api_key clears the stored key.
"""
from services import llm_providers
p = llm_providers.get_provider(provider_id)
if p is None:
raise HTTPException(status_code=404, detail=f"unknown provider {provider_id!r}")
if body.api_key is not None:
llm_providers.save_key(provider_id, body.api_key.strip())
llm_providers.save_overrides(
provider_id, base_url=body.base_url, model=body.model,
account_id=body.account_id,
)
# An explicit save also claims the active slot when the user has never
# chosen a provider (#963). Without this, a saved-and-tested local
# provider (Ollama/LM Studio) evaporates on restart: active_provider_id()
# deliberately excludes local providers from auto-select, so the plain
# "Save" left nothing persisted to resolve. Gated on the STORED selection
# only — an explicit prior choice is never stolen by a plain save, and an
# unconfigured provider can't claim the slot.
if body.make_active or (
llm_providers.stored_active_provider_id() is None
and llm_providers.is_configured(p)
):
llm_providers.set_active_provider(provider_id)
return list_llm_providers()
@router.post("/llm-providers/active")
def set_active_llm_provider(body: _LLMActiveBody):
from services import llm_providers
if llm_providers.get_provider(body.provider) is None:
raise HTTPException(status_code=404, detail=f"unknown provider {body.provider!r}")
llm_providers.set_active_provider(body.provider)
return list_llm_providers()
def _scrub_llm_detail(e: Exception, api_key: str | None) -> str:
"""Scrubbed, UI-safe failure text. scrub_text() covers env secrets and
home paths but a STORE-persisted key isn't in the env, and some
providers echo the key in error bodies, so redact the exact resolved key
explicitly before the generic pass."""
from core.scrub import scrub_text
detail = f"{type(e).__name__}: {e}"
if api_key and api_key != "local" and len(api_key) >= 8:
detail = detail.replace(api_key, "•••")
return scrub_text(detail)
def _classify_llm_error(e: Exception) -> str:
"""Map a provider-call failure to an actionable kind the UI can localize.
Kinds: auth (bad/missing key), not_found (model or endpoint path),
rate_limit, network (DNS/conn/timeout), error (everything else).
Status codes win when the OpenAI SDK provides one; exception-family
names catch the non-HTTP failures (DNS, refused, TLS, timeout).
"""
status = getattr(e, "status_code", None)
if status in (401, 403):
return "auth"
if status == 404:
return "not_found"
if status == 429:
return "rate_limit"
name = type(e).__name__
if name in ("APIConnectionError", "APITimeoutError", "ConnectError",
"ConnectTimeout", "TimeoutError"):
return "network"
if name == "AuthenticationError":
return "auth"
if name == "NotFoundError":
return "not_found"
if name == "RateLimitError":
return "rate_limit"
return "error"
@router.post("/llm-providers/{provider_id}/test")
def test_llm_provider(provider_id: str):
"""One cheap round-trip against a provider to prove the key/URL work.
Temporarily activates the provider for the probe by resolving its config
directly (does not change the persisted active selection). Returns
latency_ms plus, on failure, a classified ``kind`` (config / auth /
not_found / rate_limit / network / error) so the UI shows an actionable,
localizable message instead of a raw exception string.
"""
import time as _time
from services import llm_providers
p = llm_providers.get_provider(provider_id)
if p is None:
raise HTTPException(status_code=404, detail=f"unknown provider {provider_id!r}")
base_url = llm_providers.resolve_base_url(p)
api_key = llm_providers.resolve_api_key(p)
if not base_url:
return {"ok": False, "kind": "config", "detail": "No Base URL set for this provider."}
if not api_key:
return {"ok": False, "kind": "config", "detail": "No API key configured for this provider."}
t0 = _time.monotonic()
try:
from openai import OpenAI
# max_retries=0: this is an interactive probe with a live spinner — the
# SDK's default 2 automatic retries turn a 429/timeout into a ~34s hang.
# Surface the first failure immediately instead.
client = OpenAI(api_key=api_key, base_url=base_url, max_retries=0)
res = client.chat.completions.create(
model=llm_providers.resolve_model(p),
messages=[{"role": "user", "content": "Reply with the single word: ok"}],
timeout=20,
)
reply = (res.choices[0].message.content or "").strip()
return {
"ok": True,
"model": llm_providers.resolve_model(p),
"reply": reply[:80],
"latency_ms": int((_time.monotonic() - t0) * 1000),
}
except Exception as e: # noqa: BLE001 — surface a clean, scrubbed error to the UI
return {
"ok": False,
"kind": _classify_llm_error(e),
"detail": _scrub_llm_detail(e, api_key),
"latency_ms": int((_time.monotonic() - t0) * 1000),
}
@router.get("/llm-providers/{provider_id}/models")
def list_llm_provider_models(provider_id: str):
"""List model ids the provider's key can access (OpenAI-compat /models).
Powers the model-picker datalist in Settings LLM Providers so users
don't have to guess model names. Read-only; failures return the same
classified shape as /test; capped so a huge catalog can't bloat the UI.
"""
from services import llm_providers
p = llm_providers.get_provider(provider_id)
if p is None:
raise HTTPException(status_code=404, detail=f"unknown provider {provider_id!r}")
base_url = llm_providers.resolve_base_url(p)
api_key = llm_providers.resolve_api_key(p)
if not base_url or not api_key:
return {"ok": False, "kind": "config", "models": []}
try:
from openai import OpenAI
# max_retries=0: interactive probe — fail fast, don't burn ~34s on the
# SDK's default retry ladder when the key/URL is wrong (matches /test).
client = OpenAI(api_key=api_key, base_url=base_url, max_retries=0)
ids = sorted(m.id for m in client.models.list(timeout=10))
# Cap so a huge catalog can't bloat the datalist; flag the cap so the UI
# can say "first 200 shown" rather than implying it's the full list.
return {"ok": True, "models": ids[:200], "truncated": len(ids) > 200}
except Exception as e: # noqa: BLE001
return {
"ok": False,
"kind": _classify_llm_error(e),
"detail": _scrub_llm_detail(e, api_key),
"models": [],
}
# ── LLM Skills (Settings → LLM Skills) ─────────────────────────────────────
# Per-feature enable/route control for every LLM consumption point. Each
# skill can be toggled off (degrades exactly like "no LLM configured") or
# routed to a specific provider (local Ollama/LM Studio vs a remote key)
# instead of the one global active provider. Loopback-gated (router dep).
class _LLMSkillBody(BaseModel):
enabled: bool | None = Field(None, description="None leaves the toggle unchanged")
provider_override: str | None = Field(
None,
description="provider id to route this skill to; '' or null clears "
"it (skill follows the active provider). Omit to leave "
"unchanged.",
)
@router.get("/llm-skills")
def list_llm_skills():
"""Every LLM skill with its toggle, routing, and resolved ready status."""
from services import llm_skills
return {"skills": [llm_skills.describe(s.id) for s in llm_skills.all_skills()]}
@router.put("/llm-skills/{skill_id}")
def set_llm_skill(skill_id: str, body: _LLMSkillBody):
"""Toggle a skill and/or set its provider routing.
Field semantics match the providers PUT: an omitted field is left
unchanged; ``provider_override: ""``/``null`` clears the override.
404 for an unknown skill or an unknown provider id.
"""
from services import llm_skills
if llm_skills.get_skill(skill_id) is None:
raise HTTPException(status_code=404, detail=f"unknown LLM skill {skill_id!r}")
kwargs = {}
if body.enabled is not None:
kwargs["enabled"] = body.enabled
if "provider_override" in body.model_fields_set:
kwargs["provider_override"] = body.provider_override
try:
if kwargs:
llm_skills.configure_skill(skill_id, **kwargs)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
return list_llm_skills()
# ── License acceptance (Phase 3 Plan 03-01 / TTS-05) ──────────────────────
# Frontend ``SupertonicLicenseDialog`` flips the engine-license bit via this
# endpoint. The handler is loopback-gated (router-level dep) and the
@@ -397,6 +650,41 @@ def set_models_dir(body: _ModelsDirBody):
return {"configured": path, "effective": _effective_models_dir(), "restart_required": True}
# ── Storage report (Settings → Storage) ────────────────────────────────────
# Per-volume disk totals + du-style sizes for everything the app owns (HF
# model cache, app data subtotals, engine venvs, temp files) with server-side
# warnings. Heavy directory walks run in a worker thread with per-category
# deadlines and a 5-minute in-process cache (services.storage_report), so the
# endpoint stays cheap on repeat Settings visits. Loopback-gated via the
# router-level dep like every sibling.
@router.get("/storage")
async def get_storage_report(refresh: bool = Query(False)):
"""Disk + per-category storage usage for the Settings → Storage panel.
`refresh=1` bypasses the 5-minute cache and rescans. `min_free_gb`
reuses the setup wizard's constant so both surfaces warn at the same
threshold.
"""
from api.routers.setup.wizard import MIN_FREE_GB
from core.config import DATA_DIR
from services import storage_report
try:
return await asyncio.to_thread(
storage_report.get_report,
data_dir=DATA_DIR,
hf_cache_dir=_effective_models_dir(),
app_venv=storage_report.default_app_venv(),
min_free_gb=MIN_FREE_GB,
refresh=refresh,
)
except Exception:
logger.exception("storage report failed")
raise HTTPException(status_code=500, detail="Failed to compute storage report")
# ── HF mirror endpoint (parity program Wave 4.3 / §R4 c) ──────────────────
# Restricted-network users (e.g. behind the Great Firewall) need to point
# huggingface_hub at a mirror. HF reads HF_ENDPOINT at import time, so a
@@ -439,6 +727,11 @@ def set_hf_mirror(body: _HFMirrorBody):
url = (body.url or "").strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Mirror URL must start with http(s)://")
# Compare against the currently-persisted value (normalised the same way) so
# a no-op save doesn't nag the user to restart. Only a real change to the
# persisted endpoint can require a restart.
previous = (user_env.get_user_env(_HF_ENDPOINT_ENV) or "").strip().rstrip("/")
changed = url != previous
try:
if url:
user_env.set_user_env(_HF_ENDPOINT_ENV, url)
@@ -449,6 +742,98 @@ def set_hf_mirror(body: _HFMirrorBody):
except Exception:
logger.exception("set_hf_mirror failed")
raise HTTPException(status_code=500, detail="Failed to persist mirror setting")
# HF endpoint is read at import time by huggingface_hub, so the override
# is only guaranteed once the backend restarts.
return {"configured": url, "restart_required": True, "presets": _HF_MIRROR_PRESETS}
# Model Store downloads pick up the new mirror immediately — the download
# path resolves the endpoint per-call and we updated os.environ above. Only
# transformers-side model *loads* (which read HF_ENDPOINT at import time)
# need a restart, so restart_required is True ONLY when the value actually
# changed — a no-op re-save never asks for a restart.
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
# without waiting on transformers to ship a direct Qwen3-ASR integration.
# base_url/model are plain settings_store text rows; the key is encrypted via
# settings_store.set_secret — same convention as /llm-providers, never
# returned to the client, '' clears it, omitted/None leaves it unchanged.
class _ASROpenAICompatBody(BaseModel):
base_url: str | None = None
model: str | None = None
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
@router.get("/asr-openai-compat")
def get_asr_openai_compat():
from services import asr_backend
return {
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
"model": asr_backend.resolve_openai_compat_asr_model(),
"has_key": asr_backend.openai_compat_asr_has_key(),
}
@router.put("/asr-openai-compat")
def set_asr_openai_compat(body: _ASROpenAICompatBody):
from services import asr_backend, settings_store
if body.base_url is not None:
url = body.base_url.strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
if body.model is not None:
settings_store.set_text(
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
)
if body.api_key is not None:
settings_store.set_secret(
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
)
return get_asr_openai_compat()
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
# (feat/safe-updates). Both are read-only, local-first surfaces for
# Settings → Updates: the "What's new" viewer reads the CHANGELOG.md that
# ships with the app, and the backup line shows the newest pre-migration
# snapshot written by core.db_backup before `alembic upgrade head` runs.
@router.get("/changelog")
def get_changelog(limit_versions: int = Query(5, ge=1, le=50)):
"""Structured release notes from the shipped CHANGELOG.md (newest first).
Bullets are raw markdown-lite (bold leads, `code`, (#NNN) refs) — the
frontend renders them safely without HTML. `available: false` when this
install has no changelog (never an error: the viewer just hides)."""
from core import changelog
path = changelog.changelog_path()
if not path:
return {"available": False, "releases": []}
try:
with open(path, encoding="utf-8") as fh:
releases = changelog.parse_changelog(fh.read(), limit_versions)
except Exception:
logger.exception("changelog parse failed")
return {"available": False, "releases": []}
return {"available": bool(releases), "releases": releases}
@router.get("/db-backup")
def get_db_backup_state():
"""Newest pre-migration database backup (or none yet). Feeds the
"your data is backed up before every update" line in Settings Updates."""
from core import db_backup
from core.config import DB_PATH
latest = db_backup.latest_backup(DB_PATH)
return {
"available": latest is not None,
"latest": latest,
"count": len(db_backup.list_backups(DB_PATH)),
"keep": db_backup.KEEP_BACKUPS,
}
+59 -42
View File
@@ -21,7 +21,18 @@ from pydantic import BaseModel
from core import prefs
from utils import hf_progress
from utils import download_aggregator
from .models import KNOWN_MODELS, invalidate_cache
# Weight-floor scan (MM2-07 / #352) lives in ``models.py`` — the lowest module in
# the setup import graph — so install-time validation here, the first-run
# install-state detector (#622), and load-time repair share one set of floors and
# can't drift apart. ``_MIN_WEIGHT_BYTES``/``_WEIGHT_FLOORS`` re-exported for tests.
from .models import ( # noqa: F401
KNOWN_MODELS,
invalidate_cache,
snapshot_has_weights,
disk_space_error,
_MIN_WEIGHT_BYTES,
_WEIGHT_FLOORS,
)
logger = logging.getLogger("omnivoice.setup.download")
router = APIRouter()
@@ -120,11 +131,15 @@ def compute_plan(plan_files) -> dict:
def _segmented_enabled() -> bool:
"""Opt-in IDM-style accelerator (FDL-09), default OFF. Most useful when Xet
is inactive (the app's default): the legacy-LFS path is single-stream, so
this restores parallel speed AND gives real live byte progress."""
"""IDM-style multi-connection accelerator (FDL-09), default **ON**. The app
forces the legacy-LFS path (HF_HUB_DISABLE_XET=1) for clear progress, but that
path is single-stream and slow this restores parallel byte-range speed AND
real live progress, and falls back to snapshot_download on any error so it
can never compromise a correct install. Default-on so first-run downloads are
fast out of the box (pairs with an HF token for higher rate limits); set
OMNIVOICE_SEGMENTED_DOWNLOAD=0 to force the single-stream path."""
return _truthy(prefs.resolve(
"segmented_downloader", env="OMNIVOICE_SEGMENTED_DOWNLOAD", default=False,
"segmented_downloader", env="OMNIVOICE_SEGMENTED_DOWNLOAD", default=True,
))
@@ -223,51 +238,26 @@ def _safe_put(queue: asyncio.Queue, event) -> None:
# model.safetensors" (#352). 5 MB clears every weight format we ship
# (safetensors/bin shards, onnx, pt, gguf) without false-positiving on
# config-only aux repos.
_MIN_WEIGHT_BYTES = 5 * 1024 * 1024
# Per-role weight-file floors (MM2-07). A valid model has at least one
# recognized weight file at or above its extension's floor. ONNX graphs are
# legitimately small (a complete model can be well under 5 MB), so a single
# 5 MB rule false-positives on them as "truncated" (#352 over-trigger); give
# .onnx a lower floor while still rejecting a 0/KB partial. Tensor formats keep
# the original 5 MB floor.
_WEIGHT_FLOORS = {
".safetensors": _MIN_WEIGHT_BYTES,
".bin": _MIN_WEIGHT_BYTES,
".ckpt": _MIN_WEIGHT_BYTES,
".pt": _MIN_WEIGHT_BYTES,
".pth": _MIN_WEIGHT_BYTES,
".gguf": _MIN_WEIGHT_BYTES,
".onnx": 64 * 1024, # a real ONNX graph is ≥ tens of KB; a truncated one is bytes
}
def _validate_snapshot_has_weights(repo_id: str, snapshot_path: str) -> None:
"""Raise OSError when a finished snapshot has no plausible weight file —
surfaces the truncated-download class (#352) at install time, where the
retry loop and the UI's re-download path can deal with it, instead of at
first synthesis with an opaque transformers error.
A snapshot is valid if it contains a recognized weight file meeting its
per-extension floor (MM2-07) OR any file the global 5 MB floor (the
original lenient catch kept so this is never stricter than before)."""
Delegates the weight check to ``models.snapshot_has_weights`` (single source of
the floors); only the install-time error message lives here."""
if snapshot_has_weights(snapshot_path):
return
biggest = 0
try:
biggest = 0
for root, _dirs, files in os.walk(snapshot_path, followlinks=True):
for f in files:
try:
size = os.path.getsize(os.path.join(root, f))
biggest = max(biggest, os.path.getsize(os.path.join(root, f)))
except OSError:
continue
biggest = max(biggest, size)
ext = os.path.splitext(f)[1].lower()
floor = _WEIGHT_FLOORS.get(ext)
if floor is not None and size >= floor:
return # a recognized weight file of plausible size
if size >= _MIN_WEIGHT_BYTES:
return # original lenient catch (non-standard weight names)
except OSError:
return # can't inspect — don't block the install on the checker itself
pass
raise OSError(
f"{repo_id}: download finished but no model weights were found in the "
"snapshot (largest file "
@@ -415,6 +405,26 @@ async def install_model(req: InstallModelRequest):
try:
_plan = snapshot_download(**_preflight_kwargs)
_summary = compute_plan(_plan)
# Disk-space guard (before a single byte flows): the preflight
# gives an exact "to download" size, so reject an install that
# would overrun the cache volume — with the numbers named —
# instead of failing mid-download with a cryptic OSError. No-op
# when it fits or the size is unknown. Same on every platform.
_disk_err = disk_space_error(_summary["to_download_bytes"])
if _disk_err:
logger.info("model install %s: rejected — %s", req.repo_id, _disk_err)
_resolving.set() # stop the heartbeat thread before we bail
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
"downloaded": 0, "total": 0, "pct": 0.0,
"phase": "install_error",
"error": _disk_err,
})
# A disk-full is not a transient network failure — don't set
# a cooldown (freeing space, not waiting, is the fix). The
# outer finally still cleans up the aggregator + context.
return
download_aggregator.start(
req.repo_id,
total_bytes=_summary["to_download_bytes"],
@@ -449,10 +459,11 @@ async def install_model(req: InstallModelRequest):
raise _InstallCancelled()
_attempt += 1
try:
# Opt-in segmented accelerator (FDL-09): parallel byte-range
# fetch with real live progress, for the legacy-LFS path.
# Any failure falls through to snapshot_download — the
# accelerator can never compromise a correct install.
# Segmented accelerator (FDL-09, default ON): parallel
# byte-range fetch with real live progress, for the
# legacy-LFS path. Any failure falls through to
# snapshot_download — the accelerator can never compromise a
# correct install.
_snapshot_path = None
if _attempt == 1 and _segmented_enabled() and not _xet_active():
try:
@@ -517,12 +528,18 @@ async def install_model(req: InstallModelRequest):
logger.info("model install failed for %s: %s", req.repo_id, e)
import time as _time_fail
_install_cooldowns[req.repo_id] = _time_fail.time()
# #874: when the install failed because the configured HF mirror is
# unreachable, name the mirror + the setting instead of leaking the
# raw connectivity error. #959: likewise for the SOCKS-proxy class
# (missing socksio fails the download's session construction).
# No-op for every other failure.
from core.failure import append_hint
hf_progress.emit({
"repo_id": req.repo_id,
"filename": req.repo_id,
"downloaded": 0, "total": 0, "pct": 0.0,
"phase": "install_error",
"error": str(e),
"error": append_hint(str(e)),
})
finally:
_cancelled.discard(req.repo_id)
+163 -2
View File
@@ -123,6 +123,66 @@ def hf_cache_dir() -> str:
)
# ── Disk-space guard (shared, single-sourced) ──────────────────────────────
# MIN_FREE_GB is the headroom we insist on keeping free on the model-cache
# volume — the wizard's absolute pre-install floor AND the extra buffer the
# per-install check demands on top of the download itself, so an "Install all"
# can't fill the disk to the brim (setup/download.py). Lives here — the lowest
# module in the setup import graph — so the wizard, the /models header, and the
# install endpoint can't drift apart (mirrors the weight-floor single-sourcing).
_GIB = 1024 ** 3
MIN_FREE_GB = 10
def disk_free_bytes(path: "str | None" = None) -> int:
"""Free bytes on the volume backing *path* (defaults to the HF cache).
Walks up to the nearest existing ancestor so a not-yet-created cache dir
still probes the correct mount point. ``shutil.disk_usage`` is cross-platform
(macOS/Windows/Linux) so this behaves identically everywhere. Never raises.
"""
import shutil
try:
p = Path(path or hf_cache_dir()).resolve()
while not p.exists():
parent = p.parent
if parent == p: # reached the volume root
break
p = parent
return int(shutil.disk_usage(str(p)).free)
except Exception:
return 0
def disk_space_error(to_download_bytes: "int | None", *, cache_dir: "str | None" = None) -> "str | None":
"""Actionable message when *to_download_bytes* (+ MIN_FREE_GB headroom) won't
fit on the cache volume; ``None`` when it fits, the size is unknown, or the
volume can't be probed (never block on missing information).
Names the three numbers a user needs to act needs X, headroom Y, have Z
so "Install all" can't silently overrun the disk (issue: no pre-install disk
check). Platform-agnostic; applied identically on macOS/Windows/Linux.
"""
if not to_download_bytes or to_download_bytes <= 0:
return None # unknown plan (older/gated repo, mirror without dry-run) → don't block
cache = cache_dir or hf_cache_dir()
free = disk_free_bytes(cache)
if free <= 0:
return None # couldn't probe the volume → don't block on missing info
required = int(to_download_bytes) + MIN_FREE_GB * _GIB
if free >= required:
return None
def _gb(n: int) -> str:
return f"{n / _GIB:.1f} GB"
return (
f"Not enough disk space to install: this download needs {_gb(int(to_download_bytes))} "
f"plus {MIN_FREE_GB} GB free headroom ({_gb(required)} total), but only {_gb(free)} "
f"is free at {cache}. Free up space (or move the model cache to a bigger volume) and retry."
)
def _repo_dir_name(repo_id: str) -> str:
"""HF cache dir name for a repo: 'k2-fsa/OmniVoice''models--k2-fsa--OmniVoice'."""
return "models--" + repo_id.replace("/", "--")
@@ -146,6 +206,94 @@ def _hub_cache_roots() -> list[str]:
return roots
# ── Weight-presence (truncated-cache) detection ─────────────────────────────
# A cache that downloaded config/tokenizer files but not the weight shard still
# occupies bytes on disk, so a size-only "installed" check (#352/#581/#606) reads
# it as installed and the first-run wizard hides the re-download button, stranding
# the user (#622). These helpers tell a *complete* snapshot from a truncated one by
# checking for a plausible weight file — the same class `download.py` guards at
# install time and `model_manager.py` repairs at load time. Shared here (the lowest
# module in the setup import graph; `download.py` imports from this module) so the
# floors live in exactly one place and can't drift between the three call sites.
_MIN_WEIGHT_BYTES = 5 * 1024 * 1024 # tensor formats: a real shard is ≥ a few MB
# Per-extension floors. ONNX graphs are legitimately small (a complete model can be
# well under 5 MB), so they get a lower floor that still rejects a bytes-only partial.
_WEIGHT_FLOORS = {
".safetensors": _MIN_WEIGHT_BYTES,
".bin": _MIN_WEIGHT_BYTES,
".ckpt": _MIN_WEIGHT_BYTES,
".pt": _MIN_WEIGHT_BYTES,
".pth": _MIN_WEIGHT_BYTES,
".gguf": _MIN_WEIGHT_BYTES,
".onnx": 64 * 1024,
}
def snapshot_has_weights(snapshot_path: str) -> bool:
"""True when a finished snapshot dir holds a plausible weight file.
A snapshot is complete if it contains a recognized weight file meeting its
per-extension floor OR any file the global 5 MB floor (the lenient catch for
non-standard weight names). Returns True when the path can't be inspected — an
un-walkable dir must never be reported as truncated, only a confirmed weight-less
one. `getsize` follows symlinks, so HF's snapshot→blob links resolve correctly;
a broken link (missing blob) raises OSError and is skipped, i.e. counts as absent.
"""
try:
for root, _dirs, files in os.walk(snapshot_path, followlinks=True):
for f in files:
try:
size = os.path.getsize(os.path.join(root, f))
except OSError:
continue
ext = os.path.splitext(f)[1].lower()
floor = _WEIGHT_FLOORS.get(ext)
if floor is not None and size >= floor:
return True
if size >= _MIN_WEIGHT_BYTES:
return True
except OSError:
return True # can't inspect — don't mislabel as truncated
return False
def _snapshot_dirs(repo_id: str) -> list[str]:
"""Existing snapshot revision dirs for a repo across the candidate cache roots."""
name = _repo_dir_name(repo_id)
dirs: list[str] = []
for root in _hub_cache_roots():
snaps = os.path.join(root, name, "snapshots")
try:
for rev in os.listdir(snaps):
rev_dir = os.path.join(snaps, rev)
if os.path.isdir(rev_dir):
dirs.append(rev_dir)
except OSError:
continue
return dirs
def cache_is_complete(model: dict) -> bool:
"""True when this model's on-disk cache is usable (not a truncated download).
Config-only repos (``config_only: true`` in models.yaml e.g. pyannote's
diarisation pipeline, whose real weights live in referenced sub-repos) carry no
weight file of their own, so the weight check would false-positive them as
incomplete (#622 caveat). They're exempt: cache presence alone means complete.
A weight-bearing repo is complete only if at least one of its snapshots has
weights; if no snapshot dir is found on disk we can't prove truncation, so we
don't downgrade (the size-based caller already decided it's cached).
"""
if model.get("config_only"):
return True
dirs = _snapshot_dirs(model["repo_id"])
if not dirs:
return True
return any(snapshot_has_weights(d) for d in dirs)
def _is_cached_on_disk(repo_id: str) -> bool:
"""Direct-filesystem fallback for is_cached when scan_cache_dir is unavailable.
@@ -286,9 +434,15 @@ def list_models():
out = []
for m in KNOWN_MODELS:
cached = cached_by_repo.get(m["repo_id"])
on_disk = cached is not None and cached["size_on_disk"] > 0
# A size-positive cache can still be a truncated download (config landed,
# weight shard didn't). Treat that as not-installed + incomplete so the
# wizard re-offers the download instead of stranding the user (#622).
incomplete = on_disk and not cache_is_complete(m)
out.append({
**m,
"installed": cached is not None and cached["size_on_disk"] > 0,
"installed": on_disk and not incomplete,
"incomplete": incomplete,
"size_on_disk_bytes": cached["size_on_disk"] if cached else 0,
"nb_files": cached["nb_files"] if cached else 0,
"supported": _model_supported(m),
@@ -297,6 +451,10 @@ def list_models():
"models": out,
"total_installed_bytes": sum(m["size_on_disk_bytes"] for m in out),
"hf_cache_dir": hf_cache_dir(),
# Free space on the cache volume, so the Model Store header can warn
# BEFORE an "Install all" overruns the disk (pairs with the per-install
# disk_space_error guard in setup/download.py).
"disk_free_gb": round(disk_free_bytes() / _GIB, 1),
"platform_tags": _current_platform_tags(),
}
_set_cache("models", response)
@@ -383,6 +541,9 @@ def recommendations():
entries = []
for rid in recommended_ids:
meta = known_by_id.get(rid, {})
# Mirror /models: a truncated cache (weights missing) is not installed, so
# the wizard counts it toward the remaining download instead of "all set".
installed = rid in cached_ids and cache_is_complete(meta or {"repo_id": rid})
entries.append({
"repo_id": rid,
"label": meta.get("label", rid),
@@ -390,7 +551,7 @@ def recommendations():
"size_gb": meta.get("size_gb", 0),
"required": bool(meta.get("required", False)),
"note": meta.get("note"),
"installed": rid in cached_ids,
"installed": installed,
})
to_download_gb = sum(e["size_gb"] for e in entries if not e["installed"])
+76 -31
View File
@@ -18,33 +18,20 @@ import sys
from fastapi import APIRouter
from api.schemas import SetupStatusResponse, PreflightResponse
from .models import REQUIRED_MODELS, hf_cache_dir, is_cached
# MIN_FREE_GB + disk_free_bytes are single-sourced in ``.models`` (the lowest
# module in the setup import graph) so the wizard gate, the /models header, and
# the per-install disk guard can't drift apart.
from .models import REQUIRED_MODELS, hf_cache_dir, is_cached, MIN_FREE_GB, disk_free_bytes
logger = logging.getLogger("omnivoice.setup.wizard")
router = APIRouter()
MIN_FREE_GB = 10
def _disk_free_gb(path: str) -> float:
"""Return free GB on the volume containing *path*.
If *path* doesn't exist yet (e.g. after a fresh wipe), walk up to the
nearest existing ancestor so ``shutil.disk_usage`` can still probe the
correct mount point.
"""
try:
from pathlib import Path
p = Path(path).resolve()
# Walk up until we find a directory that exists
while not p.exists():
parent = p.parent
if parent == p: # root
break
p = parent
return _shutil.disk_usage(str(p)).free / (1024 ** 3)
except Exception:
return 0.0
"""Free GB on the volume containing *path* (thin GB wrapper over the shared
``models.disk_free_bytes``, which walks up to the nearest existing ancestor
for a not-yet-created path)."""
return disk_free_bytes(path) / (1024 ** 3)
# ── Setup Status ───────────────────────────────────────────────────────────
@@ -181,16 +168,40 @@ def _detect_gpu() -> dict:
return info
def _probe_network(host: str = "huggingface.co", timeout: float = 2.0) -> bool:
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 2.0) -> bool:
"""Tiny TCP connect test."""
import socket
try:
with socket.create_connection((host, 443), timeout=timeout):
with socket.create_connection((host, port), timeout=timeout):
return True
except Exception:
return False
def _hf_endpoint_host() -> tuple[str, int]:
"""Host/port of the Hugging Face endpoint actually in effect.
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
point HF_ENDPOINT at a mirror via Settings Models Hugging Face
mirror. Probing hardcoded huggingface.co would fail them even when their
configured mirror works fine.
"""
try:
from core.failure import configured_hf_mirror
mirror = configured_hf_mirror()
except Exception:
mirror = ""
if mirror:
try:
from urllib.parse import urlsplit
u = urlsplit(mirror)
if u.hostname:
return u.hostname, u.port or (80 if u.scheme == "http" else 443)
except Exception:
pass
return "huggingface.co", 443
def _ram_gb() -> float:
try:
import psutil
@@ -413,15 +424,49 @@ def preflight():
"status": r_status, "detail": r_detail, "fix": r_fix,
})
# ── Network
net_ok = _probe_network()
# ── Network — probes the HF endpoint actually in effect (mirror-aware),
# and a dead network is a WARNING, not a blocker. The app is local-first:
# already-downloaded models work offline, and a hard fail here dead-ends
# restricted-network users (e.g. China, where huggingface.co is blocked)
# on the very first screen — before they can reach the mirror setting
# that fixes it. Model downloads surface their own actionable errors.
net_host, net_port = _hf_endpoint_host()
net_ok = _probe_network(net_host, net_port)
mirror_reachable = False
if not net_ok and net_host == "huggingface.co":
# Official endpoint blocked — if the community mirror is reachable,
# tell the user exactly which switch unblocks them.
mirror_reachable = _probe_network("hf-mirror.com")
if net_ok:
net_fix = None
elif mirror_reachable:
net_fix = (
"huggingface.co is blocked on this network, but the hf-mirror.com "
"community mirror is reachable — apply it below and re-check. "
"Model downloads will use the mirror immediately."
)
elif net_host != "huggingface.co":
net_fix = (
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
"— it may be down or blocked. Pick another mirror or the official "
"endpoint below, or continue offline: models already downloaded "
"keep working."
)
else:
net_fix = (
"Check internet connection, VPN, or corporate firewall whitelist "
"for huggingface.co. You can continue — models already downloaded "
"keep working offline; new downloads need a connection or a "
"mirror (configurable below)."
)
checks.append({
"id": "network", "label": "Network (huggingface.co)",
"status": "pass" if net_ok else "fail",
"detail": "Reachable" if net_ok else "Unreachable on port 443",
"fix": None if net_ok else
"Check internet connection, VPN, or corporate firewall "
"whitelist for huggingface.co.",
"id": "network", "label": f"Network ({net_host})",
"status": "pass" if net_ok else "warn",
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
"fix": net_fix,
# Frontend affordance hint: the wizard offers the mirror quick-pick
# when the endpoint is unreachable (PreflightCheck allows extras).
"mirror_reachable": mirror_reachable,
})
# Aggregate
+2 -2
View File
@@ -18,7 +18,7 @@ import shutil
from core.config import OUTPUTS_DIR, DATA_DIR, CRASH_LOG_PATH, LOG_PATH, IDLE_TIMEOUT_SECONDS
from core.version import APP_VERSION
from services.model_manager import get_model_status, get_best_device
from services.model_manager import get_model_status, get_best_device, resolve_omnivoice_checkpoint
from services.ffmpeg_utils import find_ffmpeg, run_ffmpeg
# Router-level loopback gate. Every route mounted on `router` (GET + POST,
@@ -208,7 +208,7 @@ def system_info():
"outputs_dir": OUTPUTS_DIR,
"crash_log_path": CRASH_LOG_PATH,
"idle_timeout_seconds": IDLE_TIMEOUT_SECONDS,
"model_checkpoint": os.environ.get("OMNIVOICE_MODEL", "k2-fsa/OmniVoice"),
"model_checkpoint": resolve_omnivoice_checkpoint(), # #693: show the effective checkpoint, not a leaked raw value
"asr_model": os.environ.get("ASR_MODEL", "Systran/faster-whisper-large-v3"),
"translate_provider": os.environ.get("TRANSLATE_PROVIDER", "google"),
"has_hf_token": _has_hf_token(),
+5
View File
@@ -77,6 +77,10 @@ async def probe(req: ProbeReq):
class IncrementalReq(BaseModel):
segments: list[dict]
stored_hashes: Optional[dict[str, str]] = None
# P1.3 — the ACTIVE track's language code. When set, fingerprints are
# scoped to that language (pass that language's stored hashes alongside);
# omitted → legacy language-agnostic hashing, kept for old callers.
lang: Optional[str] = None
@router.post("/tools/incremental")
@@ -84,6 +88,7 @@ def plan_incremental(req: IncrementalReq):
return incremental.plan_incremental(
req.segments,
stored_hashes=req.stored_hashes or {},
track_lang=req.lang,
)
+9 -4
View File
@@ -182,8 +182,8 @@ async def ws_tts(websocket: WebSocket):
sentences = [text]
# Run generation in the GPU pool
from services.model_manager import _gpu_pool
loop = asyncio.get_running_loop()
import functools
from services.model_manager import run_on_gpu_pool_guarded
def _generate(sentence_text):
from services.audio_dsp import apply_mastering, normalize_audio
@@ -204,8 +204,13 @@ async def ws_tts(websocket: WebSocket):
started = False
for sentence in sentences:
wav_tensor, sr = await loop.run_in_executor(
_gpu_pool, _generate, sentence
# Bounded + pool-reset on hang so a wedged generate can't
# starve the GPU pool and brick the backend (#730 class). On
# timeout GpuJobTimeoutError propagates to the handler below,
# which sends an actionable error frame.
wav_tensor, sr = await run_on_gpu_pool_guarded(
functools.partial(_generate, sentence),
what="TTS generate",
)
if not started:
Binary file not shown.
+75
View File
@@ -14,6 +14,10 @@
# required (optional) — true if the app needs this model to function
# platforms (optional) — restrict to specific OS+arch tags (e.g. darwin-arm64, cuda)
# note (optional) — shown in the UI as a tooltip/footnote
# config_only (optional) — true for pipeline repos that ship no weight file of
# their own (weights live in referenced sub-repos). Such
# a cache is legitimately tiny, so the truncated-download
# (weights-missing) detector must NOT flag it incomplete.
# ─────────────────────────────────────────────────────────────────────────
models:
@@ -116,12 +120,83 @@ models:
size_gb: 0.05
note: "Smallest/fastest Moonshine, sub-200ms latency. Lower accuracy than base. Requires moonshine-onnx."
# ── sherpa-onnx live dictation (ONNX, CPU, streaming + offline) ────────
# Live faster-than-real-time dictation via the k2-fsa/sherpa-onnx runtime.
# `engine: sherpa-onnx`, `dictation_id` (backend model id), and `tag`
# (offline | streaming) are extra fields the model-store list passes through
# so the dictation UI can filter/group these (role=ASR, engine=sherpa-onnx).
# Requires `uv add sherpa-onnx` (CPU wheels, all platforms).
- repo_id: "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8"
label: "Parakeet TDT v3 (sherpa-onnx — dictation, 25 EU langs)"
role: ASR
size_gb: 0.18
engine: sherpa-onnx
dictation_id: sherpa-parakeet-tdt-v3
tag: offline
note: "Recommended live-dictation default. CPU, int8 ONNX. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8"
label: "Parakeet TDT v2 (sherpa-onnx — dictation, English)"
role: ASR
size_gb: 0.17
engine: sherpa-onnx
dictation_id: sherpa-parakeet-tdt-v2
tag: offline
note: "English live dictation. CPU, int8 ONNX. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20"
label: "Zipformer Bilingual (sherpa-onnx — streaming, zh+en)"
role: ASR
size_gb: 0.13
engine: sherpa-onnx
dictation_id: sherpa-zipformer-bilingual-zh-en
tag: streaming
note: "True streaming partials as you speak (zh+en). CPU. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en"
label: "Paraformer Bilingual (sherpa-onnx — streaming, zh+en)"
role: ASR
size_gb: 0.115
engine: sherpa-onnx
dictation_id: sherpa-paraformer-bilingual-zh-en
tag: streaming
note: "True streaming partials (zh+en). CPU. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-en-20M-2023-02-17"
label: "Zipformer Streaming EN 20M (sherpa-onnx — streaming, English)"
role: ASR
size_gb: 0.128
engine: sherpa-onnx
dictation_id: sherpa-zipformer-en-20m
tag: streaming
note: "Tiny English streaming model, very low latency. CPU. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-streaming-zipformer-zh-14M-2023-02-23"
label: "Zipformer Streaming ZH 14M (sherpa-onnx — streaming, Chinese)"
role: ASR
size_gb: 0.074
engine: sherpa-onnx
dictation_id: sherpa-zipformer-zh-14m
tag: streaming
note: "Tiny Chinese streaming model, very low latency. CPU. Requires sherpa-onnx."
- repo_id: "csukuangfj/sherpa-onnx-whisper-tiny"
label: "Whisper Tiny (sherpa-onnx — dictation, 90+ langs)"
role: ASR
size_gb: 0.116
engine: sherpa-onnx
dictation_id: sherpa-whisper-tiny
tag: offline
note: "Multilingual offline dictation (auto-detect). CPU, int8 ONNX. Requires sherpa-onnx."
# ── Diarisation ───────────────────────────────────────────────────────
- repo_id: "pyannote/speaker-diarization-3.1"
label: "pyannote speaker diarisation (multi-speaker videos)"
role: Diarisation
size_gb: 0.8
config_only: true # pipeline repo; real weights live in referenced sub-repos
note: "Needs an HF_TOKEN with license accepted."
# ── Optional TTS ──────────────────────────────────────────────────────
+138
View File
@@ -0,0 +1,138 @@
"""Parse the shipped CHANGELOG.md into structured release notes.
Feeds ``GET /api/settings/changelog`` the Settings Updates "What's new"
viewer. Local-first by design: the changelog ships with the app (repo root in
dev; copied into the packaged project dir by the Tauri bootstrap alongside
README.md), so the viewer works fully offline.
The house format (see CHANGELOG.md / the release-notes hard rule):
## [X.Y.Z] — DATE
one-paragraph headline (the "intro")
### Added / Fixed / Changed / ...
- **Bold one-line lead.** 1-3 lines of plain-English why. (#NNN)
Bullets may be a single long line (recent sections) *or* hard-wrapped across
indented continuation lines (older sections) the parser normalizes both to
one logical line per bullet. Bullets stay raw markdown-lite; the frontend's
safe renderer handles **bold** / `code` / (#NNN) refs.
"""
from __future__ import annotations
import os
import re
#: ``## [0.3.9] — 2026-07-02`` (em/en dash or hyphen; date optional).
_RELEASE_RE = re.compile(r"^##\s+\[(?P<version>[^\]]+)\]\s*(?:[—–-]\s*(?P<date>.+?))?\s*$")
_SECTION_RE = re.compile(r"^###\s+(?P<title>.+?)\s*$")
_BULLET_RE = re.compile(r"^\s*[-*]\s+(?P<text>.*\S)\s*$")
def changelog_path() -> str | None:
"""The shipped CHANGELOG.md, or None when this install doesn't have one.
``backend/core/changelog.py`` two levels up is the project root: the
repo root in dev, and ``<env>/project`` in packaged installs (where the
bootstrap copies CHANGELOG.md next to README.md). ``OMNIVOICE_CHANGELOG``
overrides for tests/containers.
"""
override = os.environ.get("OMNIVOICE_CHANGELOG")
if override:
return override if os.path.isfile(override) else None
here = os.path.dirname(os.path.abspath(__file__))
candidate = os.path.join(os.path.dirname(os.path.dirname(here)), "CHANGELOG.md")
return candidate if os.path.isfile(candidate) else None
def _looks_like_release_version(version: str) -> bool:
"""Only released ``X.Y.Z...`` sections (skip ``[Unreleased]`` etc.)."""
return bool(re.match(r"^v?\d", version.strip()))
def parse_changelog(text: str, limit_versions: int = 5) -> list[dict]:
"""CHANGELOG.md text → newest-first list of releases::
{"version": "0.3.9", "date": "2026-07-02", "intro": "",
"sections": [{"title": "Fixed", "bullets": ["", ]}, ]}
Tolerates both single-line bullets and older hard-wrapped bullets
(continuation lines are joined with a space). Content between the version
heading and the first ``###`` becomes ``intro`` (paragraphs joined by
blank lines).
"""
releases: list[dict] = []
release: dict | None = None
section: dict | None = None
intro_parts: list[str] = []
bullet_open = False # last bullet may still absorb continuation lines
intro_new_para = True
def close_release():
nonlocal release, section, intro_parts, bullet_open, intro_new_para
if release is not None:
release["intro"] = "\n\n".join(p for p in intro_parts if p)
release["sections"] = [s for s in release["sections"] if s["bullets"]]
releases.append(release)
release = None
section = None
intro_parts = []
bullet_open = False
intro_new_para = True
for raw in text.splitlines():
m = _RELEASE_RE.match(raw)
if m:
close_release()
if len(releases) >= limit_versions:
break
version = m.group("version").strip().lstrip("v")
if not _looks_like_release_version(version):
continue # e.g. [Unreleased] — skip until the next heading
release = {
"version": version,
"date": (m.group("date") or "").strip(),
"intro": "",
"sections": [],
}
continue
if release is None:
continue
line = raw.strip()
if not line:
bullet_open = False
intro_new_para = True
continue
sm = _SECTION_RE.match(raw)
if sm:
section = {"title": sm.group("title"), "bullets": []}
release["sections"].append(section)
bullet_open = False
continue
bm = _BULLET_RE.match(raw)
if bm:
if section is None:
# Rare: a bullet before any ### heading — group it untitled.
section = {"title": "", "bullets": []}
release["sections"].append(section)
section["bullets"].append(bm.group("text"))
bullet_open = True
continue
if section is not None:
if bullet_open and section["bullets"]:
# Hard-wrapped bullet continuation (older sections) → join.
section["bullets"][-1] += " " + line
continue
# Headline paragraph(s) before the first ### section.
if intro_new_para or not intro_parts:
intro_parts.append(line)
else:
intro_parts[-1] += " " + line
intro_new_para = False
close_release()
return releases[:limit_versions]
+227 -8
View File
@@ -3,6 +3,8 @@ import sqlite3
import logging
from contextlib import contextmanager
from core.config import DB_PATH
from core import db_backup
from core.version import APP_VERSION
logger = logging.getLogger("omnivoice.db")
@@ -157,6 +159,22 @@ _BASE_SCHEMA = """
last_seen_at REAL,
created_at REAL
);
-- Expressive-TTS Spec 01 Phase 1: user pronunciation dictionary. A
-- per-language wordrespelling map applied as pure text substitution
-- before synthesis (Settings Pronunciation). Fresh installs create it
-- here; existing DBs get it via alembic 0008_pronunciation_dictionary.
-- Both paths converge on this identical schema (dual-path discipline).
CREATE TABLE IF NOT EXISTS pronunciation_entries (
id TEXT PRIMARY KEY,
term TEXT NOT NULL,
replacement TEXT NOT NULL DEFAULT '',
type TEXT NOT NULL DEFAULT 'respelling',
language TEXT NOT NULL DEFAULT '*',
enabled INTEGER NOT NULL DEFAULT 1,
created_at REAL
);
CREATE INDEX IF NOT EXISTS idx_pron_lang ON pronunciation_entries(language);
"""
# Only tables/columns this module is allowed to ALTER. Prevents SQL injection via
@@ -206,6 +224,72 @@ def _migrate(conn, current: int) -> int:
return current
def _reconcile_additive_columns(conn) -> None:
"""Make the live schema converge to ``_BASE_SCHEMA`` by ADDing any column the
canonical schema declares but an existing table is missing the belt for
when alembic can't run on an upgraded DB.
``CREATE TABLE IF NOT EXISTS`` (init_db) never adds columns to a table that
already exists, the legacy ``_migrate`` only knows pre-0.3 columns, and
``_run_alembic_upgrade`` swallows failures. So a DB whose ``alembic_version``
is stamped at a removed revision (e.g. after running a preview build), or
where alembic isn't importable in the bundled interpreter, would otherwise
lose every alembic-era additive column forever the ``no such column:
consent_audio_path`` 500 (#552/#547), and the same class for
``kind``/``vd_states``/``is_demo``/.... Additive only: never drops or retypes
a column, so it is safe and backward-compatible with existing user data. The
canonical names/types/defaults come solely from ``_BASE_SCHEMA`` (developer
controlled), so the ALTER is injection-safe.
"""
canon = sqlite3.connect(":memory:")
try:
canon.executescript(_BASE_SCHEMA)
_tables_sql = "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'"
live_tables = {r[0] for r in conn.execute(_tables_sql)}
for table in (r[0] for r in canon.execute(_tables_sql)):
if table not in live_tables:
continue # whole table missing → init_db's CREATE already made it
have = {r[1] for r in conn.execute(f"PRAGMA table_info({table})")}
# (cid, name, type, notnull, dflt_value, pk)
for _cid, name, ctype, notnull, dflt, _pk in canon.execute(f"PRAGMA table_info({table})"):
if name in have or not _IDENT_RE.match(name):
continue
ddl = f'ALTER TABLE "{table}" ADD COLUMN "{name}" {ctype or "TEXT"}'
if dflt is not None:
ddl += f" DEFAULT {dflt}"
elif notnull:
ddl += " DEFAULT ''" # SQLite requires a default to ADD a NOT NULL column
try:
conn.execute(ddl)
logger.info("schema reconcile: added missing column %s.%s", table, name)
except sqlite3.OperationalError as exc:
if "duplicate column" not in str(exc).lower():
logger.warning("schema reconcile ALTER %s.%s failed: %s", table, name, exc)
conn.commit()
finally:
canon.close()
def ensure_schema() -> None:
"""Idempotently ensure the base tables + additive columns exist.
A runtime self-heal for a DB that somehow missed init e.g. a write hitting
``no such table: generation_history`` (#710) because ``init_db()``'s
``executescript`` never took on that DB. Safe to call anytime: it's just
``CREATE ... IF NOT EXISTS`` plus the additive-only column reconcile, so it
never drops or retypes anything and is backward-compatible with user data.
Cheaper than ``init_db()`` (skips the legacy ``_migrate`` + alembic), so a
write path can call it on a schema error and retry without a 500.
"""
conn = get_db()
try:
conn.executescript(_BASE_SCHEMA)
_reconcile_additive_columns(conn)
conn.commit()
finally:
conn.close()
def init_db():
conn = get_db()
try:
@@ -214,6 +298,11 @@ def init_db():
new_version = _migrate(conn, version)
if new_version != version:
conn.execute(f"PRAGMA user_version = {new_version}")
# Converge any alembic-era additive columns that CREATE TABLE IF NOT
# EXISTS + the legacy _migrate don't add to a pre-existing table
# (consent_audio_path, kind, ...). Runs regardless of whether alembic
# below succeeds, so an unrunnable alembic can't leave a 500-ing schema.
_reconcile_additive_columns(conn)
conn.commit()
finally:
conn.close()
@@ -225,12 +314,88 @@ def init_db():
_run_alembic_upgrade()
class MigrationError(RuntimeError):
"""A schema migration failed *while executing*. Startup must NOT continue
on a possibly half-migrated database the caller lets this propagate so
the process stops with an actionable message naming the pre-migration
backup (see ``core.db_backup``). Restore is deliberately manual: silently
auto-restoring the snapshot could itself discard user data."""
def _reconcile_after_alembic_skip() -> None:
"""Converge the schema directly when alembic can't run at all (not
importable, or stamped at a removed revision #552/#547) so additive
columns still land instead of 500-ing on `no such column`. Only for the
"nothing was applied" classes; a mid-migration failure must NOT reach
here (see MigrationError)."""
try:
conn = get_db()
try:
_reconcile_additive_columns(conn)
finally:
conn.close()
except Exception as exc: # noqa: BLE001
logger.warning("schema reconcile after alembic skip also failed: %s", exc)
def _stamped_revisions(db_path: str) -> set | None:
"""Revisions recorded in ``alembic_version`` (empty set = never stamped),
or None when the DB can't be read."""
try:
conn = sqlite3.connect(db_path)
try:
try:
return {r[0] for r in conn.execute("SELECT version_num FROM alembic_version")}
except sqlite3.OperationalError:
return set() # table absent — nothing ever stamped
finally:
conn.close()
except Exception: # noqa: BLE001
return None
def _plan_alembic(cfg) -> str:
"""Decide what an ``upgrade head`` run would actually do:
- ``up_to_date`` stamped at head; upgrade is a no-op.
- ``pending`` migrations WILL execute (snapshot the DB first).
- ``unknown_revision`` stamped at a revision this build doesn't ship
(previewstable downgrade, #552/#547); upgrade would fail before
applying anything, so skip it and reconcile additively instead.
- ``indeterminate`` can't tell; treat like pending (snapshot, run).
"""
try:
from alembic.script import ScriptDirectory
script = ScriptDirectory.from_config(cfg)
known = {rev.revision for rev in script.walk_revisions()}
heads = set(script.get_heads())
stamped = _stamped_revisions(DB_PATH)
if stamped is None:
return "indeterminate"
if stamped and not stamped <= known:
return "unknown_revision"
if stamped == heads:
return "up_to_date"
return "pending"
except Exception: # noqa: BLE001
return "indeterminate"
def _run_alembic_upgrade() -> None:
"""Best-effort `alembic upgrade head` on startup. Non-fatal: if alembic
isn't reachable (e.g. user running a stripped-down install or migrations
were already applied out-of-band), log a warning and move on. The
_BASE_SCHEMA CREATE TABLE IF NOT EXISTS above guarantees the runtime
schema is correct regardless."""
"""`alembic upgrade head` on startup, wrapped in the data-safety net.
Failure classes are handled differently on purpose:
- alembic unavailable / stamped at an unknown revision **non-fatal**
(nothing was applied; warn + `_reconcile_additive_columns` keeps the
schema converged, exactly the pre-existing #552/#547 behavior).
- migrations actually pending the DB is snapshotted first
(``omnivoice.db.backup-<version>-<n>``, newest 3 kept), then upgraded.
- a migration fails **while executing** raise :class:`MigrationError`:
startup stops with a message naming the backup, instead of silently
running the app on a half-migrated DB.
"""
try:
import os
from alembic import command
@@ -246,8 +411,62 @@ def _run_alembic_upgrade() -> None:
return
cfg = Config(ini)
cfg.set_main_option("sqlalchemy.url", f"sqlite:///{DB_PATH}")
except Exception as exc: # noqa: BLE001 — alembic not importable / bad ini
logger.warning("alembic upgrade head skipped: %s", exc)
_reconcile_after_alembic_skip()
return
plan = _plan_alembic(cfg)
if plan == "up_to_date":
return
if plan == "unknown_revision":
logger.warning(
"alembic_version is stamped at a revision this build doesn't ship "
"(preview/newer build ran on this DB) — skipping alembic and "
"reconciling the schema additively (#552/#547)"
)
_reconcile_after_alembic_skip()
return
# Migrations may actually execute: snapshot the DB first so a failed or
# interrupted migration can never cost user data. A backup problem alone
# must not brick startup (the >500 MB skip is by design), so log and go on.
# ``db_backup``/``APP_VERSION`` are module-level imports (top of file), not
# re-imported here: a test that patches ``core.db_backup.MAX_BACKUP_DB_BYTES``
# on the object it imported at collection must see the same object this
# function uses. A lazy ``from core import db_backup`` would re-resolve
# through the (possibly re-imported) ``core`` package and silently miss the
# patch after another suite purged ``core.*`` from ``sys.modules``.
backup_path = None
try:
backup_path = db_backup.snapshot_before_migration(DB_PATH, APP_VERSION)
except Exception: # noqa: BLE001
logger.exception("Pre-migration DB backup failed — continuing without one")
try:
command.upgrade(cfg, "head")
except Exception as exc:
# Don't block startup on a migration tooling problem. The runtime
# schema is already correct via _BASE_SCHEMA.
logger.warning("alembic upgrade head skipped: %s", exc)
if "Can't locate revision" in str(exc):
# Belt for an unknown-revision case _plan_alembic missed: alembic
# bails before applying anything, so the old non-fatal path is safe.
logger.warning("alembic upgrade head skipped: %s", exc)
_reconcile_after_alembic_skip()
return
backup_note = (
f"A backup of your data from just before the migration is at: {backup_path}"
if backup_path
else "No pre-migration backup was written this run (see the log above)"
)
msg = (
f"Database migration failed while running: {exc}. "
f"OmniVoice stopped instead of running on a partially migrated database, "
f"and nothing was auto-restored (your database at {DB_PATH} was left "
f"exactly as the failed migration left it). "
f"{backup_note}. "
"What to do: relaunch to retry; if it keeps failing, report it at "
"https://github.com/debpalash/OmniVoice-Studio/issues (keep the backup file). "
"To roll back manually: quit the app, replace omnivoice.db with the backup "
"file, and reinstall the previous version."
)
logger.error(msg)
raise MigrationError(msg) from exc
+169
View File
@@ -0,0 +1,169 @@
"""Pre-migration SQLite safety net (data-safe updates).
Before ``alembic upgrade head`` applies *pending* migrations at startup
which is exactly the first launch of a new app version that changed the
schema the live database is snapshotted next to itself as
``omnivoice.db.backup-<version>-<n>`` so a failed or interrupted migration
can never cost user data (voices, projects, history, settings).
Design rules (owner intent: "never corrupt/erase user data on update"):
- Snapshots use the SQLite online-backup API (``sqlite3.Connection.backup``),
not a file copy the live DB runs in WAL mode, so a plain copy could miss
everything still sitting in ``omnivoice.db-wal``.
- Only the most recent ``KEEP_BACKUPS`` snapshots are kept; older ones are
pruned so backups can't grow without bound.
- DBs larger than ``MAX_BACKUP_DB_BYTES`` are skipped with a log line (a
multi-hundred-MB copy on every schema upgrade is worse than the risk it
hedges on those installs).
- Restore is NEVER automatic. On migration failure the caller
(``core.db._run_alembic_upgrade``) stops startup and names the backup path
so the user (or a support thread) decides a silent auto-restore could
itself discard data written after the snapshot.
"""
from __future__ import annotations
import logging
import os
import re
import sqlite3
import time
logger = logging.getLogger("omnivoice.db.backup")
#: Keep this many snapshots; older ones are pruned after each new snapshot.
KEEP_BACKUPS = 3
#: Skip the snapshot (with a log line) when the DB exceeds this size.
MAX_BACKUP_DB_BYTES = 500 * 1024 * 1024
#: ``<db name>.backup-<version>-<n>`` — ``<version>`` may itself contain
#: dashes (preview builds stamp ``0.3.9-41``), so the counter is the final
#: ``-<digits>`` group.
_BACKUP_SUFFIX_RE = re.compile(r"\.backup-(?P<version>.+)-(?P<n>\d+)$")
def _sanitize_version(version: str) -> str:
"""Version string → filesystem-safe fragment (defense in depth; real
versions are semver and already safe)."""
safe = re.sub(r"[^A-Za-z0-9._-]", "_", str(version).strip()) or "unknown"
return safe[:64]
def list_backups(db_path: str) -> list[str]:
"""All backup files for ``db_path``, newest first (mtime desc)."""
directory = os.path.dirname(os.path.abspath(db_path)) or "."
base = os.path.basename(db_path)
try:
names = os.listdir(directory)
except OSError:
return []
out = []
for name in names:
if not name.startswith(base + ".backup-"):
continue
if not _BACKUP_SUFFIX_RE.search(name[len(base):]):
continue
out.append(os.path.join(directory, name))
out.sort(key=lambda p: (_mtime(p), p), reverse=True)
return out
def _mtime(path: str) -> float:
try:
return os.path.getmtime(path)
except OSError:
return 0.0
def latest_backup(db_path: str) -> dict | None:
"""Newest backup as ``{"path", "created_at", "size_bytes"}`` or None."""
backups = list_backups(db_path)
if not backups:
return None
path = backups[0]
try:
st = os.stat(path)
except OSError:
return None
return {"path": path, "created_at": st.st_mtime, "size_bytes": st.st_size}
def _next_counter(db_path: str, safe_version: str) -> int:
"""Next free ``<n>`` for this version so a re-run never overwrites an
earlier snapshot of the same version."""
base = os.path.basename(db_path)
prefix = f"{base}.backup-{safe_version}-"
highest = 0
for path in list_backups(db_path):
name = os.path.basename(path)
if not name.startswith(prefix):
continue
tail = name[len(prefix):]
if tail.isdigit():
highest = max(highest, int(tail))
return highest + 1
def prune_backups(db_path: str, keep: int = KEEP_BACKUPS) -> list[str]:
"""Delete all but the ``keep`` newest backups. Returns deleted paths."""
deleted = []
for path in list_backups(db_path)[keep:]:
try:
os.remove(path)
deleted.append(path)
logger.info("Pruned old DB backup %s", path)
except OSError as exc:
logger.warning("Could not prune old DB backup %s: %s", path, exc)
return deleted
def snapshot_before_migration(db_path: str, version: str) -> str | None:
"""Snapshot ``db_path`` to ``<db>.backup-<version>-<n>``.
Returns the backup path, or None when skipped (no DB yet, or DB larger
than ``MAX_BACKUP_DB_BYTES``). Raises on an actual backup failure so the
caller can decide (the caller treats that as "continue without a backup",
logged loudly a backup problem must not brick startup by itself).
"""
if not os.path.isfile(db_path):
logger.debug("No DB at %s yet — nothing to back up", db_path)
return None
size = os.path.getsize(db_path)
if size > MAX_BACKUP_DB_BYTES:
logger.info(
"Skipping pre-migration DB backup: %s is %.0f MB (> %.0f MB limit)",
db_path, size / (1024 * 1024), MAX_BACKUP_DB_BYTES / (1024 * 1024),
)
return None
safe_version = _sanitize_version(version)
target = f"{db_path}.backup-{safe_version}-{_next_counter(db_path, safe_version)}"
tmp = f"{target}.part-{os.getpid()}"
src = sqlite3.connect(db_path)
try:
dst = sqlite3.connect(tmp)
try:
# Online backup: consistent snapshot including WAL contents.
src.backup(dst)
dst.commit()
finally:
dst.close()
except BaseException:
try:
os.remove(tmp)
except OSError:
pass
raise
finally:
src.close()
os.replace(tmp, target)
# A same-second rotation must still rank the new file newest.
try:
now = time.time()
os.utime(target, (now, now))
except OSError:
pass
logger.info("Pre-migration DB backup written: %s (%.1f MB)", target, size / (1024 * 1024))
prune_backups(db_path)
return target
+6
View File
@@ -76,6 +76,12 @@ _CLASS_RULES: tuple[tuple[str, tuple[str, ...]], ...] = (
"connection refused",
"connection reset",
"connection aborted",
# transformers' download-failure wording ("We couldn't connect to
# '<endpoint>' to load the files") — the #874 mirror-down class was
# journaled as UNKNOWN without these.
"couldn't connect to",
"could not connect to",
"max retries exceeded",
"timed out",
"timeout",
"name or service not known",
+252 -1
View File
@@ -21,6 +21,7 @@ import re
import sys
from pathlib import Path
from typing import Any, Optional
from urllib.parse import urlsplit
from core import error_docs_map
from core.logging_filter import REDACTED, _HF_TOKEN_RE
@@ -37,9 +38,166 @@ _HINTS: dict[str, str] = {
"APPIMAGE_WEBKIT_WHITESCREEN": "Launch with WEBKIT_DISABLE_DMABUF_RENDERER=1 set.",
"HF_AUTH_FAILED": "Set a valid HF_TOKEN in Settings → Hugging Face and retry.",
"PYANNOTE_LICENSE_REQUIRED": "Accept the pyannote model licenses on Hugging Face, then retry.",
"COMPUTE_TYPE_UNSUPPORTED": "Your GPU doesn't support float16 — OmniVoice retried on int8. If transcription still fails, set OMNIVOICE/ASR_COMPUTE_TYPE=int8 or use CPU.",
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
"OS_INVALID_ARGUMENT": "The OS rejected a file operation (Errno 22 / invalid argument) — in the transcribe path this is the temporary WAV write before ASR. It's almost always the temp directory: missing, read-only, on a full or removed drive, or blocked by antivirus. Check that your system TEMP/TMP folder exists and is writable and the drive has free space (add an OmniVoice antivirus exclusion if you use one), then retry.",
"SOCKS_PROXY_SUPPORT_MISSING": "A SOCKS proxy is configured in your environment (ALL_PROXY/HTTPS_PROXY=socks5://…) and the backend's HTTP client is missing SOCKS support. Newer OmniVoice builds ship SOCKS support (the socksio package) — update the app. If you still see this, unset ALL_PROXY/HTTPS_PROXY for OmniVoice, or run `uv pip install 'httpx[socks]'` in the backend venv, then restart.",
"SSL_HANDSHAKE_FAILURE": "A corporate or antivirus proxy is intercepting HTTPS traffic and re-signing certificates with its own CA — your OS trusts that CA, but Python's bundled certifi CA list doesn't, so the TLS handshake fails even though the connection reached the server. Newer OmniVoice builds trust the OS certificate store at startup (the truststore package), which should already fix this — update the app and retry. If you still see this, add an HTTPS-scanning exclusion for OmniVoice/Python in your antivirus, or ask IT for the proxy's CA bundle and set SSL_CERT_FILE to it, then restart.",
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
# HF_MIRROR_UNREACHABLE has a DYNAMIC hint (it names the configured mirror)
# — see hf_mirror_hint(); build_failure special-cases it.
}
# ── HF mirror connectivity (#874) ────────────────────────────────────────────
# When a non-default HF_ENDPOINT (a mirror, e.g. hf-mirror.com — set via
# Settings → Models → Hugging Face mirror) is configured and a model
# download/load fails with a connectivity error, the raw transformers/hf_hub
# message ("We couldn't connect to 'https://hf-mirror.com' to load the files…")
# gives the user no next step. This is the single classifier for that class,
# shared by every surface: build_failure() (model status, dub/task events),
# the global 500 handler (main.py — covers /generate and every other route
# that can leak a model-load error), and the model-install SSE
# (setup/download.py).
_OFFICIAL_HF_ENDPOINTS = {"https://huggingface.co", "https://hf.co"}
# Connectivity signatures across the layers an HF download failure surfaces
# from: transformers' wording, huggingface_hub errors, requests/urllib3, and
# raw socket/DNS failures (Linux/macOS/Windows variants).
_HF_CONNECTIVITY_SIGNATURES = (
"couldn't connect to", # transformers: "We couldn't connect to '<endpoint>' …"
"could not connect to",
"connection error", # huggingface_hub / requests
"connection refused",
"connection reset",
"connection aborted",
"max retries exceeded", # urllib3 via requests
"failed to establish a new connection",
"name or service not known", # Linux DNS
"temporary failure in name resolution",
"nodename nor servname provided", # macOS DNS
"getaddrinfo failed", # Windows DNS
"timed out",
"an error happened while trying to locate the file on the hub", # LocalEntryNotFoundError
"we cannot find the requested files", # LocalEntryNotFoundError
)
# The failure must also be Hugging-Face-shaped — the configured endpoint/host
# named in the message, or HF-download wording — so a random socket error
# (e.g. a local LLM provider being down) doesn't get the mirror hint just
# because a mirror happens to be configured.
_HF_CONTEXT_MARKERS = (
"huggingface",
"hf_hub",
"hf-hub",
"load the files", # transformers
"cached files", # transformers
"the requested files", # LocalEntryNotFoundError
"locate the file on the hub",
"snapshot_download",
)
def configured_hf_mirror() -> str:
"""The non-default Hugging Face endpoint (mirror) in effect, or "".
Same resolution the download paths use: ``HF_ENDPOINT`` env (what
Settings Models Hugging Face mirror persists via user_env, and what
the HF libraries read) with the ``hf_endpoint`` pref as fallback
(mirrors setup/download.py's ``prefs.resolve``). Never raises.
"""
ep = (os.environ.get("HF_ENDPOINT") or "").strip()
if not ep:
try:
from core import prefs
ep = str(prefs.get("hf_endpoint", "") or "").strip()
except Exception:
ep = ""
ep = ep.rstrip("/")
if not ep or ep.lower() in _OFFICIAL_HF_ENDPOINTS:
return ""
return ep
def hf_mirror_hint(reason: Optional[str]) -> str:
"""Actionable hint when ``reason`` is an HF-download connectivity failure
and a non-default mirror endpoint is configured; "" otherwise.
The hint names the configured mirror, says it may be down, points at the
setting (Settings Models Hugging Face mirror), suggests the official
endpoint when the model isn't cached yet, and notes the restart
requirement (HF reads HF_ENDPOINT at import time see the hf-mirror
endpoints in api/routers/settings.py). Never raises.
"""
mirror = configured_hf_mirror()
if not mirror:
return ""
low = (reason or "").lower()
if not any(sig in low for sig in _HF_CONNECTIVITY_SIGNATURES):
return ""
try:
host = (urlsplit(mirror).netloc or "").lower()
except Exception:
host = ""
if not (
mirror.lower() in low
or (host and host in low)
or any(m in low for m in _HF_CONTEXT_MARKERS)
):
return ""
return (
f"Your Hugging Face mirror is set to {mirror}, which couldn't be "
"reached — the mirror may be down or blocked on your network. If the "
'model isn\'t in your local cache yet, switch to "Hugging Face '
'(official)" in Settings → Models → Hugging Face mirror (or wait for '
"the mirror to recover), then restart OmniVoice — the mirror setting "
"is applied when the app starts."
)
def append_hf_mirror_hint(text: str) -> str:
"""``"{text}{hint}"`` when the mirror-connectivity class applies;
``text`` unchanged otherwise. For surfaces that hand a raw error string to
the UI (the global 500 handler, the model-install SSE). Never raises."""
try:
hint = hf_mirror_hint(text)
except Exception:
return text
return f"{text}{hint}" if hint else text
# Classes whose hint is safe to attach on the CONTEXT-FREE surfaces (the
# global 500 handler in main.py, the model-install SSE in setup/download.py),
# where all we have is a raw error string with no stage. Only classes whose
# classify() trigger is unmistakable belong here — e.g. VIDEO_DOWNLOAD_NETWORK
# must NOT be added: its bare "timed out" trigger would stamp a "video server"
# hint on a model-load timeout that leaks through the 500 handler.
_CONTEXT_FREE_HINT_CLASSES = frozenset({
"SOCKS_PROXY_SUPPORT_MISSING",
"SSL_HANDSHAKE_FAILURE",
})
def append_hint(text: str) -> str:
"""``"{text}{hint}"`` for raw-string surfaces (the global 500 handler,
the model-install SSE): the dynamic mirror hint (#874) when that class
applies, else a context-free static class hint (#959). ``text`` unchanged
otherwise a no-op for every other error. Never raises."""
try:
hint = hf_mirror_hint(text)
if not hint:
topic = classify(text)
if topic in _CONTEXT_FREE_HINT_CLASSES:
hint = _HINTS.get(topic, "")
except Exception:
return text
return f"{text}{hint}" if hint else text
def classify(reason: str) -> str:
"""Map a failure reason to a docs-taxonomy key, or "" when unknown.
@@ -55,10 +213,100 @@ def classify(reason: str) -> str:
return "APPIMAGE_WEBKIT_WHITESCREEN"
if "pyannote" in low or ("gated" in low and "model" in low) or "accept the" in low:
return "PYANNOTE_LICENSE_REQUIRED"
# ASR robustness (#551 / #549): name the class so the no-segments toast is
# actionable. Place before the generic returns so a compute-type/transformers
# failure gets its hint rather than falling through to "".
if "compute type" in low or "efficient float16" in low:
return "COMPUTE_TYPE_UNSUPPORTED"
# #763: a bare OS-level EINVAL ("[Errno 22] Invalid argument") while writing
# the per-chunk temp WAV for transcription (tempfile.NamedTemporaryFile /
# soundfile.write on the system temp dir) used to collapse into a dead-end
# "produced no segments. [Errno 22] Invalid argument" toast with no next
# step. errno 22 is EINVAL on every platform; in this path it's almost always
# a temp dir that's missing, read-only, on a full/removed drive, or blocked
# by antivirus. Name the class so build_failure attaches an actionable hint
# instead of a raw errno. Matching the errno (not the generic "invalid
# argument" wording) keeps this from mislabelling unrelated failures; the
# transformers "errno 2" rule below is unaffected — it also requires the
# transformers + site-packages markers, which this signature lacks.
if "errno 22" in low:
return "OS_INVALID_ARGUMENT"
if (
"could not import module" in low
or "autofeatureextractor" in low
# A corrupted/incomplete transformers install: a model load lazily
# resolves a module file that's MISSING from site-packages (an
# interrupted `uv sync`, antivirus removal, or a partial update), e.g.
# `[Errno 2] No such file or directory:
# '.../site-packages/transformers/models/qwen3/modeling_qwen3.py'`.
# That's a FileNotFoundError, not an ImportError, so the matches above
# miss it and the user got a useless "try restarting". Substring-match
# the package + the missing-file signal (separately, so it works on both
# POSIX `/` and Windows `\` paths).
or (
("no such file" in low or "errno 2" in low)
and "transformers" in low
and "site-packages" in low
)
):
return "TRANSFORMERS_IMPORT"
# #959: httpx raises ImportError AT CLIENT CONSTRUCTION ("Using SOCKS
# proxy, but the 'socksio' package is not installed. Make sure to install
# httpx using `pip install httpx[socks]`.") when ALL_PROXY/HTTPS_PROXY is
# socks5:// and socksio isn't importable. It surfaced from
# huggingface_hub's get_session() inside model load — a bare 500 on
# /generate with no next step. Checked BEFORE the HF-auth/mirror rules so
# a message that also carries HF wording still names this class.
if "socks proxy" in low or "socksio" in low:
return "SOCKS_PROXY_SUPPORT_MISSING"
# #976: a TLS handshake failing AFTER the TCP connection succeeds — the
# signature of a corporate/antivirus proxy that TLS-inspects traffic and
# re-signs certificates with a CA the OS trusts but Python's bundled
# certifi list doesn't (a different failure mode from #984's TCP-level
# "can't reach the host at all"). Requires "ssl" plus a handshake/cert-
# verify marker so a generic connection error isn't mislabelled.
if "ssl" in low and (
"handshake" in low
or "certificate verify failed" in low
or "sslv3_alert" in low
or "sslcertverificationerror" in low
):
return "SSL_HANDSHAKE_FAILURE"
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
):
return "HF_AUTH_FAILED"
# #874: a model download that failed because the CONFIGURED HF mirror is
# unreachable. Env-aware by design — the class only exists when a
# non-default HF_ENDPOINT is configured. Checked BEFORE the video-download
# network class so a model download's "timed out"/"connection reset"
# names the mirror instead of the "video server".
if hf_mirror_hint(reason):
return "HF_MIRROR_UNREACHABLE"
# Video download (#554/#536): a non-downloadable URL shape vs a transient
# network drop — both previously surfaced as a bare yt-dlp string with no
# next step. UNSUPPORTED first (more specific) so "Unable to download video:
# Broken pipe" still classifies as a network blip.
if "unsupported url" in low or "no video formats" in low or "is not a valid url" in low:
return "UNSUPPORTED_VIDEO_URL"
if (
"broken pipe" in low
or "connection reset" in low
or "unable to download video" in low
or "remote end closed" in low
or "timed out" in low
):
return "VIDEO_DOWNLOAD_NETWORK"
# A relocated/corrupted venv whose interpreter can't bootstrap its stdlib —
# the Rust self-heal rebuilds it; this names the class for the toast.
if "no module named 'encodings'" in low:
return "BROKEN_VENV"
# #564: the interpreter starts fine but the backend can't import its OWN
# `omnivoice` package (a venv missing the editable install). Same self-heal
# class — Clean & Retry / the bootstrap repair rebuilds it. The trailing
# quote keeps a legitimately-named `omnivoice_*` helper from matching.
if "no module named 'omnivoice'" in low:
return "BROKEN_VENV"
return ""
@@ -154,12 +402,15 @@ def build_failure(
reason = sanitize(raw) or error_class
docs_topic = classify(raw)
# HF_MIRROR_UNREACHABLE's hint is dynamic (it names the configured mirror)
# so it can't live in the static _HINTS table.
hint = hf_mirror_hint(raw) if docs_topic == "HF_MIRROR_UNREACHABLE" else _HINTS.get(docs_topic, "")
fields: dict[str, Any] = {
"reason": reason,
"error": reason, # backward-compat mirror for older frontends
"error_class": error_class,
"stage": stage,
"hint": _HINTS.get(docs_topic, ""),
"hint": hint,
"docs_topic": docs_topic,
"docs_url": error_docs_map.ERROR_DOCS.get(docs_topic, ""),
"detail": sanitize(raw),
+77
View File
@@ -0,0 +1,77 @@
"""Resolve the project's own ``omnivoice`` package from source when the venv's
editable install is missing (#564).
``omnivoice`` is normally an editable install in the backend venv. An interrupted
or offline ``uv sync`` can install dependencies yet never lay the editable record
(``_editable_impl_omnivoice.pth``), or an antivirus quarantine can remove it
leaving a venv that starts uvicorn but cannot ``import omnivoice``, so it boots
fine and only fails at the first model call (``No module named 'omnivoice'``).
The desktop layout always copies ``omnivoice/`` next to ``backend/``, so we fall
back to importing it from there. The bootstrap now also gates on omnivoice being
importable (re-syncing to re-lay the editable install), but this keeps the
backend resilient even when that repair hasn't run yet.
"""
import os
import sys
def find_omnivoice_source_root(candidates):
"""Return the first candidate dir holding ``omnivoice/__init__.py``, else None."""
for root in candidates:
if root and os.path.isfile(os.path.join(root, "omnivoice", "__init__.py")):
return root
return None
def _candidate_roots(backend_dir):
"""Source roots to probe, most-specific first.
``OMNIVOICE_PROJECT_ROOT`` lets the launcher point at the staged project dir
explicitly; otherwise the desktop layout puts ``omnivoice/`` beside
``backend/`` (parent of ``backend_dir``).
"""
roots = []
env = os.environ.get("OMNIVOICE_PROJECT_ROOT")
if env:
roots.append(env)
roots.append(os.path.dirname(os.path.abspath(backend_dir)))
return roots
def _already_importable():
import importlib.util
try:
return importlib.util.find_spec("omnivoice") is not None
except (ImportError, ValueError):
# A half-laid spec (e.g. a stale .pth pointing at a deleted dir) raises
# rather than returning None — treat it as "not importable" so we fall
# back to the on-disk source.
return False
def ensure_omnivoice_importable(backend_dir, logger=None):
"""Make ``import omnivoice`` work, falling back to the sibling source tree.
No-op when the editable/site-packages install already resolves it. Otherwise
appends the first source root containing ``omnivoice/`` to ``sys.path``
(appended, never inserted, so a real install keeps precedence). Returns the
root that was added, or ``None`` if none was needed or found.
"""
if _already_importable():
return None
root = find_omnivoice_source_root(_candidate_roots(backend_dir))
if root and root not in sys.path:
sys.path.append(root)
if logger:
logger.warning(
"omnivoice not importable from the venv (missing/broken editable "
"install) — resolving it from source at %s (#564)", root,
)
elif logger and root is None:
logger.error(
"omnivoice is not importable and no source tree was found next to "
"%s — the install is incomplete; relaunch to let the bootstrap "
"repair the venv (#564)", backend_dir,
)
return root
+8 -2
View File
@@ -61,9 +61,15 @@ def seed_sample_project():
if count > 0:
return # Not first run — skip
# Check if demo audio exists
# The demo clip is committed at backend/assets/samples/demo_voice.wav and
# bundled with the app (#621). If it's somehow absent (e.g. a partial
# checkout), skip the seed gracefully rather than seeding a profile that
# points at a missing file — run scripts/build_demos.sh to regenerate it.
if not os.path.isfile(_DEMO_AUDIO):
logger.warning("Demo audio not found at %s — skipping onboarding seed", _DEMO_AUDIO)
logger.warning(
"Demo audio not found at %s — skipping onboarding seed "
"(regenerate with scripts/build_demos.sh)", _DEMO_AUDIO,
)
return
# Copy demo audio to voices directory
+55 -7
View File
@@ -36,18 +36,39 @@ _TOKEN_PATTERNS = (
re.compile(r"github_pat_[A-Za-z0-9_]{20,}"), # GitHub fine-grained PAT
re.compile(r"gh[pousr]_[A-Za-z0-9]{30,}"), # GitHub classic tokens
re.compile(r"sk-[A-Za-z0-9_\-]{20,}"), # OpenAI-style API keys
# A backend error can carry a secret from *any* provider (the LLM-providers
# feature ships a dozen), so match the common credential shapes too, not
# just the four vendors above — a leaked key in a public issue is real harm.
re.compile(r"eyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{6,}"), # JWT (Bearer)
re.compile(r"AIza[0-9A-Za-z_\-]{35}"), # Google API key
re.compile(r"xox[baprs]-[A-Za-z0-9\-]{10,}"), # Slack token
re.compile(r"AKIA[0-9A-Z]{16}"), # AWS access key id
re.compile(r"(?i)bearer\s+[A-Za-z0-9._\-]{16,}"), # opaque bearer tokens
)
# Secrets carried in a URL query string (`?token=…`, `&api_key=…`). Redact the
# VALUE while keeping the param name + separator so the URL stays legible. Bare
# `key=` is intentionally excluded — too common in non-secret text; shaped keys
# are already caught above and named env vars by the sweep below.
_URL_SECRET_RE = re.compile(
r"((?:access[_-]?token|api[_-]?key|apikey|auth[_-]?token|token|secret|password|passwd|pwd)=)"
r"([^&\s\"'#]{6,})",
re.IGNORECASE,
)
# Home-directory shapes for all three supported platforms. Matched
# pattern-wise (not just this machine's $HOME) so paths quoted from a
# user's pasted log on another OS get cleaned too.
# IGNORECASE because Windows is case-insensitive and tools routinely emit the
# lowercase `c:\users\<name>` form, which the CLAUDE.md redaction spec still
# requires to become `~`. `Users`/`users`, `Home`/`home` all match.
_HOME_PATTERNS = (
# Windows-with-forward-slashes must run BEFORE the bare macOS shape, or
# `/Users/<name>` inside `C:/Users/<name>` gets eaten first, leaving `C:~`.
re.compile(r"[A-Za-z]:/Users/[^/\s\"']+"), # Windows, forward slashes (file URLs, normalized traces)
re.compile(r"/Users/[^/\s\"']+"), # macOS
re.compile(r"/home/[^/\s\"']+"), # Linux
re.compile(r"[A-Za-z]:\\Users\\[^\\\s\"']+"), # Windows, backslashes
re.compile(r"[A-Za-z]:/Users/[^/\s\"']+", re.IGNORECASE), # Windows, forward slashes
re.compile(r"/Users/[^/\s\"']+", re.IGNORECASE), # macOS
re.compile(r"/home/[^/\s\"']+", re.IGNORECASE), # Linux
re.compile(r"[A-Za-z]:\\Users\\[^\\\s\"']+", re.IGNORECASE), # Windows, backslashes
)
# Values shorter than this are too entropy-poor to be real secrets and too
@@ -85,19 +106,25 @@ def scrub_text(text: str | None) -> str:
except Exception:
pass
# 2. Credential-shaped substrings.
# 2. Credential-shaped substrings + URL query secrets.
for pat in _TOKEN_PATTERNS:
try:
s = pat.sub(REDACTED, s)
except Exception:
pass
try:
s = _URL_SECRET_RE.sub(lambda m: m.group(1) + REDACTED, s)
except Exception:
pass
# 3. This process's real home dir (covers symlinked/nonstandard homes
# the generic patterns miss), then the per-OS shapes.
# the generic patterns miss), then the per-OS shapes. Boundary-aware so
# a home of `/Users/john` doesn't rewrite `/Users/johnny` to `~ny`
# (leaking the fragment + mangling the path).
try:
home = os.path.expanduser("~")
if home and home not in ("/", "~"):
s = s.replace(home, "~")
s = re.sub(re.escape(home) + r"(?=[/\\\s\"']|$)", "~", s)
except Exception:
pass
for pat in _HOME_PATTERNS:
@@ -107,3 +134,24 @@ def scrub_text(text: str | None) -> str:
pass
return s
def scrub_provider_error(detail: object, api_key: str | None = None) -> str:
"""UI-safe text for an LLM/translation provider failure.
Some OpenAI-compatible providers echo the caller's key or a stable
``user_id`` back inside their error bodies, and a raw ``str(exc)`` on the
translate / glossary paths would surface that verbatim. This redacts the
exact resolved ``api_key`` first (in the provider-registry case it isn't a
shaped/known-env secret, so ``scrub_text`` alone can miss it) then runs the
generic secret + home-path scrub. Never raises scrubbing must not mask a
failure with a new one. Mirrors ``settings._scrub_llm_detail`` so every
surface redacts identically.
"""
s = str(detail if detail is not None else "")
try:
if api_key and api_key != "local" and len(api_key) >= _MIN_SECRET_LEN:
s = s.replace(api_key, REDACTED)
except Exception:
pass
return scrub_text(s)
+33 -4
View File
@@ -2,14 +2,43 @@
Read from the installed package metadata (driven by ``pyproject.toml``) so the
FastAPI/API version and exported-bundle metadata never drift to a stale literal
again (the prior "0.4.0" / "0.2.7" bug). Falls back to a literal only when
running from a raw source checkout that was never ``uv sync``'d.
the prior "0.4.0" / "0.2.7" bug, and the v0.3.6 desktop build that reported
"0.3.5" because the *frozen* backend couldn't read its own metadata.
Resolution order:
1. installed package metadata correct in any ``uv sync``'d env and, thanks
to ``copy_metadata('omnivoice')`` in ``backend.spec``, in the frozen build;
2. ``pyproject.toml`` walked up from this file correct for a raw source
checkout that was never installed;
3. ``_FALLBACK_VERSION`` a last resort, kept in lockstep with the four
version files by ``tests/test_app_version.py`` so it can never silently
drift again.
"""
from __future__ import annotations
import re
from importlib.metadata import PackageNotFoundError, version
from pathlib import Path
# Last-resort literal. Guarded by
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.3.15"
def _fallback_version() -> str:
"""Version for contexts where package metadata is unavailable."""
for parent in Path(__file__).resolve().parents:
pyproject = parent / "pyproject.toml"
if pyproject.is_file():
match = re.search(r'(?m)^version\s*=\s*"([^"]+)"', pyproject.read_text())
if match:
return match.group(1)
return _FALLBACK_VERSION
try:
APP_VERSION = version("omnivoice")
except PackageNotFoundError: # non-installed source checkout
APP_VERSION = "0.3.5"
except PackageNotFoundError: # frozen build w/o metadata, or non-installed checkout
APP_VERSION = _fallback_version()
+30 -2
View File
@@ -50,6 +50,22 @@ def _recv(stream):
return json.loads(bytes(body).decode("utf-8"))
# NOTE: keep this compute_type fallback in lockstep with
# services/asr_backend.py:_compute_type_candidates / _is_compute_type_error.
# This sidecar runs in a child proc with a clean import path, so we duplicate a
# tiny copy rather than cross-importing the heavy services package (#551).
def _ct_candidates(device):
override = os.environ.get("ASR_COMPUTE_TYPE")
if override:
return [override]
return ["float16", "int8_float16", "int8"] if device == "cuda" else ["int8", "float32"]
def _is_ct_error(msg):
low = msg.lower()
return "compute type" in low or "efficient float16" in low
def _get_model():
global _model
if _model is None:
@@ -60,8 +76,20 @@ def _get_model():
device = "cuda" if torch.cuda.is_available() else "cpu"
except Exception:
device = "cpu"
compute = "float16" if device == "cuda" else "int8"
_model = WhisperModel(name, device=device, compute_type=compute)
# Degrade fp16 → int8 rather than crash on GPUs without efficient fp16
# (older Maxwell/Pascal, GTX 16xx, CTranslate2/cuDNN mismatch) (#551).
last_err = None
for compute in _ct_candidates(device):
try:
_model = WhisperModel(name, device=device, compute_type=compute)
break
except (ValueError, RuntimeError) as e:
last_err = e
if _is_ct_error(str(e)):
continue
raise
else:
raise last_err
return _model
+134
View File
@@ -0,0 +1,134 @@
"""Confucius4-TTS sidecar package (issue #590).
Confucius4-TTS (netease-youdao) is an LLM-based multilingual / cross-lingual
zero-shot voice-cloning TTS: 14 languages, **no reference transcript required**,
cross-lingual voice transfer, Apache-2.0 (https://github.com/netease-youdao/Confucius4-TTS).
Like IndexTTS / MOSS-TTS-v1.5 / dots.tts it runs in its **own subprocess venv**
(upstream: Python 3.10 + CUDA 12.6 + its own deps), isolated from the OmniVoice
parent. It is **opt-in** selected in the engine picker and enabled only when
the user points ``OMNIVOICE_CONFUCIUS4_TTS_DIR`` at a clone so it can never
become a broken default on any platform (the strict default-parity rule).
Status (#590): **validated end-to-end** (2026-07-02, Apple Silicon, CPU) — the
synthesis API (``confuciustts.cli.inference.ConfuciusTTS``
``.generate(text, lang, prompt_wav)`` tensor, ``model.sample_rate``) produced
audible speech at 22 050 Hz; the sidecar's pure logic is unit-tested
(``tests/test_confucius4_sidecar.py``). CPU inference is slow (~17× realtime),
so CUDA is the recommended path. Gated off by default, so this affects no one
until they opt in.
Three entry points: ``Confucius4Backend`` (this module), ``main.py`` (the sidecar,
runs under the Confucius4 venv never imported by the parent), and
``bootstrap.py`` (venv probe + lazy bootstrap).
"""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from services.subprocess_backend import SubprocessBackend
if TYPE_CHECKING:
import torch # noqa: F401
logger = logging.getLogger("omnivoice.confucius4")
class Confucius4Backend(SubprocessBackend):
"""Confucius4-TTS (netease-youdao) — LLM-based, 14 langs, zero-shot clone.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv. First synthesize cold-loads the checkpoint; subsequent calls
reuse the process.
Installation::
git clone https://github.com/netease-youdao/Confucius4-TTS.git
cd Confucius4-TTS
uv venv --python 3.10 && uv pip install -r requirements.txt
(Upstream ships no pyproject.toml/setup.py, so there is nothing to
``pip install -e`` the sidecar sys.path-inserts the clone instead.)
Then set ``OMNIVOICE_CONFUCIUS4_TTS_DIR`` to the clone root and restart.
License: Apache-2.0. CUDA recommended; CPU validated but ~17× realtime.
"""
id = "confucius4-tts"
display_name = (
"Confucius4-TTS (LLM, 14 langs, cross-lingual zero-shot clone, CUDA/CPU, Apache-2.0)"
)
supports_voice_design = False # timbre comes from a reference clip
# Upstream vocoder rate (config target_sample_rate) — confirmed 22 050 Hz by
# a live run (2026-07-02); still re-read from the sidecar's ready/audio frames.
_DEFAULT_SAMPLE_RATE = 22050
# CUDA fast path + CPU fallback, both exercised (CPU end-to-end validated).
# No MPS claim — upstream has no Metal path.
gpu_compat = ("cuda", "cpu")
@classmethod
def is_available(cls) -> tuple[bool, str]:
# Verify the venv on disk only — do NOT import the engine here (separate
# interpreter). A real health-check runs on the user's "Test engine"
# action in Settings.
from engines.confucius4.bootstrap import (
CONFUCIUS4_SIDECAR_SCRIPT,
is_confucius4_installed,
)
if not is_confucius4_installed():
return False, (
"Confucius4-TTS venv not found. Set OMNIVOICE_CONFUCIUS4_TTS_DIR "
"to your Confucius4-TTS clone (the directory containing "
"requirements.txt) and restart OmniVoice. CUDA GPU recommended "
"(CPU works but is slow). See docs/engines/confucius4-tts.md."
)
if not CONFUCIUS4_SIDECAR_SCRIPT.exists():
return False, (
"Confucius4-TTS sidecar script missing at "
f"{CONFUCIUS4_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok"
@classmethod
def venv_python(cls):
from engines.confucius4.bootstrap import resolve_confucius4_venv
return resolve_confucius4_venv()
@classmethod
def sidecar_script(cls):
from engines.confucius4.bootstrap import CONFUCIUS4_SIDECAR_SCRIPT
return CONFUCIUS4_SIDECAR_SCRIPT
@property
def sample_rate(self) -> int:
return self._DEFAULT_SAMPLE_RATE
@property
def supported_languages(self) -> list[str]:
# 14 languages with the caller's language passed through at synthesize
# time; "multi" on the protocol surface.
return ["multi"]
def generate(self, text: str, **kw) -> "torch.Tensor":
"""Synthesize one utterance through the Confucius4 sidecar.
kwargs honored:
* ``ref_audio`` reference clip path ``prompt_wav`` (zero-shot
cloning). Optional but recommended for a specific voice.
* ``language`` ISO code / name ``lang`` (cross-lingual transfer).
* ``ref_text`` is intentionally ignored Confucius4 is unconstrained
cloning (no reference transcript needed).
Returns a tensor of shape (1, n_samples) at :attr:`sample_rate`.
"""
forwarded: dict = {}
ref_audio = kw.get("ref_audio")
if ref_audio:
forwarded["ref_audio"] = ref_audio
language = kw.get("language")
if language:
forwarded["language"] = str(language)
return super().generate(text, **forwarded)
__all__ = ["Confucius4Backend"]
+217
View File
@@ -0,0 +1,217 @@
"""Confucius4-TTS venv probe + lazy bootstrap (issue #590).
Confucius4-TTS (netease-youdao) is an LLM-based multilingual zero-shot cloning
TTS 14 languages, no reference transcript required, Apache-2.0. Like the other
heavyweight opt-in engines (IndexTTS / MOSS-TTS-v1.5 / dots.tts) it runs in its
**own subprocess venv**: upstream targets Python 3.10 + CUDA 12.6 with its own
dependency set, which we keep off the parent interpreter.
Probe order (existing power-user installs win zero migration):
1. ``${OMNIVOICE_CONFUCIUS4_TTS_DIR}/.venv/`` the user's clone-level venv.
2. ``backend/engines/confucius4/.venv/`` this package's own venv.
3. Bootstrap: ``uv venv`` then ``uv pip install -r <clone>/requirements.txt``
(+ ``uv pip install -e <clone>`` only if upstream ever ships packaging).
Validated end-to-end 2026-07-02 (Apple Silicon, CPU): upstream ships **no
pyproject.toml/setup.py**, so ``confuciustts`` is importable only with the
clone root on ``sys.path`` the import probe and the sidecar both handle
that. The engine is opt-in (env-dir gated) and never touched unless
``OMNIVOICE_CONFUCIUS4_TTS_DIR`` is set, so this can't affect the default
install on any platform.
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger("omnivoice.confucius4.bootstrap")
#: Absolute path to the sidecar entrypoint.
CONFUCIUS4_SIDECAR_SCRIPT: Path = Path(__file__).parent / "main.py"
#: This package's owned venv (Probe 2).
_ENGINES_VENV_DIR: Path = Path(__file__).parent / ".venv"
#: Env var pointing at the user's Confucius4-TTS clone root.
_CLONE_DIR_ENV: str = "OMNIVOICE_CONFUCIUS4_TTS_DIR"
#: The package importable from the clone (verify against upstream).
_IMPORT_PROBE = "confuciustts"
_resolved_python: Optional[Path] = None
_IMPORT_PROBE_TIMEOUT_S = 15
_UV_VENV_TIMEOUT_S = 120
_UV_PIP_INSTALL_TIMEOUT_S = 1800
def invalidate() -> None:
"""Clear the resolved-python cache. Tests call this between scenarios."""
global _resolved_python
_resolved_python = None
def is_confucius4_installed() -> bool:
"""Cheap file-existence check for a usable venv (no subprocess spawn)."""
return any(cand.is_file() for cand in _probe_paths())
def resolve_confucius4_venv() -> Path:
"""Resolve the sidecar's Python interpreter (probe order in the docstring).
Memoised. Raises :exc:`RuntimeError` if none can be located and bootstrap
is unavailable."""
global _resolved_python
if _resolved_python is not None:
return _resolved_python
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
cand = _venv_python_path(Path(clone_dir) / ".venv")
if cand.is_file() and _venv_can_import(cand):
logger.info("Confucius4 venv resolved from %s: %s", _CLONE_DIR_ENV, cand)
_resolved_python = cand
return cand
cand = _venv_python_path(_ENGINES_VENV_DIR)
if cand.is_file() and _venv_can_import(cand):
logger.info("Confucius4 venv resolved from engines path: %s", cand)
_resolved_python = cand
return cand
if not clone_dir:
raise RuntimeError(
"Confucius4-TTS is not installed. Set the "
f"{_CLONE_DIR_ENV} environment variable to your Confucius4-TTS clone "
"(the directory that contains requirements.txt), then restart "
"OmniVoice. See docs/engines/confucius4-tts.md."
)
cand = _bootstrap_engines_venv(Path(clone_dir))
_resolved_python = cand
return cand
def _venv_python_path(venv_dir: Path) -> Path:
if sys.platform == "win32":
return venv_dir / "Scripts" / "python.exe"
return venv_dir / "bin" / "python"
def _probe_paths() -> list[Path]:
out: list[Path] = []
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
out.append(_venv_python_path(Path(clone_dir) / ".venv"))
out.append(_venv_python_path(_ENGINES_VENV_DIR))
return out
def _import_probe_code() -> str:
"""Probe snippet mirroring the sidecar's import semantics: upstream is not
pip-installable, so ``confuciustts`` resolves via the clone on sys.path."""
clone = os.environ.get(_CLONE_DIR_ENV, "")
if clone:
return f"import sys; sys.path.insert(0, {clone!r}); import {_IMPORT_PROBE}"
return f"import {_IMPORT_PROBE}"
def _venv_can_import(python_path: Path) -> bool:
"""Spawn the candidate python and verify ``import confuciustts`` works."""
try:
proc = subprocess.run(
[str(python_path), "-c", _import_probe_code()],
capture_output=True, timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
logger.debug("Confucius4 import probe failed for %s: %s", python_path, exc)
return False
if proc.returncode != 0:
logger.debug(
"Confucius4 import probe non-zero for %s: %s",
python_path, proc.stderr.decode("utf-8", errors="replace")[:200],
)
return False
return True
def _locate_uv() -> Optional[str]:
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
if bundled and Path(bundled).is_file():
return bundled
return shutil.which("uv")
def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"""Create engines/confucius4/.venv and install the user's clone."""
uv = _locate_uv()
if not uv:
raise RuntimeError(
"uv is required to bootstrap the Confucius4-TTS venv but was not "
"found on PATH (and OMNIVOICE_BUNDLED_UV was not set). Install uv "
"from https://docs.astral.sh/uv/ and re-launch OmniVoice."
)
logger.info(
"Bootstrapping Confucius4 venv at %s from %s (several minutes on first "
"launch)", _ENGINES_VENV_DIR, clone_dir,
)
try:
subprocess.run(
[uv, "venv", "--python", "3.10", str(_ENGINES_VENV_DIR)],
check=True, timeout=_UV_VENV_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
f"uv venv failed for Confucius4 bootstrap at {_ENGINES_VENV_DIR}: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
python_path = _venv_python_path(_ENGINES_VENV_DIR)
requirements = clone_dir / "requirements.txt"
try:
if requirements.is_file():
subprocess.run(
[uv, "pip", "install", "--python", str(python_path),
"-r", str(requirements)],
check=True, timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
# Editable install only if upstream ever ships packaging metadata —
# as of 2026-07 there is none, and `uv pip install -e` on a bare clone
# fails outright. Import resolution is handled via sys.path instead.
if (clone_dir / "pyproject.toml").is_file() or (clone_dir / "setup.py").is_file():
subprocess.run(
[uv, "pip", "install", "--python", str(python_path), "-e", str(clone_dir)],
check=True, timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install failed during Confucius4 bootstrap "
f"({clone_dir}): "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}. "
"See docs/engines/confucius4-tts.md."
) from exc
if not _venv_can_import(python_path):
raise RuntimeError(
f"Confucius4 bootstrap completed but `import {_IMPORT_PROBE}` still "
f"fails from {python_path}. Verify {clone_dir} is a valid clone. "
"See docs/engines/confucius4-tts.md."
)
logger.info("Confucius4 venv bootstrap successful: %s", python_path)
return python_path
__all__ = [
"CONFUCIUS4_SIDECAR_SCRIPT",
"invalidate",
"is_confucius4_installed",
"resolve_confucius4_venv",
]
+216
View File
@@ -0,0 +1,216 @@
"""Confucius4-TTS sidecar entry point (issue #590).
Runs inside ``engines/confucius4/.venv`` (or the user's
``${OMNIVOICE_CONFUCIUS4_TTS_DIR}/.venv``), isolated from the OmniVoice parent.
Same isolation rationale as the IndexTTS / MOSS-TTS-v1.5 / dots.tts sidecars.
Stdlib-only at import time; ``confuciustts`` + torch are imported lazily on the
first synthesize op so the ``ready`` frame fits inside the parent's 30 s spawn
handshake.
Wire protocol length-prefixed JSON over stdin/stdout, byte-identical to
``backend/services/subprocess_backend.py``::
[ 4-byte big-endian uint32 length ][ N bytes UTF-8 JSON ]
Op flow: ready ping/pong synthesize ( progress, audio) shutdown.
Status (#590): the model API below
(``confuciustts.cli.inference.ConfuciusTTS(config_path=, device=)`` and
``model.generate(text=, lang=, prompt_wav=)`` audio tensor, ``model.sample_rate``)
is **validated end-to-end** (2026-07-02, Apple Silicon, CPU): live generate()
produced audible speech at 22 050 Hz. This sidecar's pure logic is unit-tested
in ``tests/test_confucius4_sidecar.py``. Opt-in, so it affects no one until
enabled.
Restrictions: NO imports from OmniVoice parent code. NO logging of os.environ.
"""
from __future__ import annotations
import base64
import json
import os
import struct
import sys
import traceback
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: Upstream BigVGAN vocoder rate — ``target_sample_rate: 22050`` in
#: ``config/inference_config.yaml``, confirmed by a live end-to-end run
#: (2026-07-02). The real value is still re-read from ``model.sample_rate``
#: on each generate() so a future upstream change can't corrupt audio.
CONFUCIUS_SAMPLE_RATE = 22050
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
def _recv(stream):
header = stream.read(4)
if len(header) < 4:
return None # EOF
(n,) = struct.unpack("!I", header)
if n > MAX_FRAME_BYTES:
raise IOError(f"frame too large: {n}")
body = bytearray()
while len(body) < n:
chunk = stream.read(n - len(body))
if not chunk:
raise IOError("short read")
body.extend(chunk)
return json.loads(bytes(body).decode("utf-8"))
def _measure_vram_mb() -> float:
try:
import torch
if torch.cuda.is_available():
return round(torch.cuda.memory_allocated() / (1024 ** 2), 1)
except Exception:
pass
return 0.0
_model = None
def _config_path() -> str:
"""Locate Confucius4's inference config (``config/inference_config.yaml``)
under the clone, or an explicit override."""
explicit = os.environ.get("OMNIVOICE_CONFUCIUS4_CONFIG")
if explicit:
return explicit
clone = os.environ.get("OMNIVOICE_CONFUCIUS4_TTS_DIR", "")
return os.path.join(clone, "config", "inference_config.yaml")
def _ensure_clone_on_sys_path() -> None:
"""Make ``import confuciustts`` resolve from the user's clone.
Upstream Confucius4-TTS is **not pip-installable** (no pyproject.toml /
setup.py as of 2026-07); its own ``example.py`` sys.path-inserts the repo
root instead. Mirror that here so the sidecar works from a plain
``uv pip install -r requirements.txt`` venv. Inserted at position 0 so the
clone the user pointed at always wins over any stale installed copy.
"""
clone = os.environ.get("OMNIVOICE_CONFUCIUS4_TTS_DIR", "")
if clone and clone not in sys.path:
sys.path.insert(0, clone)
def _load_model(stdout):
"""Cold-construct the Confucius4 model (CUDA, else CPU — both validated)."""
global _model
if _model is not None:
return _model
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
_ensure_clone_on_sys_path()
import torch
from confuciustts.cli.inference import ConfuciusTTS # type: ignore[import-not-found]
device = "cuda" if torch.cuda.is_available() else "cpu"
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 50})
_model = ConfuciusTTS(config_path=_config_path(), device=device)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _model
def _tensor_to_pcm_b64(audio, sample_rate: int) -> tuple[str, int, int]:
import numpy as np
arr = audio.detach().to("cpu").float().numpy() if hasattr(audio, "detach") else np.asarray(audio)
arr = np.asarray(arr, dtype=np.float32).squeeze()
if arr.ndim > 1:
arr = arr.mean(axis=0)
arr = np.clip(arr, -1.0, 1.0)
pcm = (arr * 32767.0).astype(np.int16).tobytes()
return base64.b64encode(pcm).decode("ascii"), int(sample_rate), int(arr.shape[0])
def _normalize_language(raw):
"""Confucius4 expects an ISO-ish language code (e.g. 'en', 'zh'). Empty /
'auto' 'en' as a safe default (the API requires a lang)."""
if not raw or not isinstance(raw, str):
return "en"
s = raw.strip().lower()
if not s or s == "auto":
return "en"
return s[:2] if (len(s) >= 2 and s[:2].isalpha()) else s
def _handle_synthesize(msg: dict, stdout) -> None:
text = msg.get("text")
if not text or not isinstance(text, str):
raise ValueError("synthesize: missing or non-string 'text'")
model = _load_model(stdout)
gen_kwargs: dict = {"text": text, "lang": _normalize_language(msg.get("language"))}
ref_audio = msg.get("ref_audio")
if ref_audio:
gen_kwargs["prompt_wav"] = ref_audio
audio = model.generate(**gen_kwargs)
sample_rate = int(getattr(model, "sample_rate", CONFUCIUS_SAMPLE_RATE))
pcm_b64, sr, n_samples = _tensor_to_pcm_b64(audio, sample_rate)
_send(stdout, {
"op": "audio",
"audio_pcm_b64": pcm_b64,
"sample_rate": sr,
"n_samples": n_samples,
})
def main() -> int:
stdin = sys.stdin.buffer
stdout = sys.stdout.buffer
_send(stdout, {
"op": "ready",
"engine": "confucius4-tts",
"sample_rate": CONFUCIUS_SAMPLE_RATE,
})
while True:
try:
msg = _recv(stdin)
except Exception as exc:
_send(stdout, {
"op": "error", "stage": "recv",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
return 1
if msg is None:
return 0
op = msg.get("op") if isinstance(msg, dict) else None
try:
if op == "ping":
_send(stdout, {"op": "pong", "vram_mb": _measure_vram_mb()})
elif op == "synthesize":
_handle_synthesize(msg, stdout)
elif op == "shutdown":
return 0
else:
_send(stdout, {"op": "error", "stage": "dispatch",
"message": f"unknown op: {op!r}"})
except Exception as exc:
_send(stdout, {
"op": "error", "stage": op or "unknown",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
if __name__ == "__main__":
sys.exit(main())
+187
View File
@@ -0,0 +1,187 @@
"""dots.tts sidecar package (issue #498).
dots.tts is rednote-hilab's 2B fully-continuous autoregressive TTS — widely
cited as among the strongest open zero-shot voice-cloning models. 24
languages, 48 kHz output, Apache-2.0 (code + checkpoints).
It runs in its own subprocess **and its own venv**, isolated from the
OmniVoice parent, for the same ``transformers`` reason as IndexTTS and
MOSS-TTS-v1.5: dots.tts pins ``transformers==4.57.0`` (verified against
``constraints/recommended.txt``), while OmniVoice pins
``transformers>=5.3.0``. The two cannot share one interpreter.
Cross-platform honesty (the strict default-parity rule): dots.tts's
upstream package declares **Linux + macOS** classifiers only **no
Windows** and its device code is **CUDA-or-CPU with no MPS branch**
(verified in ``runtime.py``). So:
* It is **opt-in** (engine-picker selection + a user-provided clone),
never a default so it never becomes a broken default on any platform.
* ``is_available()`` returns ``False`` with a clear reason on **Windows**
rather than offering an engine that can't run there. Windows users are
pointed at WSL2 / a Linux or macOS host.
* ``gpu_compat = ("cuda", "cpu")`` no MPS claim. On Apple Silicon the
upstream package runs on CPU (slow but correct); the faster MLX path is
a community port we deliberately don't auto-wire here.
Three public entry points: ``DotsTTSBackend`` (this module), ``main.py``
(sidecar, runs under dots.tts's ``transformers==4.57`` venv — never imported
by the parent), and ``bootstrap.py`` (venv probe + lazy bootstrap).
"""
from __future__ import annotations
import logging
import sys
from typing import TYPE_CHECKING
from services.subprocess_backend import SubprocessBackend
if TYPE_CHECKING:
import torch # noqa: F401
logger = logging.getLogger("omnivoice.dots_tts")
class DotsTTSBackend(SubprocessBackend):
"""dots.tts (rednote-hilab) — 2B, 24 langs, zero-shot clone, CUDA/CPU.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv (``transformers==4.57.0``). First synthesize cold-loads
the ~9 GB checkpoint (bf16 on CUDA); subsequent calls reuse the process.
Installation (OmniVoice prefers a user's existing ``${DIR}/.venv``)::
git clone https://github.com/rednote-hilab/dots.tts.git
cd dots.tts
uv venv && uv pip install -e . -c constraints/recommended.txt
Set ``OMNIVOICE_DOTS_TTS_DIR`` to the clone root. OmniVoice creates
``backend/engines/dots_tts/.venv`` lazily on first launch if no venv
exists yet; the user's existing ``${DIR}/.venv`` is preferred if present.
Best cloning quality uses the ``dots.tts-soar`` checkpoint (the default)
and BOTH a reference clip and its exact transcript (continuation
cloning). License: Apache-2.0.
"""
id = "dots-tts"
display_name = (
"dots.tts (2B, 24 langs, zero-shot clone, CUDA/CPU, 48 kHz, Apache-2.0)"
)
supports_voice_design = False # requires ref audio for timbre cloning
# dots.tts emits 48 kHz (verified via checkpoint vocoder.sample_rate).
_DEFAULT_SAMPLE_RATE = 48000
# CUDA + CPU only; no MPS branch in upstream runtime.py.
gpu_compat = ("cuda", "cpu")
# ── availability ───────────────────────────────────────────────────────
@classmethod
def is_available(cls) -> tuple[bool, str]:
# Cross-platform parity: dots.tts upstream is Linux/macOS-only (no
# Windows classifier, no Windows install path). Refuse cleanly on
# Windows instead of advertising an engine that can't run.
if sys.platform == "win32":
return False, (
"dots.tts is not supported on Windows — upstream targets "
"Linux and macOS only. Run OmniVoice under WSL2, or use a "
"Linux/macOS host. See docs/engines/dots-tts.md."
)
# Do NOT import dots_tts here: it pins transformers==4.57, which can't
# coexist with the parent's transformers>=5.3 in one interpreter —
# the reason for the subprocess isolation. Verify the venv on disk
# only; a real health-check is gated on the user's "Test engine"
# action in Settings.
from engines.dots_tts.bootstrap import (
DOTS_TTS_SIDECAR_SCRIPT,
is_dots_tts_installed,
)
if not is_dots_tts_installed():
return False, (
"dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR to your "
"dots.tts clone (the directory containing pyproject.toml) and "
"restart OmniVoice. CUDA or CPU only (no MPS). See "
"docs/engines/dots-tts.md for the full install walk-through."
)
if not DOTS_TTS_SIDECAR_SCRIPT.exists():
return False, (
"dots.tts sidecar script missing at "
f"{DOTS_TTS_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok (CUDA when present, else CPU)"
@classmethod
def venv_python(cls):
from engines.dots_tts.bootstrap import resolve_dots_tts_venv
return resolve_dots_tts_venv()
@classmethod
def sidecar_script(cls):
from engines.dots_tts.bootstrap import DOTS_TTS_SIDECAR_SCRIPT
return DOTS_TTS_SIDECAR_SCRIPT
# ── TTSBackend protocol ────────────────────────────────────────────────
@property
def sample_rate(self) -> int:
return self._DEFAULT_SAMPLE_RATE
@property
def supported_languages(self) -> list[str]:
# 24 languages with auto-detect; expose "multi" on the protocol
# surface and translate the caller's language at synthesize time.
return ["multi"]
# ── generate (parent-side arbitration) ─────────────────────────────────
def generate(self, text: str, **kw) -> "torch.Tensor":
"""Synthesize one utterance through the dots.tts sidecar.
kwargs honored:
* ``ref_audio`` reference clip path ``prompt_audio_path``
(zero-shot cloning). Optional.
* ``ref_text`` the reference transcript ``prompt_text``.
Best cloning fidelity ("continuation"). Upstream
REQUIRES ``prompt_audio_path`` when ``prompt_text``
is set, so we drop a stray ref_text with no
ref_audio rather than let the sidecar raise.
* ``language`` ISO code / name / None (auto-detect).
* ``num_step`` flow-matching steps ``num_steps`` (default 10;
use 4 for the ``dots.tts-mf`` checkpoint).
* ``guidance_scale`` CFG (default 1.2; >2 amplifies energy).
Returns a tensor of shape (1, n_samples) at :attr:`sample_rate`.
"""
forwarded: dict = {}
ref_audio = kw.get("ref_audio")
if ref_audio:
forwarded["ref_audio"] = ref_audio
ref_text = kw.get("ref_text")
if ref_text:
# continuation cloning — only valid alongside ref_audio.
forwarded["ref_text"] = ref_text
elif kw.get("ref_text"):
logger.info(
"dots-tts: ref_text supplied without ref_audio; ignoring "
"(upstream requires prompt_audio_path when prompt_text is set)."
)
language = kw.get("language")
if language:
forwarded["language"] = str(language)
# OmniVoice's generic num_step default is 16; dots.tts's own default
# is 10. Honor an explicit value, else use the dots-appropriate 10.
num_step = kw.get("num_step")
forwarded["num_steps"] = int(num_step) if num_step is not None else 10
# dots.tts's own CFG default is 1.2 (the generic 2.0 over-energises).
guidance = kw.get("guidance_scale")
forwarded["guidance_scale"] = float(guidance) if guidance is not None else 1.2
return super().generate(text, **forwarded)
__all__ = ["DotsTTSBackend"]
+226
View File
@@ -0,0 +1,226 @@
"""dots.tts venv probe + lazy bootstrap (issue #498).
Resolves which Python interpreter runs the dots.tts sidecar. Mirrors
``engines.indextts.bootstrap`` / ``engines.moss_tts_v15.bootstrap`` because
dots.tts has the same shape of problem: a hard ``transformers==4.57.0`` pin
that conflicts with the parent's ``transformers>=5.3`` — so it runs in its
own venv.
Probe order (priority existing power-user installs win, zero migration):
1. ``${OMNIVOICE_DOTS_TTS_DIR}/.venv/`` the user's clone-level venv.
2. ``backend/engines/dots_tts/.venv/`` this package's own venv.
3. Bootstrap: ``uv venv`` then ``uv pip install -e <clone> -c
<clone>/constraints/recommended.txt`` (the upstream-pinned stack:
torch==2.8.0, transformers==4.57.0, ).
Caching: memoised after first success. Tests reset via :func:`invalidate`.
Security: same posture as IndexTTS bootstrap never touches HF_TOKEN; the
sidecar's stderr is redacted by the parent's ``HFTokenRedactor``; the
editable install comes from a user-controlled clone they already trust.
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger("omnivoice.dots_tts.bootstrap")
#: Absolute path to the sidecar entrypoint.
DOTS_TTS_SIDECAR_SCRIPT: Path = Path(__file__).parent / "main.py"
#: This package's owned venv (Probe 2).
_ENGINES_VENV_DIR: Path = Path(__file__).parent / ".venv"
#: Env var pointing at the user's dots.tts clone root.
_CLONE_DIR_ENV: str = "OMNIVOICE_DOTS_TTS_DIR"
#: Per-process resolution cache. Cleared by :func:`invalidate` for tests.
_resolved_python: Optional[Path] = None
_IMPORT_PROBE_TIMEOUT_S = 15
_UV_VENV_TIMEOUT_S = 120
_UV_PIP_INSTALL_TIMEOUT_S = 1800
# ── public API ────────────────────────────────────────────────────────────
def invalidate() -> None:
"""Clear the resolved-python cache. Tests call this between scenarios."""
global _resolved_python
_resolved_python = None
def is_dots_tts_installed() -> bool:
"""Cheap file-existence check for a usable dots.tts venv. Does NOT spawn
the venv Python that's saved for :func:`resolve_dots_tts_venv`."""
for cand in _probe_paths():
if cand.is_file():
return True
return False
def resolve_dots_tts_venv() -> Path:
"""Resolve the sidecar's Python interpreter (probe order in the module
docstring). Memoised. Raises :exc:`RuntimeError` if none can be located
and the bootstrap path is unavailable."""
global _resolved_python
if _resolved_python is not None:
return _resolved_python
clone_dir = os.environ.get(_CLONE_DIR_ENV)
# Probe 1 — user's clone-level venv.
if clone_dir:
cand = _venv_python_path(Path(clone_dir) / ".venv")
if cand.is_file() and _venv_can_import_dots(cand):
logger.info(
"dots.tts venv resolved from %s: %s", _CLONE_DIR_ENV, cand,
)
_resolved_python = cand
return cand
# Probe 2 — this package's own venv.
cand = _venv_python_path(_ENGINES_VENV_DIR)
if cand.is_file() and _venv_can_import_dots(cand):
logger.info("dots.tts venv resolved from engines path: %s", cand)
_resolved_python = cand
return cand
# Probe 3 — bootstrap.
if not clone_dir:
raise RuntimeError(
"dots.tts is not installed. Set the "
f"{_CLONE_DIR_ENV} environment variable to your dots.tts clone "
"(the directory that contains pyproject.toml and constraints/), "
"then restart OmniVoice. See docs/engines/dots-tts.md for the "
"full install walk-through."
)
cand = _bootstrap_engines_venv(Path(clone_dir))
_resolved_python = cand
return cand
# ── internals ─────────────────────────────────────────────────────────────
def _venv_python_path(venv_dir: Path) -> Path:
if sys.platform == "win32":
return venv_dir / "Scripts" / "python.exe"
return venv_dir / "bin" / "python"
def _probe_paths() -> list[Path]:
out: list[Path] = []
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
out.append(_venv_python_path(Path(clone_dir) / ".venv"))
out.append(_venv_python_path(_ENGINES_VENV_DIR))
return out
def _venv_can_import_dots(python_path: Path) -> bool:
"""Spawn the candidate python and verify ``import dots_tts.runtime`` works.
Bounded by ``_IMPORT_PROBE_TIMEOUT_S``. False on any failure."""
try:
proc = subprocess.run(
[str(python_path), "-c", "import dots_tts.runtime"],
capture_output=True,
timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
logger.debug("dots.tts import probe failed for %s: %s", python_path, exc)
return False
if proc.returncode != 0:
logger.debug(
"dots.tts import probe non-zero for %s: %s",
python_path,
proc.stderr.decode("utf-8", errors="replace")[:200],
)
return False
return True
def _locate_uv() -> Optional[str]:
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
if bundled and Path(bundled).is_file():
return bundled
return shutil.which("uv")
def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"""Create engines/dots_tts/.venv and editable-install the user's clone
with the upstream constraints file."""
uv = _locate_uv()
if not uv:
raise RuntimeError(
"uv is required to bootstrap the dots.tts venv but was not found "
"on PATH (and OMNIVOICE_BUNDLED_UV was not set). Install uv from "
"https://docs.astral.sh/uv/ and re-launch OmniVoice, or set "
"OMNIVOICE_BUNDLED_UV to the absolute path of a uv binary."
)
logger.info(
"Bootstrapping dots.tts venv at %s from %s (this can take several "
"minutes on first launch)", _ENGINES_VENV_DIR, clone_dir,
)
try:
subprocess.run(
[uv, "venv", str(_ENGINES_VENV_DIR)],
check=True, timeout=_UV_VENV_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
f"uv venv failed for dots.tts bootstrap at {_ENGINES_VENV_DIR}: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
python_path = _venv_python_path(_ENGINES_VENV_DIR)
install_cmd = [
uv, "pip", "install",
"--python", str(python_path),
"-e", str(clone_dir),
]
# Apply the upstream pin set when it ships with the clone.
constraints = clone_dir / "constraints" / "recommended.txt"
if constraints.is_file():
install_cmd += ["-c", str(constraints)]
try:
subprocess.run(
install_cmd, check=True,
timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install -e failed during dots.tts bootstrap "
f"({clone_dir}): "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}. "
"See docs/engines/dots-tts.md."
) from exc
if not _venv_can_import_dots(python_path):
raise RuntimeError(
"dots.tts bootstrap completed but `import dots_tts.runtime` still "
f"fails from {python_path}. Verify that {clone_dir} is a valid "
"dots.tts clone. See docs/engines/dots-tts.md."
)
logger.info("dots.tts venv bootstrap successful: %s", python_path)
return python_path
__all__ = [
"DOTS_TTS_SIDECAR_SCRIPT",
"invalidate",
"is_dots_tts_installed",
"resolve_dots_tts_venv",
]
+255
View File
@@ -0,0 +1,255 @@
"""dots.tts sidecar entry point (issue #498).
Runs inside ``engines/dots_tts/.venv`` (or the user's existing
``${OMNIVOICE_DOTS_TTS_DIR}/.venv``) with ``transformers==4.57.0``, isolated
from the OmniVoice parent (``transformers>=5.3``). Same isolation rationale
as the IndexTTS / MOSS-TTS-v1.5 sidecars.
Stdlib-only at import time; ``dots_tts`` + torch are imported lazily on the
first synthesize op so the ``ready`` frame fits inside the parent's 30 s
spawn handshake even on a cold filesystem.
Wire protocol length-prefixed JSON over stdin/stdout, byte-identical to
``backend/services/subprocess_backend.py``::
[ 4-byte big-endian uint32 length ][ N bytes UTF-8 JSON ]
Op flow:
1. Sidecar -> parent: {"op": "ready", "engine": "dots-tts",
"sample_rate": 48000}
2. parent -> sidecar: {"op": "ping"} -> {"op": "pong", "vram_mb": N}
3. parent -> sidecar: {"op": "synthesize", "text": "...",
"ref_audio": "/path/ref.wav",
"ref_text": "transcript", "language": "EN",
"num_steps": 10, "guidance_scale": 1.2}
-> {"op": "progress", ...} (cold load) then
-> {"op": "audio", "audio_pcm_b64": "...", "sample_rate": 48000,
"n_samples": N}
4. parent -> sidecar: {"op": "shutdown"} -> exit 0
Restrictions: NO imports from OmniVoice parent code (different venv). NO
logging of ``os.environ`` contents. Single-frame DoS cap matches the
parent's ``MAX_FRAME_BYTES``.
"""
from __future__ import annotations
import base64
import json
import os
import struct
import sys
import traceback
# Mirrors backend/services/subprocess_backend.py::MAX_FRAME_BYTES.
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: dots.tts emits 48 kHz (checkpoint vocoder.sample_rate). Advertised in the
#: ready frame; the real value is re-read from each generate() result.
DOTS_SAMPLE_RATE = 48000
#: Default checkpoint. ``-soar`` is the best-cloning variant; ``-mf`` is the
#: fastest (use num_steps=4). Overridable for air-gapped / mirror installs.
_DEFAULT_REPO = "rednote-hilab/dots.tts-soar"
# ── wire protocol ─────────────────────────────────────────────────────────
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
def _recv(stream):
header = stream.read(4)
if len(header) < 4:
return None # EOF
(n,) = struct.unpack("!I", header)
if n > MAX_FRAME_BYTES:
raise IOError(f"frame too large: {n}")
body = bytearray()
while len(body) < n:
chunk = stream.read(n - len(body))
if not chunk:
raise IOError("short read")
body.extend(chunk)
return json.loads(bytes(body).decode("utf-8"))
def _measure_vram_mb() -> float:
"""This sidecar's own GPU memory in MB (MM2-08). 0 on CPU. Never raises."""
try:
import torch
if torch.cuda.is_available():
return round(torch.cuda.memory_allocated() / (1024 ** 2), 1)
except Exception:
pass
return 0.0
# ── model loading (lazy, on first synthesize) ─────────────────────────────
# Module-level singleton — (runtime,). Device is auto-selected inside the
# dots.tts runtime (cuda-or-cpu, no MPS); we don't pass a device.
_runtime = None
def _load_runtime(stdout):
"""Cold-construct the dots.tts runtime.
``DotsTtsRuntime.from_pretrained`` auto-selects cuda-or-cpu internally
(no MPS path). precision is bf16 on CUDA; on CPU we fall back to fp32
(bf16 CPU kernels are spotty). Both overridable via env.
"""
global _runtime
if _runtime is not None:
return _runtime
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
import torch
from dots_tts.runtime import DotsTtsRuntime # type: ignore[import-not-found]
repo = os.environ.get("OMNIVOICE_DOTS_TTS_MODEL", _DEFAULT_REPO)
default_precision = "bfloat16" if torch.cuda.is_available() else "float32"
precision = os.environ.get("OMNIVOICE_DOTS_TTS_PRECISION", default_precision)
optimize = os.environ.get("OMNIVOICE_DOTS_TTS_OPTIMIZE", "0") == "1"
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 50})
_runtime = DotsTtsRuntime.from_pretrained(
repo,
precision=precision,
optimize=optimize,
)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _runtime
def _tensor_to_pcm_b64(audio, sample_rate: int) -> tuple[str, int, int]:
"""Convert a torch waveform tensor (1, N) in [-1, 1] to base64 int16 PCM."""
import numpy as np
arr = audio.detach().to("cpu").float().numpy()
arr = np.asarray(arr, dtype=np.float32).squeeze()
if arr.ndim > 1:
arr = arr.mean(axis=0) # defensive downmix to mono
arr = np.clip(arr, -1.0, 1.0)
pcm = (arr * 32767.0).astype(np.int16).tobytes()
return base64.b64encode(pcm).decode("ascii"), int(sample_rate), int(arr.shape[0])
def _normalize_language(raw):
"""Map OmniVoice's language value to what dots.tts accepts, or None.
dots.tts accepts None/"auto_detect", ISO codes upper-cased ("EN"/"ZH"),
or names ("english"). A 2-letter ISO code is upper-cased; anything else
is passed through; empty / "auto" None (auto-detect)."""
if not raw or not isinstance(raw, str):
return None
s = raw.strip()
if not s or s.lower() == "auto":
return None
if len(s) == 2 and s.isalpha():
return s.upper()
return s
def _handle_synthesize(msg: dict, stdout) -> None:
"""Dispatch one synthesize request. Emits the audio frame or raises."""
text = msg.get("text")
if not text or not isinstance(text, str):
raise ValueError("synthesize: missing or non-string 'text'")
runtime = _load_runtime(stdout)
gen_kwargs: dict = {
"text": text,
"num_steps": int(msg.get("num_steps", 10)),
"guidance_scale": float(msg.get("guidance_scale", 1.2)),
}
ref_audio = msg.get("ref_audio")
if ref_audio:
gen_kwargs["prompt_audio_path"] = ref_audio
ref_text = msg.get("ref_text")
if ref_text:
# continuation cloning — upstream requires prompt_audio_path when
# prompt_text is set (the parent already enforces this).
gen_kwargs["prompt_text"] = ref_text
language = _normalize_language(msg.get("language"))
if language:
gen_kwargs["language"] = language
result = runtime.generate(**gen_kwargs)
audio = result["audio"]
sample_rate = int(result.get("sample_rate", DOTS_SAMPLE_RATE))
pcm_b64, sr, n_samples = _tensor_to_pcm_b64(audio, sample_rate)
_send(stdout, {
"op": "audio",
"audio_pcm_b64": pcm_b64,
"sample_rate": sr,
"n_samples": n_samples,
})
# ── main loop ─────────────────────────────────────────────────────────────
def main() -> int:
stdin = sys.stdin.buffer
stdout = sys.stdout.buffer
# Ready handshake fires BEFORE any heavy import.
_send(stdout, {
"op": "ready",
"engine": "dots-tts",
"sample_rate": DOTS_SAMPLE_RATE,
})
while True:
try:
msg = _recv(stdin)
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": "recv",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
return 1
if msg is None:
return 0
op = msg.get("op") if isinstance(msg, dict) else None
try:
if op == "ping":
_send(stdout, {"op": "pong", "vram_mb": _measure_vram_mb()})
elif op == "synthesize":
_handle_synthesize(msg, stdout)
elif op == "shutdown":
return 0
else:
_send(stdout, {
"op": "error",
"stage": "dispatch",
"message": f"unknown op: {op!r}",
})
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": op or "unknown",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
if __name__ == "__main__":
sys.exit(main())
+194
View File
@@ -0,0 +1,194 @@
"""MOSS-TTS-v1.5 sidecar package (issue #498).
MOSS-TTS-v1.5 is OpenMOSS's 8B flagship TTS — a Qwen3-8B language backbone
plus a 1.6B audio codec, 31 languages, zero-shot voice cloning, token-level
duration control and inline ``[pause Ns]`` markers. Apache-2.0.
It runs in its own subprocess **and its own venv**, isolated from the
OmniVoice parent process, for the *same* reason IndexTTS does: a hard
``transformers`` version conflict. MOSS-TTS-v1.5's ``torch-runtime`` extra
pins ``transformers==5.0.0`` (verified against the upstream
``pyproject.toml``), while OmniVoice pins ``transformers>=5.3.0``. The two
cannot share one interpreter so MOSS lives behind ``SubprocessBackend``
with a dedicated venv, exactly like ``engines.indextts``.
Three public entry points live in this package:
* ``MossTTSV15Backend`` (this module) the SubprocessBackend subclass
that ``services.tts_backend._LAZY_REGISTRY`` resolves on first access.
Defined HERE (not in ``services.tts_backend``) to break the import
cycle: ``services.subprocess_backend`` imports ``TTSBackend`` from
``services.tts_backend``, so the backend class must live downstream of
that module finishing its import. Same indirection as IndexTTS /
Supertonic-3.
* ``main.py`` the sidecar entrypoint (runs under MOSS's venv with
``transformers==5.0.0``; never imported by the parent).
* ``bootstrap.py`` the venv-probe + lazy-bootstrap helper.
Do NOT import ``main.py`` from the parent process it runs under a
different venv (``transformers==5.0.0``) and importing it in-process would
re-introduce the exact conflict this isolation exists to avoid.
Hardware honesty (cross-platform rule): MOSS-TTS-v1.5's upstream documents
only CUDA and CPU. There is **no documented or tested MPS path** the
custom ``trust_remote_code`` modelling code and the separate audio
tokenizer are unverified on Apple Silicon. We therefore advertise
``gpu_compat = ("cuda", "cpu")`` and the sidecar selects ``cuda`` when
present else ``cpu`` it never silently routes to MPS where it might
crash. On Apple Silicon the engine honestly resolves to CPU (slow but
correct), and the engine is opt-in regardless, so it never becomes a
broken default on any platform.
"""
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from services.subprocess_backend import SubprocessBackend
if TYPE_CHECKING:
import torch # noqa: F401
logger = logging.getLogger("omnivoice.moss_tts_v15")
#: 1 second of audio ≈ 12.5 codec tokens (MOSS-TTS-v1.5 model card). Used to
#: translate OmniVoice's ``duration`` (seconds) into the model's ``tokens``
#: duration-control argument.
TOKENS_PER_SECOND: float = 12.5
class MossTTSV15Backend(SubprocessBackend):
"""MOSS-TTS-v1.5 (OpenMOSS) — 8B, 31 langs, zero-shot clone, CUDA/CPU.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv (``transformers==5.0.0``). The first synthesize cold-loads
~16 GB of bf16 weights (CUDA) / fp32 (CPU); subsequent calls reuse the
process and the in-memory model.
Installation (transparent to power users who already cloned MOSS-TTS
OmniVoice prefers their existing ``${DIR}/.venv``)::
git clone https://github.com/OpenMOSS/MOSS-TTS.git
cd MOSS-TTS
# CUDA host:
uv venv && uv pip install -e ".[torch-runtime]"
# non-CUDA host (CPU): install plain torch/transformers instead of +cu128
Set ``OMNIVOICE_MOSS_TTS_V15_DIR`` to the clone root. OmniVoice creates
``backend/engines/moss_tts_v15/.venv`` lazily on first launch if no venv
exists yet (CUDA hosts only the upstream ``torch-runtime`` extra is
``+cu128``); the user's existing ``${DIR}/.venv`` is preferred if
present, so no re-install is needed.
License: Apache-2.0 (code + weights) no acceptance gate needed.
"""
id = "moss-tts-v15"
display_name = (
"MOSS-TTS-v1.5 (8B, 31 langs, zero-shot clone, CUDA/CPU, Apache-2.0)"
)
supports_voice_design = False # requires ref audio for timbre cloning
_DEFAULT_SAMPLE_RATE = 24000
# Honest hardware surface: upstream documents CUDA + CPU only. MPS is
# undocumented / untested, so we do NOT claim it (cross-platform rule).
gpu_compat = ("cuda", "cpu")
# ── availability ───────────────────────────────────────────────────────
@classmethod
def is_available(cls) -> tuple[bool, str]:
# IMPORTANT: do NOT attempt to import MOSS / its transformers==5.0.0
# here. The parent pins transformers>=5.3 — co-importing the two in
# one interpreter is exactly the conflict this subprocess isolation
# exists to avoid. We only verify the venv exists on disk; a real
# health-check (spawn + ping) is gated on the user's "Test engine"
# action in Settings, same as IndexTTS.
from engines.moss_tts_v15.bootstrap import (
MOSS_TTS_V15_SIDECAR_SCRIPT,
is_moss_tts_v15_installed,
)
if not is_moss_tts_v15_installed():
return False, (
"MOSS-TTS-v1.5 venv not found. Set OMNIVOICE_MOSS_TTS_V15_DIR "
"to your MOSS-TTS clone (the directory containing pyproject.toml) "
"and restart OmniVoice. CUDA or CPU only (no MPS). See "
"docs/engines/moss-tts-v15.md for the full install walk-through."
)
if not MOSS_TTS_V15_SIDECAR_SCRIPT.exists():
return False, (
"MOSS-TTS-v1.5 sidecar script missing at "
f"{MOSS_TTS_V15_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok (CUDA when present, else CPU)"
@classmethod
def venv_python(cls):
from engines.moss_tts_v15.bootstrap import resolve_moss_tts_v15_venv
return resolve_moss_tts_v15_venv()
@classmethod
def sidecar_script(cls):
from engines.moss_tts_v15.bootstrap import MOSS_TTS_V15_SIDECAR_SCRIPT
return MOSS_TTS_V15_SIDECAR_SCRIPT
# ── TTSBackend protocol ────────────────────────────────────────────────
@property
def sample_rate(self) -> int:
return self._DEFAULT_SAMPLE_RATE
@property
def supported_languages(self) -> list[str]:
# 31 languages with multilingual handling; expose "multi" on the
# protocol surface (same as OmniVoice / CosyVoice / Supertonic-3) and
# translate the caller's language at synthesize time.
return ["multi"]
# ── generate (parent-side arbitration) ─────────────────────────────────
def generate(self, text: str, **kw) -> "torch.Tensor":
"""Synthesize one utterance through the MOSS-TTS-v1.5 sidecar.
kwargs honored:
* ``ref_audio`` path to a reference clip. When present, MOSS
runs zero-shot voice cloning (``reference=``).
Optional: without it the model uses its own
default voice.
* ``ref_text`` accepted but unused in clone mode (MOSS's
zero-shot path needs only the audio); kept in
the signature so the common call-site doesn't
need engine-specific knowledge.
* ``language`` ISO code or name; mapped to a MOSS language name
in the sidecar, omitted (auto-detect) if unknown.
* ``duration`` target seconds ``tokens`` (1 s 12.5 tokens).
* ``max_new_tokens`` generation cap (default 4096).
Returns a tensor of shape (1, n_samples) at :attr:`sample_rate`.
"""
forwarded: dict = {}
ref_audio = kw.get("ref_audio")
if ref_audio:
forwarded["ref_audio"] = ref_audio
ref_text = kw.get("ref_text")
if ref_text:
forwarded["ref_text"] = ref_text
language = kw.get("language")
if language:
forwarded["language"] = str(language)
duration = kw.get("duration")
if duration is not None:
target_tokens = int(float(duration) * TOKENS_PER_SECOND)
if target_tokens > 0:
forwarded["tokens"] = target_tokens
max_new_tokens = kw.get("max_new_tokens")
if max_new_tokens is not None:
forwarded["max_new_tokens"] = int(max_new_tokens)
return super().generate(text, **forwarded)
__all__ = ["MossTTSV15Backend", "TOKENS_PER_SECOND"]
+272
View File
@@ -0,0 +1,272 @@
"""MOSS-TTS-v1.5 venv probe + lazy bootstrap (issue #498).
The parent process needs to know *which Python interpreter* to spawn the
MOSS-TTS-v1.5 sidecar under. This module owns that resolution. It mirrors
``engines.indextts.bootstrap`` because MOSS has the same shape of problem:
a hard ``transformers`` pin (``==5.0.0``) that conflicts with the parent's
``transformers>=5.3`` so MOSS runs in its own venv.
Probe order (priority existing power-user installs win, zero migration):
1. ``${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv/`` the user's clone-level
venv. Highest priority: a user who already cloned MOSS-TTS and ran
``uv pip install -e ".[torch-runtime]"`` (per upstream docs) gets
reused verbatim, no re-download of the ~16 GB model.
2. ``backend/engines/moss_tts_v15/.venv/`` this package's own venv,
created by step 3 if needed.
3. Bootstrap: ``uv venv`` then ``uv pip install -e
"${DIR}[torch-runtime]"``. Requires ``OMNIVOICE_MOSS_TTS_V15_DIR``.
The upstream ``torch-runtime`` extra is CUDA (``+cu128``), so the
auto-bootstrap targets CUDA hosts; non-CUDA (CPU/Mac) users set up
their own venv per docs/engines/moss-tts-v15.md (Probe 1).
Caching: resolution is memoised after the first successful call. Tests
reset via :func:`invalidate`.
Security: bootstrap never touches HF_TOKEN; the sidecar's stderr is drained
by SubprocessBackend through the parent root logger where Phase 1's
``HFTokenRedactor`` strips token bytes. ``uv pip install -e`` installs from
a user-controlled clone the user already trusts (same posture as IndexTTS).
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger("omnivoice.moss_tts_v15.bootstrap")
#: Absolute path to the sidecar entrypoint. ``MossTTSV15Backend.sidecar_script``
#: returns this; SubprocessBackend spawns it with the resolved venv python.
MOSS_TTS_V15_SIDECAR_SCRIPT: Path = Path(__file__).parent / "main.py"
#: This package's owned venv (Probe 2). The MOSS-TTS clone, when bootstrapped,
#: is installed into this venv via ``uv pip install -e``.
_ENGINES_VENV_DIR: Path = Path(__file__).parent / ".venv"
#: Env var pointing at the user's MOSS-TTS clone root.
_CLONE_DIR_ENV: str = "OMNIVOICE_MOSS_TTS_V15_DIR"
#: Per-process resolution cache. Cleared by :func:`invalidate` for tests.
_resolved_python: Optional[Path] = None
# Timeouts — bounded so a wedged venv never hangs the parent. The bootstrap
# install can take many minutes on a cold cache (MOSS pulls a CUDA torch
# build + transformers + an audio codec stack).
_IMPORT_PROBE_TIMEOUT_S = 15
_UV_VENV_TIMEOUT_S = 120
_UV_PIP_INSTALL_TIMEOUT_S = 1800
# ── public API ────────────────────────────────────────────────────────────
def invalidate() -> None:
"""Clear the resolved-python cache. Tests call this between scenarios."""
global _resolved_python
_resolved_python = None
def is_moss_tts_v15_installed() -> bool:
"""Cheap file-existence check for a usable MOSS-TTS-v1.5 venv.
Returns True if either Probe 1 or Probe 2 has a Python executable on
disk. Does NOT spawn the venv Python that's saved for
:func:`resolve_moss_tts_v15_venv`, which is only invoked on the first
generate() / health_check(). This fires on every Settings render via
``MossTTSV15Backend.is_available()``, so it stays cheap.
"""
for cand in _probe_paths():
if cand.is_file():
return True
return False
def resolve_moss_tts_v15_venv() -> Path:
"""Resolve the path to the Python interpreter that runs the sidecar.
Probe order described in the module docstring. Memoised. Raises
:exc:`RuntimeError` if no working venv can be located AND the bootstrap
path is unavailable.
"""
global _resolved_python
if _resolved_python is not None:
return _resolved_python
clone_dir = os.environ.get(_CLONE_DIR_ENV)
# Probe 1 — user's clone-level venv (highest priority for back-compat).
if clone_dir:
cand = _venv_python_path(Path(clone_dir) / ".venv")
if cand.is_file() and _venv_can_import_moss(cand):
logger.info(
"MOSS-TTS-v1.5 venv resolved from %s: %s", _CLONE_DIR_ENV, cand,
)
_resolved_python = cand
return cand
# Probe 2 — this package's own venv.
cand = _venv_python_path(_ENGINES_VENV_DIR)
if cand.is_file() and _venv_can_import_moss(cand):
logger.info("MOSS-TTS-v1.5 venv resolved from engines path: %s", cand)
_resolved_python = cand
return cand
# Probe 3 — bootstrap.
if not clone_dir:
raise RuntimeError(
"MOSS-TTS-v1.5 is not installed. Set the "
f"{_CLONE_DIR_ENV} environment variable to your MOSS-TTS clone "
"(the directory that contains pyproject.toml), then restart "
"OmniVoice. See docs/engines/moss-tts-v15.md for the full "
"install walk-through."
)
cand = _bootstrap_engines_venv(Path(clone_dir))
_resolved_python = cand
return cand
# ── internals ─────────────────────────────────────────────────────────────
def _venv_python_path(venv_dir: Path) -> Path:
"""Return the python executable path inside a venv directory.
Handles the Unix (``bin/python``) vs Windows (``Scripts/python.exe``)
layout. No filesystem access caller checks .is_file().
"""
if sys.platform == "win32":
return venv_dir / "Scripts" / "python.exe"
return venv_dir / "bin" / "python"
def _probe_paths() -> list[Path]:
"""Ordered list of candidate venv-python paths (no .is_file() check)."""
out: list[Path] = []
clone_dir = os.environ.get(_CLONE_DIR_ENV)
if clone_dir:
out.append(_venv_python_path(Path(clone_dir) / ".venv"))
out.append(_venv_python_path(_ENGINES_VENV_DIR))
return out
def _venv_can_import_moss(python_path: Path) -> bool:
"""Spawn the candidate python and verify the MOSS stack imports.
MOSS-TTS-v1.5 loads via ``transformers`` + ``trust_remote_code`` (no
fixed top-level package to import), so the readiness signal is that the
venv has a working ``transformers`` + ``torch`` which only the
``[torch-runtime]`` install provides. Bounded by
``_IMPORT_PROBE_TIMEOUT_S`` so a wedged venv never hangs the parent.
Returns False on any failure (non-zero exit, timeout, OSError).
"""
try:
proc = subprocess.run(
[str(python_path), "-c", "import transformers, torch"],
capture_output=True,
timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
logger.debug("moss-tts-v15 import probe failed for %s: %s", python_path, exc)
return False
if proc.returncode != 0:
logger.debug(
"moss-tts-v15 import probe non-zero for %s: %s",
python_path,
proc.stderr.decode("utf-8", errors="replace")[:200],
)
return False
return True
def _locate_uv() -> Optional[str]:
"""Find the uv binary — bundled first (Tauri-set env var), else PATH."""
bundled = os.environ.get("OMNIVOICE_BUNDLED_UV")
if bundled and Path(bundled).is_file():
return bundled
sys_uv = shutil.which("uv")
if sys_uv:
return sys_uv
return None
def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"""Create engines/moss_tts_v15/.venv and install the user's clone into it.
Runs ``uv venv <engines_venv>`` then ``uv pip install --python
<engines_venv>/bin/python -e "<clone>[torch-runtime]"``. Verifies the
result by re-probing the import a successful uv invocation that still
can't import the stack indicates a deeper environment problem (e.g. the
``+cu128`` torch-runtime extra can't resolve on a non-CUDA host) and we
raise with whatever stderr we captured plus a docs pointer.
"""
uv = _locate_uv()
if not uv:
raise RuntimeError(
"uv is required to bootstrap the MOSS-TTS-v1.5 venv but was not "
"found on PATH (and OMNIVOICE_BUNDLED_UV was not set). Install uv "
"from https://docs.astral.sh/uv/ and re-launch OmniVoice, or set "
"OMNIVOICE_BUNDLED_UV to the absolute path of a uv binary."
)
logger.info(
"Bootstrapping MOSS-TTS-v1.5 venv at %s from %s (this can take "
"several minutes on first launch)", _ENGINES_VENV_DIR, clone_dir,
)
try:
subprocess.run(
[uv, "venv", str(_ENGINES_VENV_DIR)],
check=True,
timeout=_UV_VENV_TIMEOUT_S,
capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
f"uv venv failed for MOSS-TTS-v1.5 bootstrap at {_ENGINES_VENV_DIR}: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
python_path = _venv_python_path(_ENGINES_VENV_DIR)
try:
subprocess.run(
[
uv, "pip", "install",
"--python", str(python_path),
"-e", f"{clone_dir}[torch-runtime]",
],
check=True,
timeout=_UV_PIP_INSTALL_TIMEOUT_S,
capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install -e failed during MOSS-TTS-v1.5 bootstrap "
f"({clone_dir}). On a non-CUDA host the upstream '[torch-runtime]' "
"extra (cu128) cannot resolve — set up the venv manually per "
"docs/engines/moss-tts-v15.md. Error: "
f"{exc.stderr.decode('utf-8', errors='replace') if exc.stderr else exc}"
) from exc
if not _venv_can_import_moss(python_path):
raise RuntimeError(
"MOSS-TTS-v1.5 bootstrap completed but the transformers/torch "
f"import still fails from {python_path}. Verify that {clone_dir} "
"is a valid MOSS-TTS clone. See docs/engines/moss-tts-v15.md."
)
logger.info("MOSS-TTS-v1.5 venv bootstrap successful: %s", python_path)
return python_path
__all__ = [
"MOSS_TTS_V15_SIDECAR_SCRIPT",
"invalidate",
"is_moss_tts_v15_installed",
"resolve_moss_tts_v15_venv",
]
+303
View File
@@ -0,0 +1,303 @@
"""MOSS-TTS-v1.5 sidecar entry point (issue #498).
Runs inside ``engines/moss_tts_v15/.venv`` (or the user's existing
``${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv``) with ``transformers==5.0.0``,
isolated from the OmniVoice parent process which pins ``transformers>=5.3``.
Same isolation rationale as the IndexTTS sidecar.
Stdlib-only at import time. The model + transformers + torch are imported
lazily on the first synthesize op so the sidecar emits its ``ready`` frame
inside the parent's 30 s spawn handshake even on a cold filesystem (an 8B
model takes well over 30 s to cold-load).
Wire protocol length-prefixed JSON over stdin/stdout, byte-identical to
``backend/services/subprocess_backend.py``::
[ 4-byte big-endian uint32 length ][ N bytes UTF-8 JSON ]
Op flow:
1. Sidecar -> parent: {"op": "ready", "engine": "moss-tts-v15",
"sample_rate": 24000}
2. parent -> sidecar: {"op": "ping"} -> {"op": "pong", "vram_mb": N}
3. parent -> sidecar: {"op": "synthesize", "text": "...",
"ref_audio": "/path/spk.wav", "language": "fr",
"tokens": 325, "max_new_tokens": 4096}
-> {"op": "progress", ...} (cold load only) then
-> {"op": "audio", "audio_pcm_b64": "...", "sample_rate": 24000,
"n_samples": N}
4. parent -> sidecar: {"op": "shutdown"} -> exit 0
Restrictions: NO imports from OmniVoice parent code (different venv). NO
logging of ``os.environ`` contents. Single-frame DoS cap matches the
parent's ``MAX_FRAME_BYTES``.
"""
from __future__ import annotations
import base64
import json
import os
import struct
import sys
import traceback
# Mirrors backend/services/subprocess_backend.py::MAX_FRAME_BYTES.
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: Native sample rate MOSS-TTS-v1.5 emits. Advertised in the ready frame so
#: the parent doesn't have to import MOSS just to learn the rate. Confirmed
#: via ``processor.model_config.sampling_rate`` (the real value is read from
#: the loaded model at synthesize time; this is the handshake default).
MOSS_SAMPLE_RATE = 24000
#: HF repo id for the weights, overridable for air-gapped / mirror installs.
_DEFAULT_REPO = "OpenMOSS-Team/MOSS-TTS-v1.5"
#: ISO-639-1 → MOSS language name. MOSS's ``build_user_message`` takes a
#: language *name* ("French"), not a code. Unknown codes are omitted so the
#: model auto-detects. Covers the high-traffic subset of MOSS's 31 langs.
_ISO_TO_NAME = {
"en": "English", "zh": "Chinese", "ja": "Japanese", "ko": "Korean",
"fr": "French", "de": "German", "es": "Spanish", "it": "Italian",
"pt": "Portuguese", "ru": "Russian", "ar": "Arabic", "hi": "Hindi",
"nl": "Dutch", "pl": "Polish", "tr": "Turkish", "vi": "Vietnamese",
"th": "Thai", "id": "Indonesian", "cs": "Czech", "el": "Greek",
"he": "Hebrew", "fa": "Persian", "uk": "Ukrainian", "sv": "Swedish",
}
# ── wire protocol ─────────────────────────────────────────────────────────
def _send(stream, obj: dict) -> None:
body = json.dumps(obj, separators=(",", ":")).encode("utf-8")
stream.write(struct.pack("!I", len(body)))
stream.write(body)
stream.flush()
def _recv(stream):
header = stream.read(4)
if len(header) < 4:
return None # EOF
(n,) = struct.unpack("!I", header)
if n > MAX_FRAME_BYTES:
raise IOError(f"frame too large: {n}")
body = bytearray()
while len(body) < n:
chunk = stream.read(n - len(body))
if not chunk:
raise IOError("short read")
body.extend(chunk)
return json.loads(bytes(body).decode("utf-8"))
def _measure_vram_mb() -> float:
"""This sidecar's own GPU memory in MB (MM2-08). The parent can't see a
child's VRAM, so we self-report it in the pong. 0 on CPU. Never raises."""
try:
import torch
if torch.cuda.is_available():
return round(torch.cuda.memory_allocated() / (1024 ** 2), 1)
except Exception:
pass
return 0.0
# ── model loading (lazy, on first synthesize) ─────────────────────────────
# Module-level singleton — populated on the first synthesize op and reused.
# Holds (processor, model, device, sample_rate).
_state = None
def _load_model(stdout):
"""Cold-construct the MOSS-TTS-v1.5 processor + model.
Device selection is CUDA-or-CPU only MOSS's upstream documents no MPS
path and the custom ``trust_remote_code`` modelling code is untested on
Apple Silicon, so we never route to MPS where it might crash. dtype is
bf16 on CUDA, fp32 on CPU (bf16 CPU ops are spotty). Emits progress
frames so the parent can surface the multi-GB cold-load latency.
"""
global _state
if _state is not None:
return _state
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
import torch
from transformers import AutoModel, AutoProcessor
repo = os.environ.get("OMNIVOICE_MOSS_TTS_V15_MODEL", _DEFAULT_REPO)
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
# "sdpa" works on CUDA + CPU and needs no extra dep. flash_attention_2
# (Ampere+ CUDA, optional flash-attn) is opt-in via env.
attn = os.environ.get("OMNIVOICE_MOSS_TTS_V15_ATTN", "sdpa")
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
# The audio tokenizer is a separate sub-module that must be moved to the
# device independently (easy to miss — see upstream README).
processor.audio_tokenizer = processor.audio_tokenizer.to(device)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 50})
model = AutoModel.from_pretrained(
repo,
trust_remote_code=True,
attn_implementation=attn,
torch_dtype=dtype,
).to(device)
model.eval()
sample_rate = int(getattr(processor.model_config, "sampling_rate", MOSS_SAMPLE_RATE))
_state = (processor, model, device, sample_rate)
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 100})
return _state
def _tensor_to_pcm_b64(audio, sample_rate: int) -> tuple[str, int, int]:
"""Convert a torch waveform tensor to base64 int16 PCM.
MOSS returns a float tensor in [-1, 1] (1-D or (1, N)); we squeeze to
mono, clip, scale to int16, and base64 so the wire frame stays JSON-safe.
"""
import numpy as np
arr = audio.detach().to("cpu").float().numpy()
arr = np.asarray(arr, dtype=np.float32).squeeze()
if arr.ndim > 1:
arr = arr.mean(axis=0) # defensive downmix to mono
arr = np.clip(arr, -1.0, 1.0)
pcm = (arr * 32767.0).astype(np.int16).tobytes()
return base64.b64encode(pcm).decode("ascii"), int(sample_rate), int(arr.shape[0])
def _resolve_language(raw):
"""Map OmniVoice's language value to a MOSS language name, or None.
Accepts an ISO-639-1 code or a full name. Unknown / empty / "auto"
values return None so MOSS auto-detects."""
if not raw or not isinstance(raw, str):
return None
s = raw.strip()
if not s or s.lower() == "auto":
return None
if s.lower() in _ISO_TO_NAME:
return _ISO_TO_NAME[s.lower()]
# Already a language name (or an unknown code) — pass it through; MOSS
# ignores a language it doesn't recognise.
return s
def _handle_synthesize(msg: dict, stdout) -> None:
"""Dispatch one synthesize request. Emits the audio frame or raises."""
import torch
text = msg.get("text")
if not text or not isinstance(text, str):
raise ValueError("synthesize: missing or non-string 'text'")
processor, model, device, sample_rate = _load_model(stdout)
user_kwargs: dict = {"text": text}
ref_audio = msg.get("ref_audio")
if ref_audio:
# Zero-shot voice cloning: the reference audio alone is enough in
# MOSS's clone mode (ref_text is not consumed here). The processor's
# audio tokenizer encodes the reference into the prompt.
user_kwargs["reference"] = [ref_audio]
language = _resolve_language(msg.get("language"))
if language:
user_kwargs["language"] = language
tokens = msg.get("tokens")
if tokens is not None:
user_kwargs["tokens"] = int(tokens)
max_new_tokens = int(msg.get("max_new_tokens", 4096))
conversations = [[processor.build_user_message(**user_kwargs)]]
with torch.no_grad():
batch = processor(conversations, mode="generation")
outputs = model.generate(
input_ids=batch["input_ids"].to(device),
attention_mask=batch["attention_mask"].to(device),
max_new_tokens=max_new_tokens,
)
decoded = processor.decode(outputs)
audio = decoded[0].audio_codes_list[0]
pcm_b64, sr, n_samples = _tensor_to_pcm_b64(audio, sample_rate)
_send(stdout, {
"op": "audio",
"audio_pcm_b64": pcm_b64,
"sample_rate": sr,
"n_samples": n_samples,
})
# ── main loop ─────────────────────────────────────────────────────────────
def main() -> int:
stdin = sys.stdin.buffer
stdout = sys.stdout.buffer
# Ready handshake fires BEFORE any heavy import — nothing above this line
# touches transformers/torch, so we make the 30 s spawn window even cold.
_send(stdout, {
"op": "ready",
"engine": "moss-tts-v15",
"sample_rate": MOSS_SAMPLE_RATE,
})
while True:
try:
msg = _recv(stdin)
except Exception as exc:
_send(stdout, {
"op": "error",
"stage": "recv",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
return 1
if msg is None:
return 0
op = msg.get("op") if isinstance(msg, dict) else None
try:
if op == "ping":
_send(stdout, {"op": "pong", "vram_mb": _measure_vram_mb()})
elif op == "synthesize":
_handle_synthesize(msg, stdout)
elif op == "shutdown":
return 0
else:
_send(stdout, {
"op": "error",
"stage": "dispatch",
"message": f"unknown op: {op!r}",
})
except Exception as exc:
# Per-op failure is recoverable — emit the error frame and stay
# alive so the parent can retry without paying the respawn +
# multi-GB model-load cost again.
_send(stdout, {
"op": "error",
"stage": op or "unknown",
"message": f"{type(exc).__name__}: {exc}",
"traceback": traceback.format_exc(),
})
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -70,6 +70,7 @@ class Supertonic3Backend(SubprocessBackend):
id = "supertonic3"
display_name = "Supertonic-3 (31 langs, CPU ONNX, 7 preset voices, OpenRAIL-M)"
supports_voice_design = False # preset voices only
supports_cloning = False # preset voices only; generate() never reads ref_audio
# TTS-04: honest hardware reporting. Supertonic-3 has no CUDA / MPS
# path in the SDK ‑‑ ONNX Runtime CPU EP only.
gpu_compat: tuple[str, ...] = ("cpu",)
+274 -26
View File
@@ -9,6 +9,16 @@ _backend_dir = os.path.dirname(os.path.abspath(__file__))
if _backend_dir not in sys.path:
sys.path.insert(0, _backend_dir)
# #564: also make the project's OWN `omnivoice` package importable from source
# when the venv's editable install is missing/broken (interrupted/offline
# `uv sync`, antivirus-quarantined `_editable_impl_omnivoice.pth`, …). Without
# this the backend boots fine and only fails at the first model call with
# `No module named 'omnivoice'`. The bootstrap now gates on omnivoice being
# importable too (re-syncing to re-lay the editable install); this is the
# runtime safety net. See core/omnivoice_path.py for the full rationale.
from core.omnivoice_path import ensure_omnivoice_importable
ensure_omnivoice_importable(_backend_dir)
# Triton is unavailable on Windows — disable torch.compile / dynamo / inductor
# to prevent TritonMissing errors at inference time. Must be set before torch
# is imported (it is lazily imported in services/model_manager.py). Uses
@@ -129,6 +139,24 @@ os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
os.environ.setdefault("HF_HUB_ETAG_TIMEOUT", "15")
os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "30")
# ── OS trust store for TLS (#976) ───────────────────────────────────────────
# Users behind a corporate/antivirus proxy that TLS-inspects HTTPS traffic get
# a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure"
# on every model install — the TCP connection succeeds (a different failure
# mode from #984's TCP-level blocked-host case), but the proxy re-signs the
# certificate with its own root CA, which the OS trusts (Windows CryptoAPI/
# SChannel) and Python's bundled `certifi` CA list does not. `inject_into_ssl`
# patches `ssl.SSLContext` process-wide to verify against the OS trust store
# instead, which is the actual fix (not just a nicer error message). Must run
# here — at MODULE level, before huggingface_hub/requests/httpx do any network
# I/O — not inside lifespan(), which runs too late. Not platform-gated: it's a
# correctness improvement everywhere. Best-effort: never block startup.
try:
import truststore
truststore.inject_into_ssl()
except Exception:
pass
# Prevent torchaudio from lazy-importing torchcodec (broken on some installs).
# Proper fix = exclude torchcodec in pyproject.toml; this is a belt-and-braces guard.
@@ -145,6 +173,16 @@ from logging.handlers import RotatingFileHandler
# written to prefs.json so they survive backend restarts. Read them back
# here — before any user code reads os.environ — so the values are available
# from startup.
#
# Legacy (≤v0.3.7) Translation-LLM rows (env.TRANSLATE_*) must migrate into
# the custom LLM provider's settings store BEFORE the re-import below — once
# TRANSLATE_BASE_URL lands in os.environ it hijacks the LLM provider
# selection for the whole session (#963). Real env vars are untouched.
try:
from services.llm_providers import migrate_legacy_translate_prefs
migrate_legacy_translate_prefs()
except Exception:
pass # never block startup on the migration; it retries next launch
_PERSISTED_ENV_PREFIX = "env."
try:
from core.prefs import _load as _load_all_prefs
@@ -327,6 +365,7 @@ from api.routers import (
events,
capture,
capture_ws,
dictation,
openai_compat,
tts_stream,
marketplace,
@@ -334,6 +373,7 @@ from api.routers import (
sonitranslate,
audiobook,
longform_jobs,
pronunciation, # Expressive-TTS Spec 01: user pronunciation dictionary
settings as settings_router, # Phase 1 AUTH-03: HF token save/clear/state
)
from utils import hf_progress
@@ -375,8 +415,145 @@ def _env_flag(name: str, default: bool = False) -> bool:
return value.strip().lower() in {"1", "true", "yes", "on"}
def _capture_preload_delay_s() -> float:
"""Seconds after boot before the dictation (capture ASR) model warms.
Late enough that it never competes with startup I/O or the TTS preload;
overridable via OMNIVOICE_CAPTURE_PRELOAD_DELAY (mostly for tests)."""
raw = os.environ.get("OMNIVOICE_CAPTURE_PRELOAD_DELAY", "")
try:
v = float(raw)
if v >= 0:
return v
except (TypeError, ValueError):
pass
return 30.0
def _capture_preload_ram_ok(min_free_bytes: int = 4 * 1024**3) -> bool:
"""RAM guard for the dictation warm-up: skip below 4 GB free so the
background load never pushes a small machine into swap. If free memory
can't be measured, warm anyway (the load path has its own error handling)."""
try:
import psutil
return psutil.virtual_memory().available >= min_free_bytes
except Exception:
return True
def _mcp_start_timeout_s() -> float:
"""Seconds to wait for the MCP session manager to start before giving up
and serving without it (#632). Overridable via OMNIVOICE_MCP_START_TIMEOUT_S."""
raw = os.environ.get("OMNIVOICE_MCP_START_TIMEOUT_S", "")
try:
v = float(raw)
if v > 0:
return v
except (TypeError, ValueError):
pass
return 30.0
async def _serve_mcp(session_manager, ready: "asyncio.Event", stop: "asyncio.Event") -> None:
"""Own the MCP session manager's full enter→exit lifecycle in ONE task.
FastMCP's ``run()`` opens an anyio task group, and anyio requires the cancel
scope to be exited in the *same task* that entered it. So we must NOT enter
it via ``wait_for`` (which runs the enter in a throwaway sub-task) or on the
lifespan task and exit it elsewhere either raises "Attempted to exit cancel
scope in a different task". This coroutine enters and exits the context
itself: it signals ``ready`` once mounted, then idles until ``stop``.
"""
try:
async with session_manager.run():
ready.set()
await stop.wait()
except Exception as e:
logger.warning("MCP session manager stopped: %s", e)
finally:
ready.set() # never leave startup blocked on the readiness wait
async def _start_mcp_session_manager(session_manager, *, timeout: float):
"""Start MCP off the startup critical path; wait up to ``timeout`` for it to
signal ready. Returns ``(task, stop_event, mounted)``.
The MCP layer is best-effort and must never wedge backend startup. On some
platforms (observed: Apple-Silicon M1, #632) ``run()`` can *hang* on its
anyio task group; the old code awaited the enter before serving, so the hang
meant "Application startup complete" never fired and the whole backend was
unreachable with no error. Now the enter lives in its own task and we only
*optionally* wait on a ready signal a hang becomes a logged warning + a
backend that serves normally without MCP.
"""
stop = asyncio.Event()
if session_manager is None:
return None, stop, False
ready = asyncio.Event()
task = asyncio.create_task(_serve_mcp(session_manager, ready, stop))
try:
await asyncio.wait_for(ready.wait(), timeout=timeout)
mounted = not task.done() # ready is also set on failure → not mounted
except asyncio.TimeoutError:
logger.warning(
"MCP session manager did not signal ready within %.0fs (#632); "
"serving without waiting. Set OMNIVOICE_MCP_START_TIMEOUT_S to adjust.",
timeout,
)
mounted = False
return task, stop, mounted
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
"""Cancel each background task and give it a bounded chance to actually
finish before shutdown proceeds ``None`` entries are skipped (a task
that's conditionally created, e.g. ``capture_preload_task``, may not
exist).
``task.cancel()`` alone is not enough for a task awaiting
``run_in_executor()``: once the underlying OS thread is inside blocking
native/import work, cancellation can't stop it, so cancel-and-move-on lets
shutdown finish while that thread is still running invisible to
asyncio, but very much alive when the interpreter starts tearing down
module state under it (#1000 class). Awaiting with a bound (instead of
just cancelling) gives an early-stage task a real chance to exit cleanly
first; a task that's genuinely still deep in blocking work times out here
same as before, and the caller's own GPU-pool reset handles that case.
"""
for t in tasks:
if t is None:
continue
t.cancel()
for t in tasks:
if t is None:
continue
try:
await asyncio.wait_for(t, timeout=timeout)
except (asyncio.CancelledError, asyncio.TimeoutError):
pass
@asynccontextmanager
async def lifespan(app: FastAPI):
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
# MCP deadlock on some platforms) means "Application startup complete" never
# logs and the app sits forever with no error. If startup hasn't finished
# within the window, dump every thread's stack to stderr (→ backend_err.log)
# so the hang point is captured instead of invisible. Cancelled the instant
# startup completes, so a normal (even slow-download) boot never trips it.
# Tune with OMNIVOICE_STARTUP_WATCHDOG_S (seconds; 0 disables). Best-effort —
# never let the diagnostic itself break startup.
_watchdog_armed = False
try:
import faulthandler
_wd = float(os.environ.get("OMNIVOICE_STARTUP_WATCHDOG_S", "300"))
if _wd > 0 and hasattr(faulthandler, "dump_traceback_later"):
faulthandler.dump_traceback_later(_wd, repeat=False, exit=False)
_watchdog_armed = True
logger.info("Startup watchdog armed: thread dump if startup exceeds %.0fs (#632).", _wd)
except Exception:
pass
init_db()
# Network sharing is loopback-only by default; the PIN middleware stays
# inert until enable() sets a PIN. Seed the (disabled) state so the
@@ -425,11 +602,20 @@ async def lifespan(app: FastAPI):
worker_task = asyncio.create_task(task_manager.worker())
# Warm the TTS model in the background so first /generate is instant.
preload_task = asyncio.create_task(preload_model())
# Capture ASR is useful to keep warm, but it is another large model in
# unified memory on Apple Silicon. Keep launch lean by default; users who
# prefer instant dictation can opt in with OMNIVOICE_PRELOAD_CAPTURE_ASR=1.
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR"):
# Dictation v2: the capture ASR warms in the background BY DEFAULT — a
# deferred (~30s post-boot) load off the event loop, so startup stays
# lean and the first dictation is instant instead of a cold model load.
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
# under 4 GB free RAM (checked at warm time, not boot time).
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
async def _preload_capture_asr():
await asyncio.sleep(_capture_preload_delay_s())
if not _capture_preload_ram_ok():
logger.info(
"Capture ASR preload skipped: <4GB free RAM; "
"dictation ASR will load on first use.")
return
loading_detail = None
prev_loading_detail = None
try:
@@ -459,31 +645,64 @@ async def lifespan(app: FastAPI):
logger.info("Capture ASR preload disabled; dictation ASR will load on first use.")
# ── MCP session manager (Wave 2.2) ────────────────────────────────────
# FastMCP's Streamable-HTTP transport needs its session manager running
# for the lifetime of the app. It's created lazily by streamable_http_app()
# (called in mount_mcp below), so we stack its `run()` context into ours
# via AsyncExitStack rather than replacing this lifespan. Best-effort: a
# missing/broken MCP layer must never stop the rest of the backend.
from contextlib import AsyncExitStack
async with AsyncExitStack() as _mcp_stack:
_sm = getattr(app.state, "mcp_session_manager", None)
if _sm is not None:
try:
await _mcp_stack.enter_async_context(_sm.run())
logger.info("MCP server mounted at /mcp")
except Exception as e:
logger.warning("MCP session manager failed to start: %s", e)
yield
# FastMCP's Streamable-HTTP transport needs its session manager running for
# the lifetime of the app. Run it in its OWN task that owns the full
# enter→exit lifecycle (anyio task-affinity, see _serve_mcp) and only wait,
# with a timeout, for it to signal ready — so a hang on its anyio group
# (observed on M1, #632) can never wedge "Application startup complete".
_sm = getattr(app.state, "mcp_session_manager", None)
mcp_task, mcp_stop, mcp_mounted = await _start_mcp_session_manager(
_sm, timeout=_mcp_start_timeout_s()
)
if mcp_mounted:
logger.info("MCP server mounted at /mcp")
# Startup finished — disarm the hang watchdog before serving (#632).
if _watchdog_armed:
try:
import faulthandler
faulthandler.cancel_dump_traceback_later()
except Exception:
pass
yield
# ── Graceful shutdown (SIGTERM from Tauri, Ctrl+C, etc.) ────────────
logger.info("Shutdown: cleaning up…")
idle_task.cancel()
worker_task.cancel()
# Wait for tasks to finish their current iteration
for t in (idle_task, worker_task):
# Stop MCP first — signal its task to exit its own anyio context (correct
# task-affinity), then bound the wait so a wedged manager can't hang exit.
mcp_stop.set()
if mcp_task is not None:
try:
await asyncio.wait_for(t, timeout=3.0)
except (asyncio.CancelledError, asyncio.TimeoutError):
await asyncio.wait_for(mcp_task, timeout=5.0)
except (asyncio.TimeoutError, asyncio.CancelledError):
pass
except Exception:
pass
# preload_task/capture_preload_task matter most here (#1000 class): a quit
# mid-preload used to fall straight through to "Shutdown: done." while the
# model load was still running on a GPU-pool thread — cancel() can't stop
# a thread already inside blocking import/load work, so the process
# reported a clean exit while that background thread was still mid-
# `import transformers`, and got torn down by interpreter finalization
# instead. That surfaced as a misleading "Could not import module
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
# not a real dependency problem. Awaiting here lets an early-stage load
# (still importing, not yet mid weight-download) finish cleanly before we
# report done; a load that's genuinely deep into a multi-GB download still
# times out — _reset_gpu_pool() below abandons it either way.
#
# 20s, not the original 3s (code-review finding post-merge): a cold
# transformers import alone can take longer than 3s on a slow disk or a
# first-ever launch, so the original bound left a real residual window —
# cancellation detaches the asyncio task, but the underlying OS thread
# keeps running past it, and shutdown could still report "done" while
# that thread was alive. Python cannot forcibly kill a running thread, so
# no finite bound eliminates this outright — 20s just shrinks the window
# from "any preload" to "an unusually slow cold-import," which is the
# practical ceiling before a longer shutdown itself becomes the
# complaint. A thread that's still running past 20s was never going to
# finish in a shutdown-appropriate timeframe regardless.
await _cancel_and_await_tasks(
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
)
# Unload the model and free GPU memory
try:
import services.model_manager as mm
@@ -491,6 +710,10 @@ async def lifespan(app: FastAPI):
mm.model = None
logger.info("Shutdown: model unloaded.")
mm.free_vram()
# Abandon a still-running preload's GPU-pool thread (Python can't kill
# a thread mid blocking call) so it can't outlive this shutdown block
# holding a reference into module state that's about to be torn down.
mm._reset_gpu_pool()
except Exception:
pass
# Run GC to release any remaining references
@@ -573,8 +796,17 @@ async def global_exception_handler(request: Request, exc: Exception):
headers["Access-Control-Allow-Origin"] = origin
headers["Access-Control-Allow-Credentials"] = "true"
headers["Vary"] = "Origin"
# #874: a model download that failed because the CONFIGURED Hugging Face
# mirror (HF_ENDPOINT) is unreachable used to leak the raw transformers
# message ("We couldn't connect to 'https://hf-mirror.com' …") as the 500
# detail with no next step. #959: same story for the SOCKS-proxy class
# ("Using SOCKS proxy, but the 'socksio' package is not installed").
# Appending the shared hints HERE covers every route that can leak a
# model-load/download error (generate, dub, archetypes, …), not just TTS
# generate. append_hint is a no-op for every other error and never raises.
from core.failure import append_hint
return JSONResponse(
{"detail": str(exc), "error_class": _entry.get("error_class")},
{"detail": append_hint(str(exc)), "error_class": _entry.get("error_class")},
status_code=500,
headers=headers,
)
@@ -740,6 +972,20 @@ app.add_middleware(NetworkAccessMiddleware)
# keyed non-loopback client must reach them.
app.add_middleware(BearerKeyMiddleware)
# Register canonical audio MIME types before any StaticFiles mount.
# Python's `mimetypes.guess_type()` returns `audio/x-wav` for `.wav` and
# `audio/x-flac` for `.flac` on most platforms — these are vendor-experimental
# (x- prefix, never IANA-registered). macOS Chrome/Safari MIME-sniff leniently
# via CoreAudio so playback works there, but Linux Chrome/Firefox (FFmpeg) and
# Android Chrome (ExoPlayer) strictly honor the declared type and treat the
# x- variants as download-only — manifesting as the play button silently
# doing nothing in the browser app while working in the Tauri desktop shell.
# `audio/wav` / `audio/flac` are the IANA-canonical types.
# Ref: https://www.iana.org/assignments/media-types/media-types.xhtml#audio
import mimetypes as _mimetypes
_mimetypes.add_type("audio/wav", ".wav")
_mimetypes.add_type("audio/flac", ".flac")
app.mount("/audio", StaticFiles(directory=OUTPUTS_DIR), name="audio")
app.mount("/voice_audio", StaticFiles(directory=VOICES_DIR), name="voice_audio")
@@ -789,6 +1035,7 @@ app.include_router(watermark.router)
app.include_router(events.router)
app.include_router(capture.router)
app.include_router(capture_ws.router)
app.include_router(dictation.router)
app.include_router(openai_compat.router)
app.include_router(tts_stream.router)
app.include_router(marketplace.router)
@@ -796,6 +1043,7 @@ app.include_router(personas.router)
app.include_router(sonitranslate.router)
app.include_router(audiobook.router)
app.include_router(longform_jobs.router)
app.include_router(pronunciation.router) # Expressive-TTS Spec 01: pronunciation dictionary
app.include_router(settings_router.router) # Phase 1 AUTH-03 endpoints
from api.routers import mcp_bindings as _mcp_bindings_router # noqa: E402
app.include_router(_mcp_bindings_router.router) # Wave 2.2 per-agent voice bindings
+7 -1
View File
@@ -17,7 +17,13 @@ from core.config import DB_PATH # noqa: E402 — backend/ is on sys.path via al
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# `disable_existing_loggers=False` is deliberate: this env runs *inside* the
# live app (startup `alembic upgrade head`), so the default (True) would
# disable every already-created application logger — e.g. silence
# `omnivoice.db.backup`'s "Skipping pre-migration DB backup" line and the
# rest of the app's logging for the remainder of the process. A migration
# must never mute the app (or leak that mute across a test session).
fileConfig(config.config_file_name, disable_existing_loggers=False)
# SQLite file URL. Honour an externally-set URL (tests pass one via
# `cfg.set_main_option("sqlalchemy.url", ...)` to point at a fixture DB),
@@ -0,0 +1,36 @@
"""Heal voice_profiles.instruct poisoned with the "[object Object]" sentinel.
Revision ID: 0006_strip_object_object_instruct
Revises: 0005_unified_profiles
Create Date: 2026-06-20 00:00:00.000000
A pre-fix Voice Studio build ("Save design as profile") passed the
``buildDesignInstruct()`` *object* straight to FormData, which string-coerced it
to the literal ``"[object Object]"`` and persisted that into
``voice_profiles.instruct`` (#550 #545 #542 #537 #530 #525). On first
preview/use that value fails the engine instruct validator with a 400. The
frontend + backend fixes stop any NEW poisoned rows; this migration heals the
ones already saved on the buggy build (the local-first backward-compat rule
existing project data must keep working without manual migration).
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import inspect
revision: str = "0006_strip_object_object_instruct"
down_revision: Union[str, None] = "0005_unified_profiles"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
bind = op.get_bind()
if "voice_profiles" in inspect(bind).get_table_names():
# Idempotent: only touches rows whose instruct is literally the sentinel.
op.execute("UPDATE voice_profiles SET instruct='' WHERE instruct='[object Object]'")
def downgrade() -> None:
# Irreversible heal — the original garbage sentinel is not worth restoring.
pass
@@ -0,0 +1,118 @@
"""Rebuild design-profile instructs poisoned with prose / "[object Object]".
Revision ID: 0007_rebuild_poisoned_design_instruct
Revises: 0006_strip_object_object_instruct
Create Date: 2026-06-22 00:00:00.000000
Migration 0006 *blanked* the literal ``"[object Object]"`` sentinel. That stops
the 400 on use, but it also throws away the designed voice: a row that read
``"[object Object]"`` (or freeform prose like "A gentle, quiet male voice…")
becomes ``instruct=''`` and then renders with the engine's neutral default —
which is why an Indonesian *female* designed voice came out *male* (#594), and
why prose-poisoned designs still 400 (#571 #596).
This migration heals it properly: for every design profile it recomputes a
validator-safe instruct, preferring any whitelist tags already in the stored
value and otherwise rebuilding the tags from ``vd_states`` (the authoritative
categorypick map the Voice Design picker persists). Non-design rows simply get
their instruct sanitized (poison dropped). Idempotent a healthy row is left
byte-for-byte unchanged, so re-running is a no-op.
Self-contained by design: alembic migrations must not import evolving app code
(``omnivoice`` would also drag in torch at startup), so the tag whitelist is a
frozen snapshot of ``omnivoice.utils.voice_design._INSTRUCT_ALL_VALID``.
``tests/test_migration_0007_instruct_rebuild.py`` asserts the snapshot stays in
sync with the canonical set.
"""
import json
import re
from typing import Sequence, Union
from alembic import op
from sqlalchemy import inspect
revision: str = "0007_rebuild_poisoned_design_instruct"
down_revision: Union[str, None] = "0006_strip_object_object_instruct"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# Frozen snapshot of the design-instruct whitelist + mutually-exclusive
# categories (omnivoice/utils/voice_design.py). Kept self-contained so the
# migration's behaviour is pinned to the data it heals, not to future vocab
# edits. Parity is guarded by the migration test.
_CATEGORIES = [
{"male", "", "female", ""},
{"child", "teenager", "young adult", "middle-aged", "elderly",
"儿童", "少年", "青年", "中年", "老年"},
{"very low pitch", "low pitch", "moderate pitch", "high pitch", "very high pitch",
"极低音调", "低音调", "中音调", "高音调", "极高音调"},
{"whisper", "耳语"},
{"american accent", "british accent", "australian accent", "chinese accent",
"canadian accent", "indian accent", "korean accent", "portuguese accent",
"russian accent", "japanese accent"},
{"河南话", "陕西话", "四川话", "贵州话", "云南话", "桂林话",
"济南话", "石家庄话", "甘肃话", "宁夏话", "青岛话", "东北话"},
]
_ALL_VALID = set().union(*_CATEGORIES)
def _valid_from_items(items) -> str:
"""One whitelist tag per category, first-seen order; everything else dropped."""
seen = set()
out = []
for raw in items:
tag = str(raw if raw is not None else "").strip().lower()
if not tag or tag not in _ALL_VALID:
continue
ci = next((i for i, c in enumerate(_CATEGORIES) if tag in c), -1)
if ci in seen:
continue
seen.add(ci)
out.append(tag)
return ", ".join(out)
def _heal(instruct, vd_states, is_design) -> str:
healed = _valid_from_items(re.split(r"\s*[,]\s*", str(instruct or "").strip()))
if healed or not is_design:
return healed
# Stored instruct was all-poison — recover the design from vd_states.
if not vd_states:
return ""
try:
vd = json.loads(vd_states)
except (ValueError, TypeError):
return ""
return _valid_from_items(vd.values()) if isinstance(vd, dict) else ""
def upgrade() -> None:
bind = op.get_bind()
insp = inspect(bind)
if "voice_profiles" not in insp.get_table_names():
return
cols = {c["name"] for c in insp.get_columns("voice_profiles")}
has_kind = "kind" in cols
has_vd = "vd_states" in cols
select = "SELECT id, instruct"
select += ", kind" if has_kind else ""
select += ", vd_states" if has_vd else ""
select += " FROM voice_profiles"
for row in bind.exec_driver_sql(select).mappings().all():
instruct = row["instruct"] or ""
is_design = (row["kind"] == "design") if has_kind else bool(instruct)
vd = row["vd_states"] if has_vd else None
healed = _heal(instruct, vd, is_design)
if healed != instruct:
bind.exec_driver_sql(
"UPDATE voice_profiles SET instruct = ? WHERE id = ?",
(healed, row["id"]),
)
def downgrade() -> None:
# Irreversible heal — the original poisoned value isn't worth restoring.
pass
@@ -0,0 +1,67 @@
"""Expressive-TTS Spec 01 Phase 1: user pronunciation dictionary
Revision ID: 0008_pronunciation_dictionary
Revises: 0007_rebuild_poisoned_design_instruct
Create Date: 2026-06-25 00:00:00.000000
Adds the ``pronunciation_entries`` table backing the user-editable, per-language
pronunciation dictionary (Settings Pronunciation). Each row maps a ``term`` to
a ``replacement`` the engine pronounces correctly, scoped global (``language='*'``)
or to a 2-letter language. Applied as pure text substitution before synthesis, so
every engine honors it.
* ``id`` TEXT PRIMARY KEY stable row id.
* ``term`` TEXT the word/phrase to match (whole-word, case-insensitive).
* ``replacement`` TEXT the respelling (or, for phoneme rows, the markup).
* ``type`` TEXT 'respelling' | 'ipa' | 'cmu'.
* ``language`` TEXT '*' = global, else a language code (e.g. 'en', 'de').
* ``enabled`` INTEGER 1 = applied, 0 = parked.
* ``created_at`` REAL.
Additive + idempotent (guarded by sqlite_master), matching 0002/0003/0004, so
re-running on a fresh-install DB where ``_BASE_SCHEMA`` already created the table
is a no-op (Backward-compatible project data constraint). The same table is
mirrored into ``core/db.py::_BASE_SCHEMA`` so fresh installs and migrated DBs
converge on an identical end-state (the dual-path discipline).
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "0008_pronunciation_dictionary"
down_revision: Union[str, None] = "0007_rebuild_poisoned_design_instruct"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _has_table(name: str) -> bool:
bind = op.get_bind()
row = bind.execute(
sa.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:n"),
{"n": name},
).fetchone()
return row is not None
def upgrade() -> None:
if _has_table("pronunciation_entries"):
return
op.create_table(
"pronunciation_entries",
sa.Column("id", sa.Text(), primary_key=True),
sa.Column("term", sa.Text(), nullable=False),
sa.Column("replacement", sa.Text(), nullable=False, server_default=""),
sa.Column("type", sa.Text(), nullable=False, server_default="respelling"),
sa.Column("language", sa.Text(), nullable=False, server_default="*"),
sa.Column("enabled", sa.Integer(), nullable=False, server_default="1"),
sa.Column("created_at", sa.Float(), nullable=True),
)
op.create_index("idx_pron_lang", "pronunciation_entries", ["language"])
def downgrade() -> None:
if _has_table("pronunciation_entries"):
op.drop_index("idx_pron_lang", table_name="pronunciation_entries")
op.drop_table("pronunciation_entries")
+1 -1
View File
@@ -129,7 +129,7 @@ class TranslateRequest(BaseModel):
provider: Optional[str] = None
source_lang: Optional[str] = None # ISO 639-1; overrides job detection
job_id: Optional[str] = None # Dub job id, used to resolve detected source_lang
quality: Optional[str] = "fast" # "fast" (one-shot) | "cinematic" (reflectadapt)
quality: Optional[str] = "fast" # "fast" (one-shot) | "cinematic" (reflectadapt) | "autofit" (cinematic + strict fit-to-slot)
glossary: Optional[List[dict]] = None # [{"source": "...", "target": "...", "note": "..."}]
# Optional regional dialect (BCP-47, e.g. "es-AR", "pt-BR") — #280 item 2.
# Applied by LLM-backed paths (provider="openai" or quality="cinematic"):
File diff suppressed because it is too large Load Diff
+23 -18
View File
@@ -1,9 +1,12 @@
"""
Audio DSP pipeline broadcast-grade mastering + configurable effects chain.
The default `apply_mastering()` is the same chain shipped since v0.1.0
(highpass + compressor + light reverb). The new `apply_effects_chain()`
lets callers build custom pipelines from a list of named effects.
`apply_mastering()` is the shared pre-stage that runs before the user's
effect preset: highpass + gentle compression only (see `MASTERING_CHAIN`).
Reverb is deliberately NOT part of it it is preset-declared only (e.g.
cinematic, warm); a hidden reverb here used to bake echo into every non-raw
synthesis, which field reports flagged. `apply_effects_chain()` lets callers
build custom pipelines from a list of named effects.
All effects use Spotify's `pedalboard` library. When pedalboard isn't
installed, every function degrades gracefully (returns audio unmodified).
@@ -97,24 +100,26 @@ def get_effect_chain(preset_id: str) -> list[dict]:
# ── Core DSP functions ──────────────────────────────────────────────────
#: Shared pre-preset mastering stage: highpass + gentle compression ONLY.
#: Reverb must never live here — a hidden Reverb in this chain baked echo
#: into every non-raw synthesis regardless of the chosen preset (field
#: reports of echoey voices; the podcast preset even promises "no reverb").
#: Reverb is preset-declared only (see EFFECT_PRESETS: cinematic, warm).
MASTERING_CHAIN = [
{"type": "highpass", "cutoff_hz": 60},
{"type": "compressor", "threshold_db": -15, "ratio": 1.5, "attack_ms": 2.0, "release_ms": 100},
]
def apply_mastering(audio_tensor, sample_rate=24000):
"""Applies professional Broadcast-grade DSP (EQ, Compressor, light Reverb) to the clone voice."""
"""Applies the broadcast pre-stage (highpass + gentle compression) to the clone voice.
Reverb is intentionally absent only user-chosen effect presets declare
it. Degrades gracefully: pedalboard missing or any DSP error returns the
input unmodified.
"""
try:
from pedalboard import Pedalboard, Compressor, Reverb, HighpassFilter
import numpy as np
board = Pedalboard([
HighpassFilter(cutoff_frequency_hz=60),
Compressor(threshold_db=-15, ratio=1.5, attack_ms=2.0, release_ms=100),
Reverb(room_size=0.10, wet_level=0.08, dry_level=0.95)
])
audio_np = audio_tensor.cpu().numpy()
if audio_np.ndim == 1:
audio_np = audio_np[np.newaxis, :]
effected = board(audio_np, sample_rate, reset=False)
return torch.from_numpy(effected).to(audio_tensor.device)
except ImportError:
return audio_tensor # Fail gracefully if pedalboard isn't installed
return apply_effects_chain(audio_tensor, sample_rate, MASTERING_CHAIN)
except Exception as e:
logger.warning("Mastering DSP Error: %s", e)
return audio_tensor
+26 -9
View File
@@ -65,10 +65,14 @@ class AudiobookPlan:
def char_count(self) -> int:
return sum(c.char_count for c in self.chapters)
@property
def chapter_count(self) -> int:
return len(self.chapters)
def to_dict(self) -> dict:
return {
"chapters": [c.to_dict() for c in self.chapters],
"chapter_count": len(self.chapters),
"chapter_count": self.chapter_count,
"char_count": self.char_count,
}
@@ -100,9 +104,10 @@ def synthesize_chapter(
):
"""Render a chapter's spans to one waveform via an injected ``synth``.
``synth(text, voice_id, speed)`` returns a 1-D float32 audio tensor for a
span of text in the given voice (``speed`` may be ``None`` for the engine
default). Long spans are split with the ``chunked_tts`` splitter and
``synth(text, voice_id, speed)`` returns a float32 audio tensor 1-D
``(samples,)`` or ``(channels, samples)``; real engines emit ``(1, samples)``
per the ``TTSBackend`` contract (#897) — for a span of text in the given
voice (``speed`` may be ``None`` for the engine default). Long spans are split with the ``chunked_tts`` splitter and
crossfaded; inter-span ``pause_ms_after`` becomes silence. ``lexicon`` (when
given) respells each span's text before chunking so the engine pronounces
tricky words correctly; a ``None``/empty lexicon is a no-op pass-through.
@@ -114,23 +119,35 @@ def synthesize_chapter(
from services.chunked_tts import concatenate_audio_chunks, split_text_into_chunks
from services.pronunciation import apply_lexicon
parts: list = []
items: list = [] # ("a", tensor) for audio, ("s", n_samples) for silence
for span in spans:
if span.text:
chunks = split_text_into_chunks(apply_lexicon(span.text, lexicon))
rendered = [synth(c, span.voice_id, span.speed) for c in chunks]
rendered = [r for r in rendered if r is not None and getattr(r, "numel", lambda: 0)()]
if len(rendered) == 1:
parts.append(rendered[0])
items.append(("a", rendered[0]))
elif rendered:
parts.append(concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms))
items.append(("a", concatenate_audio_chunks(rendered, sample_rate, crossfade_ms=crossfade_ms)))
if span.pause_ms_after > 0:
n = int(sample_rate * span.pause_ms_after / 1000.0)
if n > 0:
parts.append(torch.zeros(n, dtype=torch.float32))
items.append(("s", n))
if not parts:
if not items:
return torch.zeros(0, dtype=torch.float32), 0.0
# Engines return (1, samples) per the TTSBackend contract while a bare
# zeros(n) is 1-D — mixing the two crashed the final concat (#897). So
# materialize inter-span silence AFTER the loop, matching the rendered
# audio's channel dims / dtype / device (same pattern as generation.py's
# _render_with_pauses). A silence-only chapter stays 1-D float32 as before.
ref = next((t for kind, t in items if kind == "a"), None)
parts: list = [
val if kind == "a"
else (torch.zeros(val, dtype=torch.float32) if ref is None
else torch.zeros(*ref.shape[:-1], val, dtype=ref.dtype, device=ref.device))
for kind, val in items
]
# Hard-concat spans + silences (crossfading silence would bleed the gap).
audio = parts[0] if len(parts) == 1 else concatenate_audio_chunks(parts, sample_rate, crossfade_ms=0)
return audio, audio.shape[-1] / float(sample_rate)
+75 -2
View File
@@ -42,6 +42,46 @@ _ABBREVIATIONS = frozenset({
# [pause 300ms] markers). The splitter must never cut inside one.
_BRACKET_TAG_RE = re.compile(r"\[[^\]]*\]")
# Dense scripts (CJK ideographs, kana, Hangul) where ~1 character = 1 syllable,
# so an N-char chunk is far more *speech* than N Latin chars. Counted by code
# point (see _dense_char_count) so there are no literal CJK chars in source.
def _dense_char_count(text: str) -> int:
"""Number of CJK / kana / Hangul characters in *text* (dense scripts)."""
n = 0
for ch in text:
o = ord(ch)
if (0x3040 <= o <= 0x30FF or 0x3400 <= o <= 0x4DBF
or 0x4E00 <= o <= 0x9FFF or 0xAC00 <= o <= 0xD7AF
or 0xF900 <= o <= 0xFAFF):
n += 1
return n
# A chunk that is predominantly dense-script (>= this fraction) gets the smaller
# limit; below it, the text is mostly spaced/Latin and the full limit applies.
_DENSE_FRACTION_THRESHOLD = 0.3
# Speech-per-char multiplier for dense scripts vs Latin (~1 ideograph ≈ 2.5
# Latin chars of audio). Used to scale the char limit down.
_DENSE_SPEECH_FACTOR = 2.5
def _effective_max_chars(text: str, max_chars: int) -> int:
"""Scale *max_chars* down for dense-script text (#505).
Long-form (5+ min) generation degrades repeated / skipped / mispronounced
words when a single chunk's acoustic sequence gets too long. With CJK /
kana / Hangul, ~1 char = 1 syllable, so an 800-char chunk is ~4-5 minutes of
audio in one shot, well past the model's reliable range. When a chunk is
predominantly dense-script, cap it to ``max_chars / _DENSE_SPEECH_FACTOR``
(floored) so each chunk's spoken length stays bounded. Latin / spaced text
is unchanged. ``max_chars <= 0`` (chunking disabled) is left untouched.
"""
if max_chars <= 0 or not text:
return max_chars
dense = _dense_char_count(text)
if dense and dense / len(text) >= _DENSE_FRACTION_THRESHOLD:
return max(120, min(max_chars, round(max_chars / _DENSE_SPEECH_FACTOR)))
return max_chars
def split_text_into_chunks(text: str, max_chars: int = DEFAULT_MAX_CHUNK_CHARS) -> List[str]:
"""Split *text* at natural boundaries into chunks of at most *max_chars*.
@@ -54,6 +94,9 @@ def split_text_into_chunks(text: str, max_chars: int = DEFAULT_MAX_CHUNK_CHARS)
text = text.strip()
if not text:
return []
# #505: dense-script text packs far more speech per char, so cap the chunk
# smaller to keep each chunk's spoken length in the model's reliable range.
max_chars = _effective_max_chars(text, max_chars)
if max_chars <= 0 or len(text) <= max_chars:
return [text]
@@ -140,14 +183,43 @@ def _safe_hard_cut(segment: str, max_chars: int) -> int:
return cut
def _normalize_chunk_shapes(chunks: list) -> list:
"""Coerce mixed-rank / mixed-channel chunks to one concat-compatible shape.
Engines return ``(1, samples)`` per the ``TTSBackend.generate`` contract,
but silence buffers and some model paths hand over bare ``(samples,)``
tensors ``torch.cat`` then dies with "Tensors must have same number of
dimensions" (#897). Promote lower-rank chunks with leading singleton dims
to the highest rank present, then broadcast singleton channel dims up to
the widest channel count (mono follows stereo). Rank-homogeneous,
channel-homogeneous input is returned untouched, so all-1-D / all-2-D
callers keep their exact output shape; a genuine channel conflict
(e.g. 2 vs 3 channels) still raises, which is the honest outcome.
"""
target = max(c.dim() for c in chunks)
if any(c.dim() != target for c in chunks):
promoted = []
for c in chunks:
while c.dim() < target:
c = c.unsqueeze(0)
promoted.append(c)
chunks = promoted
if target > 1:
lead = tuple(max(c.shape[i] for c in chunks) for i in range(target - 1))
chunks = [c if tuple(c.shape[:-1]) == lead else c.expand(*lead, -1)
for c in chunks]
return chunks
def concatenate_audio_chunks(chunks: list, sample_rate: int,
crossfade_ms: int = DEFAULT_CROSSFADE_MS):
"""Join per-chunk waveforms with a linear crossfade on the sample axis.
``chunks`` are torch tensors as returned by the engine (1-D, or N-D with
samples on the last axis matching what ``_render_with_pauses`` handles).
Crossfade overlap is clamped to the shorter neighbor; ``crossfade_ms=0``
is a hard concat.
Mixed ranks / mono-vs-multichannel chunks are normalized to one shape
first (#897), so no producer can crash the concat. Crossfade overlap is
clamped to the shorter neighbor; ``crossfade_ms=0`` is a hard concat.
"""
import torch
@@ -156,6 +228,7 @@ def concatenate_audio_chunks(chunks: list, sample_rate: int,
return torch.zeros(1, dtype=torch.float32)
if len(chunks) == 1:
return chunks[0]
chunks = _normalize_chunk_shapes(chunks)
crossfade_samples = int(sample_rate * crossfade_ms / 1000)
result = chunks[0]
+8 -1
View File
@@ -26,6 +26,10 @@ from services.llm_backend import get_active_llm_backend, OffBackend
logger = logging.getLogger("omnivoice.director")
# LLM Skills registry id — Settings → LLM Skills can disable the LLM parse
# or route it to a specific provider. Disabled == the heuristic parser.
_SKILL_ID = "direction_parse"
# ── Taxonomy (stable contract) ──────────────────────────────────────────────
# Additive per dimension — multiple values allowed. Unknown tokens are ignored
@@ -147,7 +151,10 @@ def parse(text: str) -> Direction:
if not text or not text.strip():
return Direction(source=text or "")
llm = get_active_llm_backend()
from services import llm_skills
# `active=` forwards this module's (monkeypatch-able) name so the
# no-override path is byte-identical to the pre-skills behavior.
llm = llm_skills.skill_backend(_SKILL_ID, active=lambda: get_active_llm_backend())
if isinstance(llm, OffBackend):
return _heuristic_parse(text)
+126 -9
View File
@@ -187,6 +187,14 @@ def put_job(job_id: str, job: dict) -> None:
def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0, content_hash: str = "") -> None:
"""Persist dub job state to SQLite so it survives restarts. Uses UPSERT
on `id` so repeated saves in a session keep the latest snapshot.
language / language_code / content_hash only update when the incoming
value is non-empty: the ingest-time insert runs before the target
language is known (both columns ""), generation sets them on the job
dict, and a later save from a job that lost them (e.g. hydrated from an
old row) must not clobber the healed columns back to "". The frontend
keys history restore off language_code, so a frozen "" hid finished
tracks until the user re-picked a language.
"""
try:
segments = job.get("segments") or []
@@ -200,6 +208,8 @@ def save_job(job_id: str, job: dict, filename: str = "", duration: float = 0.0,
filename=excluded.filename,
duration=excluded.duration,
segments_count=excluded.segments_count,
language=CASE WHEN excluded.language != '' THEN excluded.language ELSE dub_history.language END,
language_code=CASE WHEN excluded.language_code != '' THEN excluded.language_code ELSE dub_history.language_code END,
tracks=excluded.tracks,
job_data=excluded.job_data,
content_hash=CASE WHEN excluded.content_hash != '' THEN excluded.content_hash ELSE dub_history.content_hash END""",
@@ -435,6 +445,60 @@ def _ensure_browser_playable_mp4(video_path: str) -> str:
return video_path
# Bounded retry for transient download failures (#579/#598). yt-dlp's own
# `retries`/`fragment_retries` cover per-fragment HTTP flakes, but a broken
# pipe ([Errno 32]) raised while the write side of a pipe closes mid-stream
# (a killed ffmpeg merge child, a CDN reset during muxing) aborts the whole
# `extract_info` call and is NOT covered by them — so a single transient blip
# failed the entire ingest with a raw "Broken pipe". We add a small download-
# level retry on top, cleaning up the partial download between attempts so a
# half-written `original.*` can't poison the next try.
_YT_DOWNLOAD_RETRIES = 2 # total attempts = 1 + retries = 3
def _is_transient_download_error(exc: BaseException) -> bool:
"""True when a download failure is worth retrying (broken pipe / net drop).
Reuses the single failure taxonomy (`VIDEO_DOWNLOAD_NETWORK`) rather than a
parallel keyword list, so "what counts as transient" stays single-sourced
with the error-hint classification. ``BrokenPipeError``/``ConnectionError``
are matched by class too, since a bare instance may be wrapped or re-raised
with a stripped message that no longer contains "broken pipe".
"""
if isinstance(exc, (BrokenPipeError, ConnectionError)):
return True
return failure.classify(str(exc)) == "VIDEO_DOWNLOAD_NETWORK"
# YouTube serves some videos' high-quality formats signature-protected to the
# default player client, so the media download 403s even though extraction
# worked. Forcing an alternate client commonly bypasses it; on a 403 we escalate
# through these (in order) before giving up (#625).
_YT_PLAYER_CLIENTS = ["tv", "android", "web_safari"]
def _is_forbidden_download_error(exc: BaseException) -> bool:
"""True for an HTTP 403 — not transient (the same client keeps 403ing), but
often fixable by switching the YouTube player client."""
s = str(exc)
return "403" in s or "Forbidden" in s
def _cleanup_partial_download(job_dir: str) -> None:
"""Remove any half-written `original.*` files before a retry.
A partial download left on disk would otherwise be picked up as a "finished"
file by the post-download codec probe, or collide with the next attempt's
output. Best-effort never raises on the failure path.
"""
import glob
for stale in glob.glob(os.path.join(job_dir, "original.*")):
try:
os.remove(stale)
except OSError:
pass
def yt_download_sync(
url: str,
job_dir: str,
@@ -480,6 +544,12 @@ def yt_download_sync(
"quiet": True,
"no_warnings": True,
"restrictfilenames": True,
# Don't stamp the downloaded file's mtime with the video's upload date
# (#642): on Windows an out-of-range/invalid timestamp makes the os.utime
# call raise `[Errno 22] Invalid argument`, failing the whole ingest. We
# download to a throwaway `original.*` and never use its mtime, so skip
# it entirely (equivalent to yt-dlp's --no-mtime).
"updatetime": False,
"socket_timeout": 30,
# Resilience against YouTube CDN flakes: a single empty fragment
# (commonly the very last one — "Did not get any data blocks")
@@ -491,17 +561,64 @@ def yt_download_sync(
"extractor_retries": 5,
"skip_unavailable_fragments": True,
}
# #712: the format selector above pulls separate video+audio streams, so
# yt-dlp muxes them via ffmpeg (merge_output_format=mp4). yt-dlp only looks
# for ffmpeg on PATH and aborts with "you have requested merging of multiple
# formats but ffmpeg is not installed" — but OmniVoice's ffmpeg is often a
# bundled Tauri sidecar / imageio-ffmpeg binary that isn't on PATH (common on
# Windows). Point yt-dlp at the exact ffmpeg we resolve so the merge works.
_ffmpeg_bin = find_ffmpeg()
if _ffmpeg_bin:
ydl_opts["ffmpeg_location"] = _ffmpeg_bin
if progress_hook is not None:
ydl_opts["progress_hooks"] = [progress_hook]
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
info = ydl.extract_info(url, download=True)
path = ydl.prepare_filename(info)
root, _ = os.path.splitext(path)
mp4 = root + ".mp4"
if os.path.exists(mp4):
video_path = mp4
else:
video_path = path
# Download with a bounded retry on transient/broken-pipe-class failures
# (#579/#598). A broken pipe mid-mux isn't recoverable inside yt-dlp's own
# fragment retries, but a fresh `extract_info` usually succeeds. Between
# attempts we wipe the partial `original.*` so a half-written file can't be
# mistaken for a finished download.
info = None
path = None
transient_used = 0
client_idx = 0
while True:
try:
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
info = ydl.extract_info(url, download=True)
path = ydl.prepare_filename(info)
break
except Exception as exc:
_cleanup_partial_download(job_dir)
# 403 Forbidden: not transient — escalate the YouTube player client,
# which commonly bypasses a signature-protected format set (#625).
if _is_forbidden_download_error(exc) and client_idx < len(_YT_PLAYER_CLIENTS):
client = _YT_PLAYER_CLIENTS[client_idx]
client_idx += 1
ydl_opts = {**ydl_opts, "extractor_args": {"youtube": {"player_client": [client]}}}
logger.warning(
"Download 403 for %s — retrying with player_client=%s (#625)", url, client,
)
continue
# Transient/broken-pipe: a fresh extract_info usually succeeds
# (#579/#598). A 403 never counts here — it's escalated above.
if (transient_used < _YT_DOWNLOAD_RETRIES
and _is_transient_download_error(exc)
and not _is_forbidden_download_error(exc)):
transient_used += 1
logger.warning(
"Transient download failure for %s (attempt %d/%d): %s — retrying",
url, transient_used, _YT_DOWNLOAD_RETRIES, exc,
)
time.sleep(2 * transient_used) # brief, increasing backoff
continue
raise
root, _ = os.path.splitext(path)
mp4 = root + ".mp4"
if os.path.exists(mp4):
video_path = mp4
else:
video_path = path
# Browser-playability guard: WKWebView (Tauri on macOS) refuses to
# decode VP9/AV1 video and Opus audio even when they're wrapped in an
# mp4 container, and refuses .webm/.mkv outright. We probe the actual
+17 -3
View File
@@ -63,7 +63,21 @@ def resolve_routing(gpu_compat: tuple[str, ...], caps: HostCaps) -> RoutingResul
"routing_reason": _caveat(caps),
}
# 3. Host has an accelerator the engine lacks, but engine supports cpu
# 3. CPU-native engine (declares ONLY cpu) has nothing to fall back FROM,
# so on ANY accelerator host it is benign cpu_only (neutral), never a
# warn-tone "CPU fallback". This must precede the fallback rule below —
# a ("cpu",) engine matches `"cpu" in targets` too, and would otherwise
# be mis-classed cpu_fallback on a GPU/MPS host. (A cpu host reaches
# rule 5 unchanged, keeping its DirectML note.) Engines that *could*
# accelerate elsewhere (e.g. ("cuda", "cpu")) are untouched.
if fam != "cpu" and targets == ("cpu",):
return {
"effective_device": "cpu",
"routing_status": "cpu_only",
"routing_reason": None,
}
# 4. Host has an accelerator the engine lacks, but engine supports cpu
# → the no-silent-fallback signal.
if fam != "cpu" and "cpu" in targets:
if fam == "rocm" and "cuda" in targets and "rocm" not in targets:
@@ -76,7 +90,7 @@ def resolve_routing(gpu_compat: tuple[str, ...], caps: HostCaps) -> RoutingResul
"routing_reason": reason,
}
# 4. Genuine CPU-only host (or DirectML, which the probe reports as cpu)
# 5. Genuine CPU-only host (or DirectML, which the probe reports as cpu)
# and engine supports cpu → benign; must not warn or block.
if fam == "cpu" and "cpu" in targets:
reason = None
@@ -93,7 +107,7 @@ def resolve_routing(gpu_compat: tuple[str, ...], caps: HostCaps) -> RoutingResul
"routing_reason": reason,
}
# 5. Engine needs an accelerator this host lacks and has no cpu path.
# 6. Engine needs an accelerator this host lacks and has no cpu path.
first = targets[0]
return {
"effective_device": first,
+22 -2
View File
@@ -49,7 +49,7 @@ def _canon_value(field: str, value):
return value
def segment_fingerprint(seg: dict) -> str:
def segment_fingerprint(seg: dict, track_lang: str | None = None) -> str:
"""Deterministic hash of the inputs that actually affect TTS output.
Any change to `_GEN_INPUT_FIELDS` flips the hash and the segment becomes
@@ -61,8 +61,20 @@ def segment_fingerprint(seg: dict) -> str:
so a fingerprint computed from the generate request (server defaults
filled in) matches one recomputed later from the client's raw segment
state the root cause of #281's "1 edit re-dubs all N lines".
``track_lang`` (P1.3) is the TRACK's language code (`req.language_code`,
e.g. "es"). It is part of the fingerprint because the same segment text
renders different audio per language without it, a bn hash could
vouch for an es WAV on a multi-track job. It is only mixed in when
provided, so hashes computed by legacy callers (and hashes stored by
previous builds, which never carried a language) keep their old values;
a legacy hash therefore never matches a lang-scoped fingerprint and the
segment reads as stale the safe direction (one clean regen, never a
wrong-language splice).
"""
payload = {k: _canon_value(k, seg.get(k)) for k in _GEN_INPUT_FIELDS}
if track_lang:
payload["track_lang"] = str(track_lang)
blob = json.dumps(payload, sort_keys=True, ensure_ascii=False)
return hashlib.sha1(blob.encode("utf-8"), usedforsecurity=False).hexdigest()[:16]
@@ -120,6 +132,7 @@ def plan_incremental(
segments: list[dict],
*,
stored_hashes: dict[str, str] | None = None,
track_lang: str | None = None,
) -> dict:
"""Return `{stale, fresh, total, fingerprints}` where:
@@ -133,6 +146,13 @@ def plan_incremental(
`stored_hashes` may come from the caller's own bookkeeping (e.g. the
`dub_history.job_data["seg_hashes"]` we'll start writing in Phase 4.5).
When missing, every segment is considered stale (first run).
`track_lang` (P1.3) scopes the plan to ONE dub track: pass the track's
language code together with THAT language's stored hashes
(`job_data["seg_hashes_by_lang"][lang]`) so staleness is judged against
the active track, never against whatever language was generated last.
Must match the language the generate run hashed with, or every segment
reads stale (#281 parity class).
"""
stored = stored_hashes or {}
stale: list[str] = []
@@ -142,7 +162,7 @@ def plan_incremental(
sid = str(seg.get("id", ""))
if not sid:
continue
fp = segment_fingerprint(seg)
fp = segment_fingerprint(seg, track_lang=track_lang)
fingerprints[sid] = fp
prev = stored.get(sid)
if prev == fp:
+56 -21
View File
@@ -54,9 +54,12 @@ class LLMBackend(ABC):
def model_name(self) -> str: ...
@abstractmethod
def chat(self, *, system: str, user: str, timeout: Optional[float] = None) -> str:
def chat(self, *, system: str, user: str, timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
"""One-shot chat completion. Returns the assistant content string.
Raises on failure callers decide whether to fallback gracefully.
``temperature`` is only sent to the provider when set callers that
leave it None keep the provider default (existing behavior).
"""
@@ -67,8 +70,18 @@ class OpenAICompatBackend(LLMBackend):
id = "openai-compat"
display_name = "OpenAI-compatible (real OpenAI, Ollama, LM Studio, …)"
def __init__(self):
def __init__(self, provider=None):
"""``provider``: optional ``llm_providers.Provider`` to bind this
instance to (LLM Skills per-skill routing). None keeps the historical
behavior resolve the ACTIVE provider at call time."""
self._client = None
self._provider = provider
def _resolve_provider(self):
if self._provider is not None:
return self._provider
from services import llm_providers
return llm_providers.active_provider()
@classmethod
def is_available(cls) -> tuple[bool, str]:
@@ -76,67 +89,89 @@ class OpenAICompatBackend(LLMBackend):
import openai # noqa: F401
except ImportError:
return False, "openai package missing (install with `pip install openai`)."
base_url = os.environ.get("TRANSLATE_BASE_URL")
api_key = (
os.environ.get("TRANSLATE_API_KEY")
or os.environ.get("OPENAI_API_KEY")
or ("local" if base_url else None)
)
if not api_key:
# Resolve through the provider registry — the active provider carries
# its own base_url/key/model. Legacy single-endpoint setups (a lone
# TRANSLATE_BASE_URL) resolve to the "custom" provider, so this stays
# backward-compatible with pre-registry configs.
from services import llm_providers
p = llm_providers.active_provider()
if p is None:
return False, (
"No LLM configured. Set TRANSLATE_BASE_URL (+ TRANSLATE_API_KEY) to "
"point at OpenAI, Ollama (http://localhost:11434/v1), or any compatible host."
"No LLM configured. Add a provider key in Settings → LLM Providers "
"(OpenAI/OpenRouter/Groq/… or a local Ollama), or set "
"TRANSLATE_BASE_URL (+ TRANSLATE_API_KEY)."
)
return True, "ready"
if not llm_providers.resolve_base_url(p):
return False, f"{p.display_name}: set a Base URL in Settings → LLM Providers."
if not llm_providers.has_key(p):
return False, f"{p.display_name}: add an API key in Settings → LLM Providers."
return True, f"ready ({p.display_name})"
@property
def model_name(self) -> str:
from services import llm_providers
p = self._resolve_provider()
if p is not None:
return llm_providers.resolve_model(p)
return os.environ.get("TRANSLATE_MODEL", "gpt-4o-mini")
def _get_client(self):
if self._client is not None:
return self._client
from openai import OpenAI
base_url = os.environ.get("TRANSLATE_BASE_URL")
api_key = (
os.environ.get("TRANSLATE_API_KEY")
or os.environ.get("OPENAI_API_KEY")
or ("local" if base_url else None)
)
from services import llm_providers
p = self._resolve_provider()
if p is None:
raise RuntimeError("LLM not configured. See `is_available()` for the hint.")
base_url = llm_providers.resolve_base_url(p)
api_key = llm_providers.resolve_api_key(p)
if not api_key:
raise RuntimeError("LLM not configured. See `is_available()` for the hint.")
kw = {"api_key": api_key}
if base_url:
kw["base_url"] = base_url
self._client = OpenAI(**kw)
# max_retries=0 so a 429 + Retry-After can't make one chat() sleep
# through the Autofit fit-pass wall-clock budget (speech_rate).
self._client = OpenAI(max_retries=0, **kw)
return self._client
def chat(self, *, system: str, user: str, timeout: Optional[float] = None) -> str:
def chat(self, *, system: str, user: str, timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
return self.chat_messages(
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
],
timeout=timeout,
temperature=temperature,
)
def chat_messages(self, *, messages: list[dict], timeout: Optional[float] = None) -> str:
def chat_messages(self, *, messages: list[dict], timeout: Optional[float] = None,
temperature: Optional[float] = None) -> str:
"""One-shot completion over a full message list.
Additive surface for callers that need structured few-shot turns
(dictation refinement, Wave 2.1) small local models pattern-match
and echo inline examples, so examples must arrive as prior chat
turns, not inside the system prompt.
``temperature`` is only forwarded when set (Cinematic/Autofit pin 0.2
the provider default of 1.0 makes local models drift and invent);
every other caller leaves it None and keeps the provider default.
"""
if timeout is None:
try:
timeout = float(os.environ.get("OMNIVOICE_LLM_TIMEOUT", "45"))
except ValueError:
timeout = 45.0
kw = {}
if temperature is not None:
kw["temperature"] = temperature
res = self._get_client().chat.completions.create(
model=self.model_name,
timeout=timeout,
messages=messages,
**kw,
)
return (res.choices[0].message.content or "").strip()
+437
View File
@@ -0,0 +1,437 @@
"""LLM provider registry — the OpenAI-compatible providers OmniVoice can use
for Cinematic / Autofit translation (and any future LLM feature).
Every provider here speaks the OpenAI chat-completions shape, so a single
client (`llm_backend.OpenAICompatBackend`) drives all of them the only
per-provider differences are ``base_url``, ``model``, and the API key. This
module is the one place that knows those defaults and resolves the live value
for the *active* provider.
Resolution precedence for every field (key / base_url / model), highest first:
1. Environment variable power-user / `.env` override, wins always.
2. Encrypted settings store (UI-entered) `settings_store.get_secret` for
keys, `get_text` for base_url/model overrides.
3. Built-in default from the table below.
Local providers (Ollama, LM Studio) need no key a "local" sentinel is used
so the OpenAI client is happy. This keeps the local-first path fully offline:
nothing is sent anywhere unless the user picks a remote provider *and* a
feature gate (quality="cinematic"/"autofit") fires.
Keys entered in the UI are stored **encrypted** (never in `.env`, never
returned to the client). `.env` keys remain a valid override for CI / power
users.
"""
from __future__ import annotations
import logging
import os
from dataclasses import dataclass
from typing import Optional
logger = logging.getLogger("omnivoice.llm_providers")
# Settings-store row names (non-secret overrides live in the plaintext table;
# keys live in the encrypted secret table under ``llm_key.<id>``).
_ACTIVE_PROVIDER_KEY = "llm.active_provider"
_BASE_URL_KEY = "llm.base_url." # + provider id
_MODEL_KEY = "llm.model." # + provider id
SECRET_PREFIX = "llm_key." # + provider id → settings_store secret name
@dataclass(frozen=True)
class Provider:
id: str
display_name: str
default_base_url: str
default_model: str
# Env var names checked (in order) for the API key. First one set wins.
key_envs: tuple[str, ...] = ()
base_url_env: Optional[str] = None
model_env: Optional[str] = None
local: bool = False # runs on the user's machine → no key, offline
# Key optional when a base_url is set (self-hosted OpenAI-compatible servers
# — vLLM, LM Studio behind a custom URL — often ignore the key). Preserves
# the pre-registry behaviour where a lone TRANSLATE_BASE_URL was usable
# keyless.
key_optional: bool = False
needs_account: bool = False # Cloudflare: base_url needs an account id
account_env: Optional[str] = None
signup_url: str = ""
notes: str = ""
# Order here is the display order in the settings page. OpenAI first (the
# canonical), then the free/fast cloud providers from the shipped .env, then
# the local engines, then Custom.
_PROVIDERS: tuple[Provider, ...] = (
Provider("openai", "OpenAI", "https://api.openai.com/v1", "gpt-4o-mini",
key_envs=("OPENAI_API_KEY", "TRANSLATE_API_KEY"),
base_url_env="OPENAI_BASE_URL", model_env="OPENAI_MODEL",
signup_url="https://platform.openai.com/api-keys",
notes="GPT-4o / o-series. Highest quality; paid."),
Provider("openrouter", "OpenRouter", "https://openrouter.ai/api/v1",
"openai/gpt-4o-mini",
key_envs=("OPENROUTER_API_KEY",), base_url_env="OPENROUTER_BASE_URL",
model_env="OPENROUTER_MODEL",
signup_url="https://openrouter.ai/keys",
notes="One key, hundreds of models incl. free tiers."),
Provider("groq", "Groq", "https://api.groq.com/openai/v1",
"llama-3.3-70b-versatile",
key_envs=("GROQ_API_KEY",), base_url_env="GROQ_BASE_URL",
model_env="GROQ_MODEL", signup_url="https://console.groq.com/keys",
notes="Very fast Llama/Mixtral inference. Generous free tier."),
Provider("cerebras", "Cerebras", "https://api.cerebras.ai/v1",
"llama-3.3-70b",
key_envs=("CEREBRAS_API_KEY",), base_url_env="CEREBRAS_BASE_URL",
model_env="CEREBRAS_MODEL", signup_url="https://cloud.cerebras.ai",
notes="Fastest Llama inference. Free tier."),
Provider("google-ai", "Google AI (Gemini)",
"https://generativelanguage.googleapis.com/v1beta/openai",
"gemini-2.0-flash",
key_envs=("GOOGLE_AI_API_KEY",), base_url_env="GOOGLE_AI_BASE_URL",
model_env="GOOGLE_AI_MODEL",
signup_url="https://aistudio.google.com/app/apikey",
notes="Gemini via OpenAI-compatible endpoint. Free tier."),
Provider("mistral", "Mistral", "https://api.mistral.ai/v1",
"mistral-small-latest",
key_envs=("MISTRAL_API_KEY",), base_url_env="MISTRAL_BASE_URL",
model_env="MISTRAL_MODEL", signup_url="https://console.mistral.ai/api-keys",
notes="Strong multilingual models. Free tier."),
Provider("cohere", "Cohere", "https://api.cohere.ai/compatibility/v1",
"command-r-08-2024",
key_envs=("COHERE_API_KEY",), base_url_env="COHERE_BASE_URL",
model_env="COHERE_MODEL", signup_url="https://dashboard.cohere.com/api-keys",
notes="Command models; good for RAG/translation. Free trial keys."),
Provider("nvidia", "NVIDIA NIM", "https://integrate.api.nvidia.com/v1",
"meta/llama-3.3-70b-instruct",
key_envs=("NVIDIA_API_KEY",), base_url_env="NVIDIA_BASE_URL",
model_env="NVIDIA_MODEL", signup_url="https://build.nvidia.com",
notes="NIM-hosted open models. Free credits."),
Provider("github-models", "GitHub Models",
"https://models.github.ai/inference", "openai/gpt-4o-mini",
key_envs=("GITHUB_MODELS_API_KEY",), base_url_env="GITHUB_MODELS_BASE_URL",
model_env="GITHUB_MODELS_MODEL",
signup_url="https://github.com/settings/tokens",
notes="Uses a GitHub PAT. Free for dev, rate-limited."),
Provider("cloudflare", "Cloudflare Workers AI",
"https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1",
"@cf/meta/llama-3.3-70b-instruct-fp8-fast",
key_envs=("CLOUDFLARE_API_KEY",), base_url_env="CLOUDFLARE_BASE_URL",
model_env="CLOUDFLARE_MODEL", needs_account=True,
account_env="CLOUDFLARE_ACCOUNT_ID",
signup_url="https://dash.cloudflare.com/profile/api-tokens",
notes="Needs an Account ID. Free tier."),
Provider("huggingface", "Hugging Face", "https://router.huggingface.co/v1",
"meta-llama/Llama-3.3-70B-Instruct",
key_envs=("HUGGINGFACE_API_KEY", "HF_TOKEN"),
base_url_env="HUGGINGFACE_BASE_URL", model_env="HUGGINGFACE_MODEL",
signup_url="https://huggingface.co/settings/tokens",
notes="HF Inference router. Reuses your HF token."),
Provider("sambanova", "SambaNova", "https://api.sambanova.ai/v1",
"Meta-Llama-3.3-70B-Instruct",
key_envs=("SAMBANOVA_API_KEY",), base_url_env="SAMBANOVA_BASE_URL",
model_env="SAMBANOVA_MODEL", signup_url="https://cloud.sambanova.ai",
notes="Fast open models. Free tier."),
Provider("siliconflow", "SiliconFlow", "https://api.siliconflow.com/v1",
"Qwen/Qwen2.5-7B-Instruct",
key_envs=("SILICONFLOW_API_KEY",), base_url_env="SILICONFLOW_BASE_URL",
model_env="SILICONFLOW_MODEL", signup_url="https://siliconflow.com",
notes="Qwen/DeepSeek and more. Strong for CJK."),
Provider("ollama", "Ollama (local)", "http://localhost:11434/v1",
"llama3.1", local=True,
base_url_env="OLLAMA_BASE_URL", model_env="OLLAMA_MODEL",
signup_url="https://ollama.com",
notes="Fully offline. Run `ollama pull llama3.1` first."),
Provider("lmstudio", "LM Studio (local)", "http://localhost:1234/v1",
"local-model", local=True,
base_url_env="LMSTUDIO_BASE_URL", model_env="LMSTUDIO_MODEL",
signup_url="https://lmstudio.ai",
notes="Fully offline. Start the LM Studio local server."),
Provider("custom", "Custom (OpenAI-compatible)", "", "",
key_envs=("TRANSLATE_API_KEY",), base_url_env="TRANSLATE_BASE_URL",
model_env="TRANSLATE_MODEL", key_optional=True,
notes="Any OpenAI-compatible host. Set Base URL + Model (+ key)."),
)
_BY_ID: dict[str, Provider] = {p.id: p for p in _PROVIDERS}
def all_providers() -> tuple[Provider, ...]:
return _PROVIDERS
def get_provider(pid: str) -> Optional[Provider]:
return _BY_ID.get(pid)
# ── Field resolution (env → store → default) ──────────────────────────────
def _env_first(names: tuple[str, ...]) -> Optional[str]:
for n in names:
v = os.environ.get(n)
if v:
return v
return None
def resolve_account_id(p: Provider) -> str:
"""The Cloudflare-style account id: env override → stored → empty."""
from services import settings_store
return (
(p.account_env and os.environ.get(p.account_env))
or settings_store.get_text(f"llm.account.{p.id}")
or ""
)
def resolve_base_url(p: Provider, *, substitute: bool = True) -> str:
"""Resolve a provider's base URL (env → stored override → default).
``substitute`` interpolates ``{account_id}`` for account-scoped providers
(Cloudflare) so the *client* gets a working URL. The UI passes
``substitute=False`` so the field shows/saves the raw template baking the
substituted value back into a stored override would freeze the URL and make
later account-id changes silently no-op (the bug this guards against).
"""
from services import settings_store
val = (
(p.base_url_env and os.environ.get(p.base_url_env))
or settings_store.get_text(_BASE_URL_KEY + p.id)
or p.default_base_url
)
if substitute and p.needs_account and val and "{account_id}" in val:
val = val.replace("{account_id}", resolve_account_id(p))
return val or ""
def resolve_model(p: Provider) -> str:
from services import settings_store
return (
(p.model_env and os.environ.get(p.model_env))
or settings_store.get_text(_MODEL_KEY + p.id)
or p.default_model
)
def resolve_api_key(p: Provider) -> Optional[str]:
"""Env key → encrypted stored key → 'local' sentinel for local/keyless."""
from services import settings_store
env_key = _env_first(p.key_envs)
if env_key:
return env_key
stored = settings_store.get_secret(SECRET_PREFIX + p.id)
if stored:
return stored
if p.local or (p.key_optional and resolve_base_url(p)):
return "local" # self-hosted OpenAI-compatible servers ignore the key
return None
def has_key(p: Provider) -> bool:
"""True if a usable key is resolvable (local, or keyless-with-base_url)."""
if p.local:
return True
if _env_first(p.key_envs) or _key_in_store(p.id):
return True
return bool(p.key_optional and resolve_base_url(p))
def _key_in_store(pid: str) -> bool:
from services import settings_store
return (SECRET_PREFIX + pid) in settings_store.list_secret_names()
def is_configured(p: Provider) -> bool:
"""Usable end-to-end: has a base_url (custom needs one set) and a key."""
if not resolve_base_url(p):
return False
return has_key(p)
# ── Active provider selection ─────────────────────────────────────────────
def stored_active_provider_id() -> Optional[str]:
"""The user's explicitly-persisted selection ONLY — no env pin, no legacy
TRANSLATE_* fallback, no auto-detect.
``None`` means the user has never chosen a provider. This is what gates
save-activates in the settings router (#963): an explicit save may claim
the *empty* slot, but must never steal it from a made choice.
"""
from services import settings_store
stored = settings_store.get_text(_ACTIVE_PROVIDER_KEY)
return stored if stored and stored in _BY_ID else None
def active_provider_id() -> Optional[str]:
"""The provider Cinematic/Autofit should use.
Precedence: env ``LLM_DEFAULT_PROVIDER`` stored selection first
configured provider None. Legacy ``TRANSLATE_BASE_URL`` users with no
explicit selection resolve to ``custom`` (its envs are TRANSLATE_*).
"""
env_pick = os.environ.get("LLM_DEFAULT_PROVIDER")
if env_pick and env_pick in _BY_ID:
return env_pick
stored = stored_active_provider_id()
if stored:
return stored
# Legacy: a lone TRANSLATE_BASE_URL means the old single-endpoint setup.
if os.environ.get("TRANSLATE_BASE_URL"):
return "custom"
# Auto-select only a provider with a real key. Local providers (Ollama/
# LM Studio) are *always* "configured" (no key needed) but we must NOT
# assume their server is running — they require an explicit selection.
for p in _PROVIDERS:
if not p.local and is_configured(p):
return p.id
return None
def set_active_provider(pid: str) -> None:
from services import settings_store
if pid not in _BY_ID:
raise ValueError(f"unknown provider {pid!r}")
settings_store.set_text(_ACTIVE_PROVIDER_KEY, pid)
def active_provider() -> Optional[Provider]:
pid = active_provider_id()
return _BY_ID.get(pid) if pid else None
# ── UI + persistence helpers ──────────────────────────────────────────────
def save_key(pid: str, api_key: str) -> None:
"""Persist (encrypted) or clear an API key for a provider."""
from services import settings_store
if pid not in _BY_ID:
raise ValueError(f"unknown provider {pid!r}")
settings_store.set_secret(SECRET_PREFIX + pid, api_key or "")
def save_overrides(pid: str, *, base_url: Optional[str] = None,
model: Optional[str] = None,
account_id: Optional[str] = None) -> None:
from services import settings_store
if pid not in _BY_ID:
raise ValueError(f"unknown provider {pid!r}")
p = _BY_ID[pid]
if base_url is not None:
bu = base_url.strip()
# Never freeze an override that equals the built-in default. Critical
# for account-templated URLs (Cloudflare): persisting the shown value
# would pin the base_url and stop later account-id edits from taking
# effect. Clearing (→ empty) falls the resolver back to the default
# template so substitution stays live. Also self-heals a stale override
# if a provider's default URL changes in a future release.
settings_store.set_text(_BASE_URL_KEY + pid, "" if bu == p.default_base_url else bu)
if model is not None:
settings_store.set_text(_MODEL_KEY + pid, model.strip())
if account_id is not None:
settings_store.set_text(f"llm.account.{pid}", account_id.strip())
def _active_env_pin() -> Optional[str]:
"""The provider id pinned by ``LLM_DEFAULT_PROVIDER`` (if set + valid)."""
pick = os.environ.get("LLM_DEFAULT_PROVIDER")
return pick if pick and pick in _BY_ID else None
def describe(p: Provider) -> dict:
"""Client-safe provider descriptor — NEVER includes the key material.
The ``*_from_env`` booleans mirror ``key_from_env`` so the UI can disable an
env-pinned field (and the make-active button) with an explainer instead of
letting the user edit a value the resolver will silently override. ``base_url``
is the RAW template (``substitute=False``) so an account-scoped default shows
``{account_id}`` rather than a baked-in value; ``account_id`` is returned
separately for account-scoped providers so the field can round-trip.
"""
d = {
"id": p.id,
"display_name": p.display_name,
"local": p.local,
"needs_account": p.needs_account,
"signup_url": p.signup_url,
"notes": p.notes,
"base_url": resolve_base_url(p, substitute=False),
"model": resolve_model(p),
"has_key": has_key(p),
"key_from_env": bool(_env_first(p.key_envs)),
"base_url_from_env": bool(p.base_url_env and os.environ.get(p.base_url_env)),
"model_from_env": bool(p.model_env and os.environ.get(p.model_env)),
"active_from_env": _active_env_pin() is not None,
"configured": is_configured(p),
}
if p.needs_account:
d["account_id"] = resolve_account_id(p)
d["account_from_env"] = bool(p.account_env and os.environ.get(p.account_env))
return d
# ── Legacy TRANSLATE_* prefs migration (#963) ──────────────────────────────
# prefs.json row → the custom-provider field it becomes.
_LEGACY_TRANSLATE_PREFS: tuple[tuple[str, str], ...] = (
("env.TRANSLATE_BASE_URL", "base_url"),
("env.TRANSLATE_MODEL", "model"),
("env.TRANSLATE_API_KEY", "api_key"),
)
def migrate_legacy_translate_prefs() -> bool:
"""Move the retired (≤v0.3.7) Translation-LLM panel's prefs rows into the
``custom`` provider's own settings-store rows, then delete them.
Those ``env.TRANSLATE_*`` rows in prefs.json are re-imported into
``os.environ`` on every launch (main.py), and a live ``TRANSLATE_BASE_URL``
makes :func:`active_provider_id` resolve to ``custom`` ahead of the stored
selection fallbacks silently hijacking the active slot on every restart
(issue #963, "Ollama works until I restart"). Must run BEFORE main.py's
prefsenv import so the rows never reach the environment.
Semantics:
* Each value is copied only where the store has no value yet a user's
later edit of the custom provider always wins over legacy leftovers.
* The prefs row is deleted afterwards either way, so it can never be
re-imported as env again (the migration is one-shot per row).
* Real process env vars are NEVER touched a shell/.env
``TRANSLATE_BASE_URL`` keeps its documented override behavior.
* A row whose store write fails is kept in prefs (it still works via the
env import this launch and the migration retries next launch).
Returns True if any prefs row was migrated/removed.
"""
from core import prefs
from services import settings_store
changed = False
for prefs_key, field in _LEGACY_TRANSLATE_PREFS:
try:
raw = prefs.get(prefs_key)
except Exception:
logger.exception("legacy TRANSLATE prefs read failed (%s)", prefs_key)
return changed
if raw is None:
continue
val = str(raw).strip()
try:
if val:
if field == "base_url":
if not settings_store.get_text(_BASE_URL_KEY + "custom"):
save_overrides("custom", base_url=val)
elif field == "model":
if not settings_store.get_text(_MODEL_KEY + "custom"):
save_overrides("custom", model=val)
else: # api_key — encrypted store, never overwrite an existing one
if not _key_in_store("custom"):
save_key("custom", val)
prefs.delete(prefs_key)
changed = True
except Exception:
# Store not ready (e.g. settings table missing) — keep the prefs
# row so the legacy env import still works and we retry next boot.
logger.exception("legacy TRANSLATE prefs migration failed (%s)", prefs_key)
return changed
+323
View File
@@ -0,0 +1,323 @@
"""LLM Skills registry — per-feature enable/route control for every LLM call.
Every LLM-powered capability ("skill") in the backend is registered here, so
the Settings LLM Skills panel can (a) toggle it and (b) route it to a
specific provider (a local Ollama/LM Studio vs a remote key) instead of
everything riding the one global active provider.
The six consumption points today:
dub_translation api/routers/dub_translate.py (the Dub tab's direct
"LLM" translation engine; provider=openai branch)
cinematic_translation services/translator.py (Cinematic + Autofit
REFLECT/ADAPT rewrite; dub_translate quality gate)
slot_fitting services/speech_rate.py (trim/expand a line to its
time slot; Autofit strict pass + /tools/rate-fit)
glossary_extract api/routers/glossary.py auto-extract
direction_parse services/director.py (natural-language direction
taxonomy tokens; /tools/direction + dub generate)
dictation_refinement services/refinement.py (dictation transcript
cleanup on finals)
Design rules:
* **Disabled == unconfigured.** A disabled skill degrades through the exact
same path the feature takes today when no LLM is configured (Fast
translation fallback, refinement pass-through, heuristic direction parse,
no-llm slot fit, 503 on glossary auto-extract). No new degradation modes.
* **Override > active > none.** A per-skill provider override (persisted in
settings_store) wins over the global active provider. No override the
active provider, resolved exactly as before (so existing setups see zero
behavior change; all skills default to enabled with no override).
* **Persistence** is two plaintext settings rows per skill:
``llm_skill.<id>.enabled`` ("1"/"0", absent = enabled) and
``llm_skill.<id>.provider`` (provider id, absent/empty = active provider).
Keys stay in the provider registry (encrypted) nothing secret here.
* ``OMNIVOICE_LLM_BACKEND=off`` remains the global kill switch: it also
silences skills routed through a per-skill override.
"""
from __future__ import annotations
import logging
import os
from dataclasses import dataclass
from typing import Any, Callable, Optional
logger = logging.getLogger("omnivoice.llm_skills")
_ENABLED_KEY = "llm_skill.{sid}.enabled"
_PROVIDER_KEY = "llm_skill.{sid}.provider"
_UNSET = object()
@dataclass(frozen=True)
class LLMSkill:
"""A registered LLM consumption point. name/description resolve via the
frontend i18n layer (localization hard rule no hardcoded UI text)."""
id: str
name_key: str
description_key: str
def _skill(sid: str) -> LLMSkill:
return LLMSkill(
id=sid,
name_key=f"settings.llmskills_{sid}_name",
description_key=f"settings.llmskills_{sid}_desc",
)
# Display order in the settings panel: the dub pipeline first (translation →
# refine → fit → glossary → direction), then dictation.
_SKILLS: tuple[LLMSkill, ...] = (
_skill("dub_translation"),
_skill("cinematic_translation"),
_skill("slot_fitting"),
_skill("glossary_extract"),
_skill("direction_parse"),
_skill("dictation_refinement"),
)
_BY_ID: dict[str, LLMSkill] = {s.id: s for s in _SKILLS}
def all_skills() -> tuple[LLMSkill, ...]:
return _SKILLS
def get_skill(skill_id: str) -> Optional[LLMSkill]:
return _BY_ID.get(skill_id)
# ── Persistence (settings_store text rows) ─────────────────────────────────
def is_enabled(skill_id: str) -> bool:
"""Skill toggle. Absent row = enabled (all skills default on)."""
from services import settings_store
raw = settings_store.get_text(_ENABLED_KEY.format(sid=skill_id))
return raw != "0"
def provider_override(skill_id: str) -> Optional[str]:
"""The per-skill provider id, or None when the skill follows the active
provider. A stored id that no longer exists in the registry reads as None
(stale override resolution falls back to the active provider)."""
from services import llm_providers, settings_store
raw = (settings_store.get_text(_PROVIDER_KEY.format(sid=skill_id)) or "").strip()
if not raw:
return None
if llm_providers.get_provider(raw) is None:
logger.warning("llm_skills: stale provider override %r on %s — ignoring",
raw, skill_id)
return None
return raw
def configure_skill(skill_id: str, *, enabled: Optional[bool] = None,
provider_override: Any = _UNSET) -> None:
"""Persist a skill's toggle and/or provider routing.
``provider_override``: omit to leave unchanged; ``None``/``""`` clears it
(skill follows the active provider); a provider id routes the skill there.
Raises KeyError for an unknown skill, ValueError for an unknown provider.
"""
if skill_id not in _BY_ID:
raise KeyError(f"unknown LLM skill {skill_id!r}. Known: {sorted(_BY_ID)}")
from services import llm_providers, settings_store
if enabled is not None:
settings_store.set_text(_ENABLED_KEY.format(sid=skill_id),
"1" if enabled else "0")
if provider_override is not _UNSET:
pid = (provider_override or "").strip()
if pid and llm_providers.get_provider(pid) is None:
raise ValueError(f"unknown provider {pid!r}")
settings_store.set_text(_PROVIDER_KEY.format(sid=skill_id), pid)
# ── Resolution (override > active > none) ──────────────────────────────────
@dataclass(frozen=True)
class SkillResolution:
skill: LLMSkill
enabled: bool
provider: Optional[Any] # llm_providers.Provider or None
source: str # "override" | "active" | "none"
ready: bool
reason: Optional[str] # None | "disabled" | "no_provider" | "unconfigured"
def resolve_skill(skill_id: str) -> SkillResolution:
"""Resolve a skill's effective provider + ready status.
Precedence: per-skill override global active provider none. Ready
means enabled AND the effective provider is configured end-to-end.
Raises KeyError for an unknown skill.
"""
skill = _BY_ID.get(skill_id)
if skill is None:
raise KeyError(f"unknown LLM skill {skill_id!r}. Known: {sorted(_BY_ID)}")
from services import llm_providers
enabled = is_enabled(skill_id)
override = provider_override(skill_id)
if override:
provider = llm_providers.get_provider(override)
source = "override"
else:
provider = llm_providers.active_provider()
source = "active" if provider is not None else "none"
if not enabled:
ready, reason = False, "disabled"
elif provider is None:
ready, reason = False, "no_provider"
elif not llm_providers.is_configured(provider):
ready, reason = False, "unconfigured"
else:
ready, reason = True, None
return SkillResolution(skill=skill, enabled=enabled, provider=provider,
source=source, ready=ready, reason=reason)
def effective_provider(skill_id: str) -> Optional[Any]:
"""The provider a skill would call (override or active), or None."""
return resolve_skill(skill_id).provider
# ── Client / backend construction ───────────────────────────────────────────
@dataclass(frozen=True)
class SkillClient:
"""A ready-to-call OpenAI-compatible client bound to the skill's provider."""
client: Any # openai.OpenAI
model: str
provider_id: str
timeout: float
def _default_timeout() -> float:
try:
return float(os.environ.get("OMNIVOICE_LLM_TIMEOUT", "45"))
except ValueError:
return 45.0
def resolve_skill_client(skill_id: str) -> Optional[SkillClient]:
"""OpenAI-compat client + model for a skill, or None.
None when the skill is disabled, no provider resolves, the provider is
unconfigured, or the openai package is missing callers treat None
exactly like "no LLM configured" (their existing degradation path).
Raises KeyError for an unknown skill (programming error, not user state).
"""
res = resolve_skill(skill_id)
if not res.ready:
return None
try:
from openai import OpenAI
except ImportError:
logger.warning("openai package not installed — LLM skill %s unavailable.",
skill_id)
return None
from services import llm_providers
api_key = llm_providers.resolve_api_key(res.provider)
if not api_key:
return None
kw: dict[str, Any] = {"api_key": api_key}
base_url = llm_providers.resolve_base_url(res.provider)
if base_url:
kw["base_url"] = base_url
# max_retries=0: a rate-limited provider returning 429 + a long Retry-After
# would otherwise let the SDK sleep+retry inside a single call, blowing the
# skill's wall-clock budget (the cinematic pass budget, the glossary call
# timeout) from inside one request. Fail fast — the per-call timeout and the
# pass-level budget are the only bounds we want. Mirrors OpenAICompatBackend.
#
# #959 class guard: OpenAI() eagerly builds its httpx client, which can
# raise AT CONSTRUCTION for environment-shaped reasons — the reported one
# is httpx's ImportError under ALL_PROXY/HTTPS_PROXY=socks5:// without
# socksio; a malformed proxy URL or broken cert bundle fails the same way.
# The contract here is already "None == LLM unavailable, degrade" — a bad
# proxy env must degrade the skill, never 500 the calling feature.
try:
client = OpenAI(max_retries=0, **kw)
except Exception as exc:
logger.warning(
"LLM client construction failed for skill %s (provider %s): %s"
"treating the skill as unavailable.",
skill_id, res.provider.id, exc,
)
return None
return SkillClient(
client=client,
model=llm_providers.resolve_model(res.provider),
provider_id=res.provider.id,
timeout=_default_timeout(),
)
def skill_backend(skill_id: str, active: Optional[Callable[[], Any]] = None):
"""LLMBackend for a skill — the drop-in for ``get_active_llm_backend()``.
* disabled skill OffBackend (same object the no-LLM path returns today,
so every caller's ``id == "off"`` / ``isinstance(…, OffBackend)`` check
degrades identically);
* no override the ``active`` callable (callers pass their module-local
``get_active_llm_backend`` so existing monkeypatch seams keep working),
defaulting to ``llm_backend.get_active_llm_backend`` the exact legacy
path, env/prefs overrides included;
* override an OpenAICompatBackend bound to that provider, or OffBackend
when the provider is unconfigured, openai is missing, or the global
``OMNIVOICE_LLM_BACKEND=off`` kill switch is set.
"""
from services.llm_backend import OffBackend, OpenAICompatBackend
res = resolve_skill(skill_id)
if not res.enabled:
return OffBackend()
if res.source != "override":
if active is not None:
return active()
from services import llm_backend
return llm_backend.get_active_llm_backend()
if os.environ.get("OMNIVOICE_LLM_BACKEND") == "off":
return OffBackend()
if not res.ready:
return OffBackend()
try:
import openai # noqa: F401
except ImportError:
return OffBackend()
return OpenAICompatBackend(provider=res.provider)
# ── API descriptor ──────────────────────────────────────────────────────────
def describe(skill_id: str) -> dict:
"""Client-safe skill descriptor for GET /api/settings/llm-skills."""
res = resolve_skill(skill_id)
p = res.provider
return {
"id": res.skill.id,
"name_key": res.skill.name_key,
"description_key": res.skill.description_key,
"enabled": res.enabled,
"provider_override": provider_override(skill_id),
"provider": p.id if p is not None else None,
"provider_display_name": p.display_name if p is not None else None,
"provider_local": p.local if p is not None else None,
"provider_source": res.source,
"ready": res.ready,
"reason": res.reason,
}
+25 -1
View File
@@ -45,11 +45,33 @@ def _asr_device() -> str:
return "cpu"
def _active_tts_id() -> Optional[str]:
"""Configured TTS engine id, or None if it can't be resolved. Attribution
is advisory a prefs/import hiccup must never break /model/loaded."""
try:
from services.tts_backend import active_backend_id
return active_backend_id()
except Exception:
return None
def _tts_attribution(engine_id: str, active: Optional[str]) -> dict:
"""Per-entry engine attribution for TTS-family models. A model can stay
resident in VRAM after the user switches engines (freed only by unload/
idle-evict), so the panel needs to know which entry synthesis actually
routes to. ``is_active_engine`` is None when the active id is unknown."""
return {
"engine_id": engine_id,
"is_active_engine": (engine_id == active) if active is not None else None,
}
def list_loaded() -> dict:
"""Enumerate every currently-loaded model. Shape: ``{"models": [...],
"count": n}`` with per-model id/name/checkpoint/device/vram_mb/unloadable
(+ optional ``note``)."""
models: list[dict] = []
active_tts = _active_tts_id()
# 1. In-process TTS model (OmniVoice)
if mm.model is not None:
@@ -60,10 +82,11 @@ def list_loaded() -> dict:
models.append({
"id": "tts",
"name": "OmniVoice TTS",
"checkpoint": os.environ.get("OMNIVOICE_MODEL", "k2-fsa/OmniVoice"),
"checkpoint": mm.resolve_omnivoice_checkpoint(), # #693: effective checkpoint, not a leaked raw value
"device": device,
"vram_mb": round(_tts_vram_mb(), 1),
"unloadable": True,
**_tts_attribution("omnivoice", active_tts),
})
# 2. ASR (WhisperX) — co-loaded with and released alongside the TTS model.
@@ -105,6 +128,7 @@ def list_loaded() -> dict:
"device": get_best_device(),
"vram_mb": round(float(s.get("vram_mb") or 0), 1),
"unloadable": True,
**_tts_attribution(s["id"], active_tts),
})
except Exception:
pass
+492 -44
View File
@@ -3,7 +3,7 @@ import time
import asyncio
import logging
import threading
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor, Executor
# ── Lazy imports ─────────────────────────────────────────────────────
# torch and OmniVoice are heavy (~2-3s import on Apple Silicon).
@@ -25,7 +25,16 @@ def _lazy_torch():
def _lazy_omnivoice():
global _OmniVoice
if _OmniVoice is None:
from omnivoice.models.omnivoice import OmniVoice as _OV
try:
from omnivoice.models.omnivoice import OmniVoice as _OV
except ModuleNotFoundError:
# The venv's editable install is missing/broken (#564). main.py wires
# the source fallback at startup, but resolve it here too so the
# model-load path self-heals and logs the paths it searched.
from core.omnivoice_path import ensure_omnivoice_importable
_backend_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
ensure_omnivoice_importable(_backend_dir, logger)
from omnivoice.models.omnivoice import OmniVoice as _OV
_OmniVoice = _OV
return _OmniVoice
@@ -35,17 +44,30 @@ from core.config import IDLE_TIMEOUT_SECONDS, CPU_POOL_WORKERS
logger = logging.getLogger("omnivoice.model")
# Per-TTS-job VRAM headroom estimate. OmniVoice's forward + autoregressive
# decode peaks around 1.6 GB on a 24 kHz 8-second utterance; we budget 2.5 GB
# to leave room for the ASR/diarization pipelines that run concurrently in
# the same process. Tuned empirically — bumps to 3 GB if anyone reports OOM
# at 16 GB on a multi-segment dub.
_GPU_VRAM_PER_JOB_GB = 2.5
# decode peaks around 1.6 GB, but the interactive clone path co-loads WhisperX
# large-v3 ASR (~3 GB) to transcribe the reference, so a *concurrent* clone job
# is realistically ~5 GB. The old 2.5 GB budget over-committed: an 8 GB card
# (~7 GB free) got 2 workers, and two concurrent clone jobs blew past VRAM into
# a sticky CUDA "illegal memory access" that aborts the whole backend process —
# the wave of "Can't reach the local backend" crash reports on 8 GB GPUs
# (#567/#570/#571/#580/#582/#583/#584). Budgeting 5 GB serializes to 1 worker on
# ≤10 GB cards (no contention → no crash) while 16/24 GB cards still parallelize.
# Power users override with OMNIVOICE_GPU_WORKERS.
_GPU_VRAM_PER_JOB_GB = 5.0
_GPU_WORKER_CAP = 4
_gpu_pool_singleton: "ThreadPoolExecutor | None" = None
_gpu_pool_singleton: "_ResilientGpuPool | None" = None
_cpu_pool = ThreadPoolExecutor(max_workers=CPU_POOL_WORKERS)
def _workers_for_free_vram(free_gb: float) -> int:
"""GPU worker count for a given free-VRAM figure: free // per-job budget,
floored at 1 and capped at _GPU_WORKER_CAP. Pure so the sizing policy is
unit-tested without a GPU (the #567 crash hinged on this returning >1 on
8 GB cards)."""
return max(1, min(_GPU_WORKER_CAP, int(free_gb // _GPU_VRAM_PER_JOB_GB)))
def _pick_gpu_workers() -> int:
"""Pick a sensible GPU worker count from the runtime environment.
@@ -68,7 +90,7 @@ def _pick_gpu_workers() -> int:
if hasattr(torch, "cuda") and torch.cuda.is_available():
free_bytes, _total = torch.cuda.mem_get_info()
free_gb = free_bytes / (1024 ** 3)
workers = max(1, min(_GPU_WORKER_CAP, int(free_gb // _GPU_VRAM_PER_JOB_GB)))
workers = _workers_for_free_vram(free_gb)
logger.info(
"GPU pool sized to %d worker(s) — %.1f GB free / %.1f GB per job (cap %d)",
workers, free_gb, _GPU_VRAM_PER_JOB_GB, _GPU_WORKER_CAP,
@@ -87,14 +109,82 @@ def _build_gpu_pool() -> ThreadPoolExecutor:
return ThreadPoolExecutor(max_workers=workers, thread_name_prefix="gpu-pool")
def _get_gpu_pool() -> ThreadPoolExecutor:
"""Internal accessor. Same singleton as the module-level `_gpu_pool`
attribute, but resolvable from inside this module (Python's module
`__getattr__` only fires for unresolved lookups from *outside*).
class _ResilientGpuPool(Executor):
"""A stable, self-healing wrapper around the GPU `ThreadPoolExecutor`.
The crash this fixes (#589 #599): `_reset_gpu_pool()` shuts the pool down on
a model-load timeout, but consumers that captured the executor *object* at
import time (`from services.model_manager import _gpu_pool` at module level
generation, dub_generate, dub_core, dub_translate, openai_compat) kept
submitting to the dead pool and got `RuntimeError: cannot schedule new
futures after shutdown` on the next generate/dub/translate.
Making `_gpu_pool` a single long-lived wrapper whose *inner* pool is swapped
means those references never go stale: every `submit()` resolves the live
pool, and a submit that races a shutdown rebuilds once and retries. Building
the inner pool stays lazy so we still size workers after torch's device
probe (the reason for the original `__getattr__` indirection).
"""
def __init__(self):
self._pool: "ThreadPoolExecutor | None" = None
self._lock = threading.Lock()
def _live_pool(self) -> ThreadPoolExecutor:
pool = self._pool
if pool is None:
with self._lock:
if self._pool is None:
self._pool = _build_gpu_pool()
pool = self._pool
return pool
def submit(self, fn, /, *args, **kwargs):
try:
return self._live_pool().submit(fn, *args, **kwargs)
except RuntimeError as e:
# "cannot schedule new futures after shutdown": the inner pool was
# reset (or torn down) under us. Rebuild once and retry so a stale
# caller self-heals instead of 500-ing. (Interpreter-shutdown races
# re-raise on the retry — we don't loop.)
if "shutdown" not in str(e).lower():
raise
with self._lock:
self._pool = _build_gpu_pool()
pool = self._pool
return pool.submit(fn, *args, **kwargs)
def reset(self) -> None:
"""Abandon the current worker pool; the next submit builds a fresh one.
Python can't kill a thread wedged in a timed-out load, but dropping the
poisoned pool means a retry gets a clean worker instead of queueing
behind the wedged one. The wrapper identity is preserved, so references
held by importers stay valid.
"""
with self._lock:
pool, self._pool = self._pool, None
if pool is not None:
try:
pool.shutdown(wait=False, cancel_futures=True)
except Exception:
pass
def shutdown(self, wait=True, *, cancel_futures=False):
with self._lock:
pool, self._pool = self._pool, None
if pool is not None:
pool.shutdown(wait=wait, cancel_futures=cancel_futures)
def _get_gpu_pool() -> "_ResilientGpuPool":
"""Internal accessor for the GPU pool singleton. Same object as the
module-level `_gpu_pool` attribute, but resolvable from inside this module
(Python's module `__getattr__` only fires for lookups from *outside*).
"""
global _gpu_pool_singleton
if _gpu_pool_singleton is None:
_gpu_pool_singleton = _build_gpu_pool()
_gpu_pool_singleton = _ResilientGpuPool()
return _gpu_pool_singleton
@@ -108,6 +198,93 @@ def __getattr__(name: str):
return _get_gpu_pool()
raise AttributeError(f"module 'services.model_manager' has no attribute {name!r}")
# ── GPU-job timeout guard (#730 class; residual #850/#802/#755 …) ─────
# A blocking GPU job that wedges on a Windows+CUDA hang keeps occupying its
# worker forever — run_in_executor can't cancel the thread. With a 12 worker
# pool that starves *every* other request, so the next user action surfaces as
# the misleading "Can't reach the local backend" even though the process is
# alive. ASR/dub/model-load already bound+reset on hang (run_transcribe_guarded,
# _reset_pool_on_wedge, _load_model_with_timeout); the TTS **generate** paths
# (generation.py, tts_stream.py) were the last unguarded dispatch — and the
# residual on-main reports all fail on generate:start (audio). This is the same
# guard generalised so every GPU dispatch shares one recovery path.
GPU_JOB_TIMEOUT_S = float(os.environ.get("OMNIVOICE_GENERATE_TIMEOUT_S", "300.0"))
class GpuJobTimeoutError(TimeoutError):
"""A GPU-pool job exceeded its wall-clock bound and was abandoned.
The backend is alive the job was too heavy for the available compute
(most often a VRAM-starved GPU). Pool capacity is restored automatically by
resetting the pool; the message carries the durable fix.
"""
async def run_on_gpu_pool_guarded(fn, *, what: str = "GPU job",
timeout: float = GPU_JOB_TIMEOUT_S,
executor=None):
"""Run blocking ``fn`` on the GPU pool with a hard wall-clock bound.
On timeout, ``reset()`` the pool (abandon the wedged worker so the next
submit gets a fresh one) and raise :class:`GpuJobTimeoutError`. ``fn`` must
be a zero-arg callable wrap args with ``functools.partial`` at the call
site. Deliberately mirrors ``asr_backend.run_transcribe_guarded`` so every
GPU dispatch shares one bound+recover path (#730 class). Executors without
``reset`` (a plain ThreadPoolExecutor in tests) still get the bound + error.
"""
loop = asyncio.get_running_loop()
ex = executor if executor is not None else _get_gpu_pool()
fut = loop.run_in_executor(ex, fn)
try:
return await asyncio.wait_for(fut, timeout=timeout)
except asyncio.TimeoutError:
_reset = getattr(ex, "reset", None)
if callable(_reset):
try:
_reset()
logger.warning(
"%s exceeded %.0fs — abandoned the GPU-pool worker to "
"restore capacity (#730).", what, timeout,
)
except Exception:
logger.exception("GPU pool reset after %s timeout failed", what)
raise GpuJobTimeoutError(_timeout_guidance(what, timeout))
def _timeout_guidance(what: str, timeout: float) -> str:
"""Device-aware timeout message (#896): a CPU-only host must never be told
to "set the engine to CPU" or blamed on VRAM on CPU the job is simply
compute-bound. GPU hosts keep the VRAM-contention guidance."""
family = "cuda" # conservative default: GPU wording if the probe fails
try:
from core.device_caps import detect_host_caps
family = detect_host_caps().family
except Exception: # noqa: BLE001 — guidance must never mask the timeout
pass
common = (
f"{what} exceeded {timeout:.0f}s and was abandoned — the backend is "
"running, but the job was too heavy for the available compute. "
"Capacity was restored automatically; "
)
if family == "cpu":
return common + (
"this machine renders on CPU, where long generations are "
"compute-bound. For a durable fix try shorter text or a lighter "
"engine (OmniVoice GGUF and Supertonic-3 are CPU-tuned). If you "
"expect very long single generations, raise "
"OMNIVOICE_GENERATE_TIMEOUT_S."
)
return common + (
"most often the GPU is VRAM-starved (a resident model and this job "
"contend for memory). For a durable fix, Flush caches / Unload the "
"resident model (top toolbar or Settings → Models) before retrying, "
"try shorter text, a lighter engine, or set the engine to CPU in "
"Settings → Models. (Raise OMNIVOICE_GENERATE_TIMEOUT_S for very "
"long single generations.)"
)
model = None # type: ignore
_model_lock = asyncio.Lock()
_last_used = time.time()
@@ -218,6 +395,19 @@ def get_best_device():
compatible, warning = check_device_compatibility()
if not compatible:
logger.warning(warning)
# #756: the GPU's compute capability isn't in this torch build's arch
# list, so CUDA kernels can't launch ("no kernel image is available
# for execution") — every generate would 500. Too-old (Pascal sm_61)
# and too-new (Blackwell sm_120 on pre-cu128 wheels) both land here.
# Fall back to CPU so the app WORKS (slowly) instead of dead-ending;
# OMNIVOICE_FORCE_CUDA=1 overrides for users who installed a matching
# torch and know the arch_list probe is wrong for their setup.
if not _env_flag("OMNIVOICE_FORCE_CUDA"):
logger.warning(
"Falling back to CPU: this GPU is unsupported by the installed "
"PyTorch build (set OMNIVOICE_FORCE_CUDA=1 to force CUDA anyway)."
)
return "cpu"
return "cuda"
# ── Intel Arc / discrete GPU via IPEX ────────────────────────────
@@ -449,6 +639,168 @@ def should_preload_tts_asr() -> bool:
return _env_flag("OMNIVOICE_PRELOAD_TTS_ASR")
def _is_incomplete_cache_error(exc: BaseException) -> bool:
"""True when `exc` is the truncated-HF-cache class (#352 / #581).
transformers raises an OSError whose message contains "does not appear to
have a file named " when the on-disk snapshot has config/tokenizer files
but no weight shard the signature of an interrupted download. We match on
that phrase (stable across transformers 4.x/5.x) rather than the error type,
since the same OSError type covers unrelated I/O failures."""
return "does not appear to have a file named" in str(exc)
def _hf_offline() -> bool:
"""Respect HF's offline switches so repair never makes a network call the
user opted out of. `snapshot_download` would itself raise offline, but
checking up front lets us skip straight to the actionable message."""
return _env_flag("HF_HUB_OFFLINE") or _env_flag("TRANSFORMERS_OFFLINE")
# Why the LAST _repair_model_cache run failed ("" when it succeeded / hasn't
# run). #886: the "could not be auto-repaired" message used to drop the cause
# entirely, so a mirror outage, offline mode, or a full disk all read the same.
_last_repair_error: str = ""
def _repair_failure_detail() -> str:
"""One sanitized clause naming why auto-repair failed, or "" (#886).
Feeds user-facing messages (the generate 500 detail / model status), so it
goes through core.failure.sanitize and because the cause text is now part
of the surfaced error, the shared HF-mirror hint (#874) fires on it when
the repair failed against an unreachable configured mirror."""
if not _last_repair_error:
return ""
try:
from core.failure import sanitize
cause = sanitize(_last_repair_error)
except Exception:
cause = _last_repair_error
return f" Auto-repair failed with: {cause}."
def _repair_model_cache(checkpoint: str, *, force: bool = False) -> bool:
"""Re-fetch a checkpoint's missing files in place and report success.
An interrupted download leaves the cache missing only some files;
`snapshot_download` resumes/fills exactly those (already-present, correctly
sized blobs are skipped by hash, so a near-complete cache repairs in
seconds and a complete one would no-op). Returns False leaving the caller
to surface the actionable delete-and-reinstall message when repair is
impossible (offline) or the re-fetch itself fails (no network, gated repo,
full disk). Never raises; repair is best-effort.
``force=True`` passes ``force_download`` so the re-fetch replaces files that
are *present but corrupt* a truncated/garbled blob that still has the right
size won't be re-fetched by the default resume (#739). It re-downloads the
whole snapshot, so it's the last resort the load path only reaches after a
plain resume-repair didn't fix the cache."""
global _last_repair_error
_last_repair_error = ""
if _hf_offline():
logger.warning(
"Model cache for %s is incomplete but HF offline mode is set — "
"cannot auto-repair.", checkpoint,
)
_last_repair_error = (
"Hugging Face offline mode is enabled (HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE)"
)
return False
try:
from huggingface_hub import snapshot_download
except Exception as imp_err: # pragma: no cover - huggingface_hub is a hard dep
logger.warning("Cannot import snapshot_download to repair cache: %s", imp_err)
_last_repair_error = f"{type(imp_err).__name__}: {imp_err}"
return False
dl_kwargs: dict = {"repo_id": checkpoint}
endpoint = os.environ.get("HF_ENDPOINT")
if endpoint:
dl_kwargs["endpoint"] = endpoint
if force:
# Replace present-but-corrupt blobs that resume would trust by size.
dl_kwargs["force_download"] = True
if os.name == "nt":
# Match the install path (download.py): avoid symlinks on Windows.
dl_kwargs["local_dir_use_symlinks"] = False
def _attempt() -> None:
"""One snapshot_download, tolerating an hf_hub that rejects the optional
symlink knob. Lets real failures (network, gated repo, disk) propagate."""
try:
snapshot_download(**dl_kwargs)
except TypeError:
# Older/newer huggingface_hub may not accept local_dir_use_symlinks
# on a cache-only call — retry without the optional knob.
dl_kwargs.pop("local_dir_use_symlinks", None)
snapshot_download(**dl_kwargs)
# Bounded retries (#739): an incomplete cache *is* an interrupted download, so
# a single transient blip mid-repair shouldn't drop the user back to a manual
# delete-and-reinstall. snapshot_download resumes between attempts (present,
# correctly-sized blobs are skipped by hash), so each retry continues where
# the last left off — cheap and idempotent. Counts/backoff are env-tunable
# for restricted networks and kept fast (backoff=0) in tests.
try:
retries = max(1, int(os.environ.get("OMNIVOICE_MODEL_REPAIR_RETRIES", "3")))
except ValueError:
retries = 3
try:
backoff = max(0.0, float(os.environ.get("OMNIVOICE_MODEL_REPAIR_BACKOFF_S", "2")))
except ValueError:
backoff = 2.0
logger.info(
"Auto-repairing incomplete model cache for %s (up to %d attempt(s)) …",
checkpoint, retries,
)
for attempt in range(1, retries + 1):
try:
_attempt()
logger.info("Auto-repair of %s completed; retrying model load.", checkpoint)
return True
except Exception as e:
logger.warning(
"Auto-repair of %s attempt %d/%d failed: %s",
checkpoint, attempt, retries, e,
)
_last_repair_error = f"{type(e).__name__}: {e}"
if attempt < retries and backoff:
time.sleep(backoff * attempt)
return False
_DEFAULT_OMNIVOICE_CHECKPOINT = "k2-fsa/OmniVoice"
def resolve_omnivoice_checkpoint() -> str:
"""Resolve the OmniVoice TTS checkpoint from ``OMNIVOICE_MODEL``, self-healing
a misconfigured value.
A valid checkpoint is either a HuggingFace repo id (``org/repo`` contains a
``/``) or an existing local directory. A bare token like ``"omnivoice"`` a
TTS *engine id* that leaked into ``OMNIVOICE_MODEL`` (e.g. a stale pref/env)
is neither, and would crash model load with *"omnivoice is not a local folder
and is not a valid model identifier listed on huggingface.co/models"* (#693).
Fall back to the default rather than 500 on every launch.
"""
checkpoint = os.environ.get("OMNIVOICE_MODEL", _DEFAULT_OMNIVOICE_CHECKPOINT).strip()
if not checkpoint:
return _DEFAULT_OMNIVOICE_CHECKPOINT
# Honor a HF repo id (org/repo) or an EXPLICIT local path (absolute, or with
# a path separator). A bare token like "omnivoice" must NOT be treated as a
# local dir even if a cwd-relative folder happens to share its name — that
# is exactly the engine-id leak (#693), so self-heal to the default.
if "/" in checkpoint or "\\" in checkpoint or os.path.isabs(checkpoint):
return checkpoint
logger.warning(
"OMNIVOICE_MODEL=%r is not a HuggingFace repo id (org/repo) or a local "
"path — falling back to %s (#693).",
checkpoint, _DEFAULT_OMNIVOICE_CHECKPOINT,
)
return _DEFAULT_OMNIVOICE_CHECKPOINT
def _load_model_sync():
global model
from utils.hf_progress import register_listener, unregister_listener
@@ -475,7 +827,7 @@ def _load_model_sync():
OmniVoice = _lazy_omnivoice()
device = get_best_device()
checkpoint = os.environ.get("OMNIVOICE_MODEL", "k2-fsa/OmniVoice")
checkpoint = resolve_omnivoice_checkpoint()
_set_loading("loading_weights", f"Loading TTS weights on {device}")
logger.info("Loading OmniVoice model on device: %s", device)
preload_asr = should_preload_tts_asr()
@@ -483,23 +835,67 @@ def _load_model_sync():
logger.info("Preloading PyTorch Whisper with TTS model.")
else:
logger.info("Skipping PyTorch Whisper preload; ASR will load on demand.")
try:
_model = OmniVoice.from_pretrained(
def _load():
return OmniVoice.from_pretrained(
checkpoint, device_map=device, dtype=torch.float16, load_asr=preload_asr,
)
try:
_model = _load()
except OSError as e:
# #352: a truncated HF cache surfaces here as "does not appear to
# have a file named pytorch_model.bin or model.safetensors".
# Translate to an actionable message instead of the raw
# transformers error.
if "does not appear to have a file named" in str(e):
# #352 / #581: a truncated HF cache surfaces here as "does not
# appear to have a file named pytorch_model.bin or
# model.safetensors". Instead of dead-ending the user with a
# manual delete-and-reinstall instruction, try to self-repair: an
# interrupted download leaves the cache missing only some files,
# and snapshot_download() resumes/fills exactly those (a complete
# cache never reaches this branch, so the fast path is untouched).
if not _is_incomplete_cache_error(e):
raise
_set_loading("loading_weights", "Repairing incomplete model cache…")
if not _repair_model_cache(checkpoint):
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"(weights missing — usually an interrupted download). "
"(weights missing — usually an interrupted download)."
f"{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
"and install it again."
) from e
raise
_set_loading("loading_weights", f"Loading TTS weights on {device}")
try:
_model = _load()
except OSError as e2:
# Resume-repair ran but the cache is still unusable. The usual
# cause beyond "repo genuinely lacks weights" is a blob that's
# present with the right size but corrupt — snapshot_download's
# resume trusts it and never re-fetches it (#739). Force a full
# re-download (replaces corrupt blobs) and retry once more before
# falling back to the manual delete-and-reinstall message.
if _is_incomplete_cache_error(e2):
_set_loading("loading_weights", "Re-downloading model files…")
if _repair_model_cache(checkpoint, force=True):
try:
_model = _load()
except OSError as e3:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete "
"and could not be auto-repaired. Open Settings → "
"Models, delete the OmniVoice TTS model, and install "
"it again."
) from e3
else:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete and "
f"could not be auto-repaired.{_repair_failure_detail()} "
"Open Settings → Models, delete the OmniVoice TTS model, "
"and install it again."
) from e2
else:
raise RuntimeError(
f"The TTS model cache for {checkpoint} is incomplete and "
"could not be auto-repaired. Open Settings → Models, delete "
"the OmniVoice TTS model, and install it again."
) from e2
try:
# plan-02 (#65): gate on Triton availability (+ user setting), not
@@ -549,9 +945,25 @@ def _load_model_sync():
logger.info("OmniVoice model loaded successfully.")
return _model
except Exception as exc:
err_msg = str(exc)
# Surface an ACTIONABLE, sanitized error in /model/status (it's shown in
# the first-run System Check). build_failure classifies the cause and
# attaches a fix hint — e.g. a corrupted transformers install
# ([Errno 2] … modeling_*.py) now says "reinstall transformers" instead
# of an unhelpful raw path + "try restarting" — and strips the home dir.
try:
from core.failure import build_failure
_f = build_failure(exc, stage="model-load", include_diagnostic=False)
err_msg = _f["reason"] + (f"{_f['hint']}" if _f.get("hint") else "")
except Exception: # never let failure-formatting mask the real error
err_msg = str(exc)
_set_loading("error", "Model loading failed", error=err_msg)
logger.error("Model loading failed: %s", err_msg)
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
# to an inner import (including one interrupted by process teardown)
# in a generic "Could not import module X. Are this object's
# requirements defined correctly?" — logging only str(exc) discarded
# the real cause in __cause__/__context__ and made a shutdown race
# look like a broken install. exc_info surfaces the full chain.
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
raise
finally:
unregister_listener(lid)
@@ -571,19 +983,15 @@ def _model_load_timeout() -> float:
def _reset_gpu_pool() -> None:
"""Drop the GPU pool singleton so the next access builds a fresh one.
"""Recover from a wedged/timed-out load by abandoning the GPU worker pool.
Python can't kill the thread stuck in a timed-out load, but abandoning the
poisoned single-worker pool means a *retry* gets a clean worker instead of
queueing forever behind the wedged one.
The resilient wrapper is kept (its identity is shared by every importer);
only its inner `ThreadPoolExecutor` is dropped, so the next submit builds a
fresh worker. This is what stops stale references from raising "cannot
schedule new futures after shutdown" after a reset (#589 #599).
"""
global _gpu_pool_singleton
pool, _gpu_pool_singleton = _gpu_pool_singleton, None
if pool is not None:
try:
pool.shutdown(wait=False, cancel_futures=True)
except Exception:
pass
if _gpu_pool_singleton is not None:
_gpu_pool_singleton.reset()
async def _load_model_with_timeout():
@@ -622,6 +1030,22 @@ async def get_model():
return model
def _checkpoint_in_local_cache(checkpoint: str) -> bool:
"""True when ``checkpoint`` is loadable with NO network: an existing local
directory, or a COMPLETE HF cache snapshot. ``snapshot_download(...,
local_files_only=True)`` never constructs an HTTP session, so a broken
proxy env (#959: ``ALL_PROXY``/``HTTPS_PROXY=socks5://`` without socksio)
can't false-negative this probe. Never raises."""
if os.path.isdir(checkpoint):
return True
try:
from huggingface_hub import snapshot_download
snapshot_download(checkpoint, local_files_only=True)
return True
except Exception:
return False
async def preload_model():
"""Background model warm-up — call from lifespan startup.
@@ -634,15 +1058,35 @@ async def preload_model():
return # already loaded
try:
# Check if the required model checkpoint exists before attempting
# a heavy load that would fail and pollute startup logs.
checkpoint = os.environ.get("OMNIVOICE_MODEL", "k2-fsa/OmniVoice")
# a heavy load that would fail and pollute startup logs. Use the same
# resolver as the load path (#693) so a leaked engine id in
# OMNIVOICE_MODEL can't make this model_info() probe fail and silently
# disable warm-up (then the first /generate eats the full load).
checkpoint = resolve_omnivoice_checkpoint()
try:
from huggingface_hub import model_info
model_info(checkpoint, timeout=5)
except Exception:
# Model not downloaded yet — skip preload
logger.info("Preload skipped: %s not available locally.", checkpoint)
return
except Exception as probe_err:
# The probe failing does NOT mean the model isn't installed — it
# means the Hub API wasn't reachable from this process. The #959
# class: under ALL_PROXY/HTTPS_PROXY=socks5:// without socksio,
# hf_hub's get_session() raises ImportError AT CLIENT CONSTRUCTION;
# same story for offline mode, DNS, or firewall failures. Fall back
# to a cache-only probe (no HTTP session involved) and warm up
# anyway when the model is locally present, instead of silently
# skipping and letting the first /generate eat the full load.
if not _checkpoint_in_local_cache(checkpoint):
logger.info(
"Preload skipped: %s not available locally (network probe "
"failed: %s: %s).",
checkpoint, type(probe_err).__name__, probe_err,
)
return
logger.warning(
"Network probe for %s failed (%s: %s) — model found in the "
"local cache; warming up from cache.",
checkpoint, type(probe_err).__name__, probe_err,
)
logger.info("Preloading TTS model in background…")
_last_used = time.time()
@@ -651,7 +1095,11 @@ async def preload_model():
model = await _load_model_with_timeout()
logger.info("Preload complete — model ready.")
except Exception as e:
logger.warning("Model preload failed (non-fatal): %s", e)
# See the matching exc_info note on the _load_model_sync handler above
# (#1000 class) — the full chain, not just str(e), is what actually
# distinguishes a real dependency problem from a shutdown-interrupted
# import.
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
def get_model_status():
is_loaded = model is not None
+111 -7
View File
@@ -10,8 +10,8 @@ plays the moment the video begins and everything feels desynchronised.
``snap_segment_starts`` post-processes segments against the actual audio
(ideally the Demucs-isolated vocals track, which the dub pipeline already
produces): for each segment it scans the waveform inside ``[start, end]``
for the first frame whose RMS rises above an adaptive threshold and moves
``start`` forward to just before that onset.
for the first *sustained* rise of frame RMS above an adaptive threshold
and moves ``start`` forward to just before that onset.
Design constraints:
@@ -23,6 +23,27 @@ Design constraints:
(no frame above the absolute floor) are left untouched.
* **Pure NumPy.** No model, no platform-specific code identical
behaviour on macOS / Windows / Linux, trivially unit-testable.
Robustness against non-speech onsets (#963): a field report showed dubbed
lines starting seconds off because "when a noise is heard (a sigh or
footsteps), it's interpreted as the start of the conversation". Three
guards address that class of failure:
* **Sustained energy.** A frame only counts as an onset when the energy
stays up for a speech-like duration (``SUSTAIN_MIN_S`` within the
following ``SUSTAIN_WINDOW_S``). Footsteps/door thuds/clicks light up
one or two 20 ms frames and die; syllables keep the energy up.
* **Bounded snap distance.** Shifts beyond ``MAX_SNAP_S`` are only
trusted when everything being skipped is (near-)silence the genuine
#280 whisper start-stretch, where Demucs removed the leading music and
left real silence on the vocals track. Jumping far over *audible*
content (e.g. quiet speech sitting under the relative threshold) would
play the dub seconds late, so it is refused.
* **Source-aware.** Snapping only runs on a separated vocals track
(``separated_vocals=True``). On mixed/original audio Demucs skipped
or failed music, ambience and room tone are all legitimate sustained
energy, so any detected "onset" is as likely the score as the speaker;
whisper's own timestamps beat a confidently wrong snap.
"""
from __future__ import annotations
@@ -51,6 +72,23 @@ RELATIVE_THRESHOLD = 0.10
# the whole window is treated as silence and left alone (we'd only be
# snapping to noise).
ABS_RMS_FLOOR = 1e-3
# An onset must be *sustained* to count as speech (#963): within the
# SUSTAIN_WINDOW_S that follows a candidate frame, at least SUSTAIN_MIN_S
# worth of frames must also sit above the threshold. A ~100 ms footstep
# burst fails this; real speech (syllables every few hundred ms) passes.
SUSTAIN_WINDOW_S = 0.30
SUSTAIN_MIN_S = 0.16
# Snaps larger than this are only trusted when the skipped span is
# (near-)silence — see _region_mostly_silent (#963).
MAX_SNAP_S = 1.5
# The skipped span counts as "mostly silent" when at most this fraction of
# its frames is audible. Non-zero so an isolated transient bleeding through
# separation (a footstep) doesn't block a genuine long silence-trim…
SKIPPED_AUDIBLE_FRAC = 0.10
# …where "audible" = above max(ABS_RMS_FLOOR, this fraction of the span's
# own peak); the relative term keeps a slightly raised residual noise floor
# from reading as content.
SKIPPED_FLOOR_PEAK_FRAC = 0.02
def _frame_rms(x: np.ndarray, frame_len: int) -> np.ndarray:
@@ -70,6 +108,15 @@ def detect_speech_onset(
) -> float | None:
"""Return the absolute time (s) of the first speech-like frame inside
``[start_s, end_s]``, or ``None`` when the window is empty / silent.
"Speech-like" requires *sustained* energy (#963): within the
``SUSTAIN_WINDOW_S`` look-ahead after a candidate frame, at least
``SUSTAIN_MIN_S`` worth of frames must also exceed the threshold.
Short broadband transients footsteps, door thuds, mouse clicks
light up one or two 20 ms frames and then die, so they no longer read
as "the conversation started here"; real speech keeps the energy up
across syllables. A candidate too close to the window's end to prove
sustain is rejected (conservative: the ASR timestamp stands).
"""
if sr <= 0 or end_s <= start_s:
return None
@@ -86,10 +133,21 @@ def detect_speech_onset(
if peak < ABS_RMS_FLOOR:
return None # whole window is effectively silent
threshold = max(RELATIVE_THRESHOLD * peak, ABS_RMS_FLOOR)
above = np.nonzero(rms >= threshold)[0]
if above.size == 0:
above = rms >= threshold
candidates = np.nonzero(above)[0]
if candidates.size == 0:
return None
return start_s + float(above[0]) * (frame_len / sr)
frame_s = frame_len / sr
win_frames = max(1, int(round(SUSTAIN_WINDOW_S / frame_s)))
need_frames = max(1, int(round(SUSTAIN_MIN_S / frame_s)))
# counts[k] = above-threshold frames within rms[c : c + win_frames]
# for candidate c — O(n) via a cumulative sum, no per-candidate scan.
cum = np.concatenate(([0], np.cumsum(above)))
counts = cum[np.minimum(candidates + win_frames, above.size)] - cum[candidates]
sustained = candidates[counts >= need_frames]
if sustained.size == 0:
return None # only transient bursts in this window
return start_s + float(sustained[0]) * frame_s
# Hysteresis for full-track onset listing: after a frame crosses the
@@ -138,21 +196,60 @@ def detect_speech_onsets(audio: np.ndarray, sr: int) -> list[float]:
return onsets
def _region_mostly_silent(
audio: np.ndarray,
sr: int,
start_s: float,
end_s: float,
) -> bool:
"""True when ``[start_s, end_s]`` contains (almost) no audible content.
Gates long snaps (> ``MAX_SNAP_S``, #963): jumping far forward is only
trustworthy when everything being skipped is silence the genuine
whisper start-stretch of #280, where Demucs stripped the leading music
and left real silence on the vocals track. A small fraction of audible
frames is tolerated so an isolated transient bleeding through
separation (a footstep) doesn't block the trim; *sustained* audible
content e.g. quiet speech sitting below the relative onset
threshold does block it, because skipping past it would desync the
dub by the full jump.
"""
i0 = max(0, int(start_s * sr))
i1 = min(len(audio), int(end_s * sr))
if i1 <= i0:
return True
rms = _frame_rms(audio[i0:i1], max(1, int(FRAME_S * sr)))
if rms.size == 0:
return True
floor = max(ABS_RMS_FLOOR, SKIPPED_FLOOR_PEAK_FRAC * float(rms.max()))
return float((rms >= floor).mean()) <= SKIPPED_AUDIBLE_FRAC
def snap_segment_starts(
segments: Sequence[dict],
audio: np.ndarray,
sr: int,
*,
min_shift_s: float = MIN_SHIFT_S,
separated_vocals: bool = True,
) -> int:
"""Snap each segment's ``start`` forward to the actual speech onset.
Mutates the segment dicts in place (the shape the dub pipeline passes
around). Returns the number of segments adjusted.
``audio`` should be mono float; the Demucs vocals track gives the best
signal but the mixed track still beats nothing.
``audio`` should be the mono-float **separated vocals** track. When the
caller only has mixed/original audio (Demucs skipped or failed), pass
``separated_vocals=False``: snapping is then disabled entirely (#963) —
on a mixed track music, ambience and footsteps are all sustained energy,
so a detected "onset" is as likely the score as the speaker, and
whisper's own timestamps beat a confidently wrong snap.
"""
if not separated_vocals:
logger.info(
"onset-align: skipped — audio is not a separated vocals track "
"(Demucs unavailable/failed); keeping ASR timestamps as-is")
return 0
if sr <= 0 or audio is None or len(audio) == 0:
return 0
if audio.ndim > 1:
@@ -174,6 +271,13 @@ def snap_segment_starts(
shift = new_start - start
if shift < min_shift_s:
continue
if shift > MAX_SNAP_S and not _region_mostly_silent(audio, sr, start, onset):
# Long jump over audible content (#963): the "onset" is more
# likely a louder late event than the true start — quiet speech
# under the relative threshold would be skipped wholesale and
# the dub would play seconds LATE. Bounded corrections are fine;
# unbounded ones only over true silence (the #280 case).
continue
# Preserve a minimum playable duration.
new_start = min(new_start, end - MIN_SEG_DUR_S)
if new_start - start < min_shift_s:
+176
View File
@@ -145,3 +145,179 @@ def save_lexicon(path, lexicon: Optional[dict]) -> dict[str, str]:
encoding="utf-8",
)
return clean
# ── DB-backed global / per-language dictionary (Expressive-TTS Spec 01) ───────
#
# The JSON ``load_lexicon``/``save_lexicon`` above stay the per-project audiobook
# override. THIS layer is the user-editable, DB-persisted, per-language default
# dictionary surfaced in Settings → Pronunciation. Rows scoped ``language="*"``
# apply to every request; a 2-letter language row applies only when the request
# language's prefix matches (case-insensitive), so a German entry never fires on
# an English render. Both layers are pure text substitution — they ride the same
# ReDoS-safe ``apply_lexicon`` matcher, so every engine honors them.
_ALL_LANG = "*"
def _lang_prefix(language: Optional[str]) -> Optional[str]:
"""Normalize a request language to a lowercase 2-letter prefix.
``"Auto"``/``None``/``""`` ``None`` (means "no language pin": only global
``*`` rows apply, language-tagged rows are skipped, mirroring how the engines
treat an unset language). A value like ``"en-US"`` / ``"English"``
``"en"`` (first two letters); matching against entries is on this prefix.
"""
if not language:
return None
s = str(language).strip().lower()
if not s or s == "auto":
return None
return s[:2]
def entries_for_language(entries, language: Optional[str]) -> dict[str, str]:
"""Collapse DB rows into a ``{term: replacement}`` map for ``apply_lexicon``.
Filters to ``enabled`` rows whose scope is global (``*``) OR whose language
prefix matches the request language. Only the **respelling** path produces a
plain substitution here (Phase 1); IPA/CMU rows that carry no respelling are
skipped at this layer (they're handled — or honestly degraded — by the
engine-markup path, never silently mangling text). A language-specific row
overrides a global row with the same (case-folded) term, so a per-language
pronunciation can refine the global default.
``entries`` is any iterable of mappings/rows with ``term``, ``replacement``,
``type``, ``language``, ``enabled`` keys (a ``sqlite3.Row`` works directly).
"""
req_prefix = _lang_prefix(language)
# Two passes so language rows win over global rows on the same term: collect
# global first, then overlay matching-language rows.
glob: dict[str, str] = {}
lang: dict[str, str] = {}
for e in entries:
try:
if not int(e["enabled"]):
continue
except (KeyError, IndexError, TypeError, ValueError):
continue
term = (e["term"] or "").strip()
if not term:
continue
etype = (e["type"] or "respelling").strip().lower()
replacement = e["replacement"] if e["replacement"] is not None else ""
# Phase 1: only respelling rows substitute text. IPA/CMU rows without a
# respelling fall through (Phase 2 lowers them to engine markup); we do
# NOT feed a raw IPA string into the grapheme stream.
if etype != "respelling":
continue
scope = (e["language"] or _ALL_LANG).strip() or _ALL_LANG
if scope == _ALL_LANG:
glob[term] = str(replacement)
else:
if req_prefix is not None and scope[:2].lower() == req_prefix:
lang[term] = str(replacement)
merged = dict(glob)
merged.update(lang) # language rows override global on the same term
return merged
# ── Inline one-off override: [[term|replacement]] / [[replacement]] ─────────
#
# Double brackets are unambiguous against the single-bracket grammar
# (``[voice:]``/``[pause]``/SSML-lite/``[Name]``): ``_VOICE_RE`` is
# ``\[voice:([^\]\[]*)\]`` — it forbids inner brackets, so it can't span a
# ``[[…]]``; the SSML-lite / pause vocabularies are closed literal sets that
# ``[[…]]`` is not a member of. We resolve ``[[…]]`` BEFORE chunking so the
# splitter never sees it. ReDoS-safe: ``\[\[[^\]]*\]\]`` is a bounded literal
# class, no nested quantifier.
#
# [[gif|jiff]] → replaces the literal "gif" → "jiff" for this occurrence
# [[Nuh-VAD-uh]] → the bracket content itself is spoken (brackets stripped)
# Bounded inner repetition ({0,256}) keeps this strictly linear: ``[^\]]`` also
# matches ``[``, so an unbounded run of ``[`` with no closing ``]]`` would let the
# engine re-scan O(n) content from O(n) start positions (polynomial ReDoS). The
# bound caps per-position work; an inline override is a short respelling, so 256
# chars is far more than any real ``[[term|replacement]]`` needs.
_INLINE_RE = re.compile(r"\[\[([^\]]{0,256})\]\]")
def apply_inline_overrides(text: str) -> str:
"""Resolve ``[[…]]`` one-off pronunciation overrides to plain spoken text.
``[[term|replacement]]`` ``replacement`` (the ``term`` half is a label for
the author; only the replacement is spoken). ``[[replacement]]`` (no pipe)
``replacement`` with the brackets stripped. Empty ``[[]]`` collapses away.
Applied once per occurrence; nothing persists. Single ``[]`` tags are left
untouched (the regex requires a double bracket on both sides).
"""
if not text or "[[" not in text:
return text or ""
def _repl(m: re.Match) -> str:
inner = m.group(1)
if "|" in inner:
inner = inner.split("|", 1)[1]
return inner
return _INLINE_RE.sub(_repl, text)
def apply_pronunciation(
text: str,
entries=None,
language: Optional[str] = None,
*,
lexicon: Optional[dict] = None,
) -> str:
"""Apply the pronunciation dictionary + inline overrides to ``text``.
Order (load-bearing):
1. DB dictionary rows (``entries``) filtered to ``language`` + an optional
per-project ``lexicon`` JSON overlay (project wins on term conflict,
matching the audiobook layering). Both go through one ``apply_lexicon``
pass (longest-term-first, word-boundary aware, idempotent).
2. Inline ``[[]]`` one-off overrides resolved last, so an inline override
always wins over any dictionary entry for that occurrence.
A falsy ``text`` / empty dictionary / no inline markers is a pass-through, so
legacy plain text is byte-identical.
"""
if not text:
return text or ""
merged = entries_for_language(entries or [], language)
if lexicon:
# Project-local JSON overlays the DB defaults; project wins on conflict.
merged.update(normalize_lexicon(lexicon))
out = apply_lexicon(text, merged) if merged else text
return apply_inline_overrides(out)
# ── DB load/save ──────────────────────────────────────────────────────────────
def load_entries_from_db() -> list[dict]:
"""Return every pronunciation_entries row as a list of plain dicts.
Import-light: the DB module is imported lazily so the pure-parser path (and
the audiobook JSON path) never pull in sqlite/config.
"""
from core.db import db_conn
with db_conn() as conn:
rows = conn.execute(
"SELECT id, term, replacement, type, language, enabled, created_at "
"FROM pronunciation_entries ORDER BY created_at ASC, id ASC"
).fetchall()
return [dict(r) for r in rows]
def load_dict_for_request(language: Optional[str] = None) -> dict[str, str]:
"""Convenience: DB rows → ``{term: replacement}`` for a request language.
Returns ``{}`` (a no-op for ``apply_pronunciation``) if the table is absent
or the DB can't be opened — pronunciation is never allowed to break synth.
"""
try:
return entries_for_language(load_entries_from_db(), language)
except Exception: # noqa: BLE001 — table missing / DB locked → no-op
return {}
+139 -16
View File
@@ -19,13 +19,69 @@ Two tiers, both applied only to FINAL transcripts (never partials):
from __future__ import annotations
import asyncio
import json
import logging
import os
import re
import time
from dataclasses import dataclass
logger = logging.getLogger("omnivoice.refinement")
# Hard wall-clock budget (seconds) for a single dictation refinement LLM call.
# The dictation FINAL must never be delayed longer than this by a slow or dead
# LLM endpoint — refinement is best-effort and falls back to the unrefined
# (but polished) text on timeout. 4s keeps a healthy local model (Ollama /
# LM Studio, sub-second on the tiny cleanup prompt) fully usable while turning
# the old worst case — a placeholder/dead endpoint blocking the send ~51s until
# the widget's 15s fallback fired — into a bounded ~4s at most. Env-tunable so
# power users on a slow local LLM can raise it. Guarded by the regression tests
# in tests/backend/services/test_refinement_llm.py and tests/test_capture_ws.py.
_DEFAULT_REFINE_TIMEOUT_S = 4.0
def _refine_timeout_s() -> float:
"""The refinement LLM budget in seconds (OMNIVOICE_REFINE_TIMEOUT_S).
Falls back to :data:`_DEFAULT_REFINE_TIMEOUT_S` on an unset/invalid/non-
positive value so a bad env var can never disable the bound."""
raw = os.environ.get("OMNIVOICE_REFINE_TIMEOUT_S", "")
try:
v = float(raw)
if v > 0:
return v
except (TypeError, ValueError):
pass
return _DEFAULT_REFINE_TIMEOUT_S
# Most-recent refinement outcome, so the Settings panel can tell the user when a
# configured LLM is actually failing/timing out (the honesty layer behind the
# `llm_ready` flag, which only means "an endpoint is configured"). Best-effort,
# process-local, cleared on success.
_last_refine_status: dict | None = None
def _note_refine_status(*, ok: bool, reason: str | None = None) -> None:
global _last_refine_status
_last_refine_status = {"ok": bool(ok), "reason": reason, "at": time.time()}
def get_last_refine_status() -> dict | None:
"""The last refinement outcome as ``{ok, reason, at}`` or None if refinement
hasn't run this session. ``ok=False`` with ``reason`` ("timeout" or a short
error string) means a configured LLM failed the most recent final."""
return dict(_last_refine_status) if _last_refine_status else None
def _short_reason(exc: Exception) -> str:
"""A compact, non-leaky label for a refinement failure (for the UI hint)."""
name = type(exc).__name__
if "Timeout" in name or "timeout" in str(exc).lower():
return "timeout"
return name
# A token (or unit) must repeat at least this many times consecutively to be
# treated as an STT artifact. Rhetorical repetition ("no, no, no, no, no" —
# five repeats) stays below the threshold and survives.
@@ -248,6 +304,19 @@ REFINEMENT_EXAMPLES: list[tuple[str, str]] = [
# settings_store key holding the user's refinement config (plain JSON).
_SETTINGS_KEY = "dictation_refinement"
# LLM Skills registry id — Settings → LLM Skills can disable refinement's LLM
# use or route it to a specific provider. Disabled == identical pass-through
# (the same path as "no LLM configured").
_SKILL_ID = "dictation_refinement"
def _skill_llm():
"""The skill-resolved backend (OffBackend when disabled/unconfigured)."""
from services import llm_skills
from services.llm_backend import get_active_llm_backend
return llm_skills.skill_backend(_SKILL_ID, active=get_active_llm_backend)
def get_refinement_config() -> dict:
"""Read the persisted config: {auto, smart_cleanup, self_correction,
@@ -274,43 +343,97 @@ def set_refinement_config(cfg: dict) -> dict:
return merged
def refine_transcript(transcript: str, flags: RefinementFlags | None = None) -> str:
def refine_transcript(
transcript: str,
flags: RefinementFlags | None = None,
*,
timeout_s: float | None = None,
) -> str:
"""Run the transcript through the configured LLM. Raises on failure —
callers decide the fallback (maybe_refine swallows into pass-through)."""
from services.llm_backend import get_active_llm_backend
callers decide the fallback (maybe_refine swallows into pass-through).
The LLM HTTP call is bounded by ``timeout_s`` (default: the refinement
budget) so a dead/slow endpoint can't tie the call up for the client's full
45s LLM timeout the class of stall this whole module guards against."""
flags = flags or RefinementFlags()
backend = get_active_llm_backend()
backend = _skill_llm()
messages = [{"role": "system", "content": build_refinement_prompt(flags)}]
for user_turn, assistant_turn in REFINEMENT_EXAMPLES:
messages.append({"role": "user", "content": user_turn})
messages.append({"role": "assistant", "content": assistant_turn})
messages.append({"role": "user", "content": transcript})
return backend.chat_messages(messages=messages).strip()
budget = timeout_s if timeout_s is not None else _refine_timeout_s()
return backend.chat_messages(messages=messages, timeout=budget).strip()
def maybe_refine(transcript: str) -> str | None:
def maybe_refine(transcript: str, *, timeout_s: float | None = None) -> str | None:
"""Best-effort refinement for the dictation final path.
Returns the refined text, or None when refinement is off, no LLM
backend is configured, the result is empty, or anything fails the
raw transcript always stands. Never raises.
raw transcript always stands. Never raises. Records the outcome via
:func:`get_last_refine_status` so the UI can flag a failing LLM.
Blocking (network I/O); the WS/REST callers run it off-thread. Prefer
:func:`maybe_refine_async` on the live-dictation path it adds the hard
wall-clock bound so a slow endpoint can never delay the ``final`` send.
"""
if not transcript or not transcript.strip():
return None
cfg = get_refinement_config()
if not cfg.get("auto", True):
return None
backend = _skill_llm()
if backend.id == "off":
# No LLM configured — or the dictation_refinement skill is disabled /
# routed to an unconfigured provider — is not a failure. Leave the last
# status untouched (same pass-through as today).
return None
try:
cfg = get_refinement_config()
if not cfg.get("auto", True):
return None
from services.llm_backend import get_active_llm_backend
backend = get_active_llm_backend()
if backend.id == "off":
return None
refined = refine_transcript(transcript, RefinementFlags.from_dict(cfg))
refined = refine_transcript(
transcript, RefinementFlags.from_dict(cfg), timeout_s=timeout_s
)
if not refined:
return None
_note_refine_status(ok=True)
return refined
except Exception as e: # noqa: BLE001 — pass-through is the contract
logger.warning("Dictation refinement skipped: %s", e)
_note_refine_status(ok=False, reason=_short_reason(e))
return None
async def maybe_refine_async(
transcript: str, *, timeout_s: float | None = None
) -> str | None:
"""Async, hard-time-bounded refinement for the live-dictation final path.
Runs :func:`maybe_refine` off-thread under a hard ``OMNIVOICE_REFINE_TIMEOUT_S``
(~4s) budget so a slow or dead LLM endpoint can NEVER block the caller and
therefore the dictation ``final`` send longer than the budget. On timeout
(or any failure) it returns None and the raw, already-polished transcript
stands. Never raises.
``asyncio.wait_for`` can't cancel the worker thread, but the LLM call it runs
is itself bounded to the same budget (see :func:`refine_transcript`), so an
orphaned thread unwinds shortly after rather than lingering the full 45s.
"""
if not transcript or not transcript.strip():
return None
budget = timeout_s if timeout_s is not None else _refine_timeout_s()
try:
return await asyncio.wait_for(
asyncio.to_thread(maybe_refine, transcript, timeout_s=budget),
timeout=budget,
)
except asyncio.TimeoutError:
logger.warning(
"Dictation refinement exceeded its %.1fs budget — sending the "
"unrefined final (set OMNIVOICE_REFINE_TIMEOUT_S to adjust).", budget,
)
_note_refine_status(ok=False, reason="timeout")
return None
except Exception as e: # noqa: BLE001 — best-effort; the raw final stands
logger.warning("Dictation refinement failed: %s", e)
_note_refine_status(ok=False, reason=_short_reason(e))
return None
+176 -6
View File
@@ -532,13 +532,183 @@ def assign_speakers_from_turns(
return segments
def assign_speakers_heuristic(segments: List[dict]) -> List[dict]:
"""Two-speaker alternation based on silence gaps."""
current = 1
def assign_speakers_heuristic(
segments: List[dict], num_speakers: Optional[int] = None
) -> List[dict]:
"""Silence-gap speaker assignment (used when no diarization model runs).
Base signal: a gap > SPEAKER_GAP seconds between consecutive segments is
treated as a speaker change. Without a ``num_speakers`` hint this keeps
the legacy behavior alternate between exactly two labels. With a hint:
* ``num_speakers=1`` every segment gets ``"Speaker 1"``.
* ``num_speakers>=2`` labels round-robin across N speakers at each
gap boundary, so the user's requested count is represented instead of
being silently capped at 2.
Limits (be honest with callers): this honors the *count*, not voice
identity. The rotation order is arbitrary (a returning speaker gets the
next label in the cycle, not their own), rapid exchanges with no
> SPEAKER_GAP pause still collapse into one label, and N is an upper
bound audio with fewer gap boundaries than N yields fewer labels.
Real per-speaker attribution needs pyannote (or an inline-diarizing ASR
backend); callers should warn the user accordingly (see dub_core).
Invalid hints (non-int, < 1) fall back to the legacy two-speaker cycle.
"""
try:
n = int(num_speakers) if num_speakers is not None else 2
except (TypeError, ValueError):
n = 2
if n < 1:
n = 2
current = 0 # zero-based rotation index; rendered one-based below
last_end = 0.0
for i, s in enumerate(segments):
if i > 0 and (s["start"] - last_end) > SPEAKER_GAP:
current = 2 if current == 1 else 1
s["speaker_id"] = f"Speaker {current}"
if i > 0 and n > 1 and (s["start"] - last_end) > SPEAKER_GAP:
current = (current + 1) % n
s["speaker_id"] = f"Speaker {current + 1}"
last_end = s["end"]
return segments
# ── Speaker-aware re-split (#486) ────────────────────────────────────────────
#
# Segmentation runs BEFORE diarization and groups words by sentence/duration
# only, so one segment can span two speakers' turns. assign_speakers_* then only
# *relabels* each segment with its majority speaker — the boundary is lost and a
# two-speaker exchange reads as one line. This pass re-splits such a segment at
# the word-level speaker boundary, after diarization.
#
# Hard invariant (the single-speaker no-regression guarantee): a segment whose
# words all map to ONE speaker is returned byte-for-byte unchanged — same dict,
# id, text, start, end — so single-speaker dubs and their timing never move.
def _word_speaker(w: "Word", turns: Sequence[tuple]) -> Optional[str]:
"""Majority-overlap speaker label for a word; midpoint membership as a
fallback; ``None`` when the word has no diarization coverage at all."""
acc: dict = {}
for ts, te, label in turns:
left = max(w.start, ts)
right = min(w.end, te)
if right > left:
acc[label] = acc.get(label, 0.0) + (right - left)
if acc:
return max(acc.items(), key=lambda kv: kv[1])[0]
mid = (w.start + w.end) / 2.0
for ts, te, label in turns:
if ts <= mid <= te:
return label
return None
def _fill_and_smooth(labels: List[Optional[str]]) -> List[Optional[str]]:
"""Forward/back-fill gaps (words with no coverage inherit a neighbor) and
smooth single-word flips, so one mis-attributed word inside a speaker's run
(diarization noise) doesn't trigger a spurious split."""
out = list(labels)
n = len(out)
last = None
for i in range(n):
if out[i] is None:
out[i] = last
else:
last = out[i]
nxt = None
for i in range(n - 1, -1, -1):
if out[i] is None:
out[i] = nxt
else:
nxt = out[i]
for i in range(1, n - 1):
if out[i] != out[i - 1] and out[i - 1] == out[i + 1]:
out[i] = out[i - 1]
return out
def _resplit_core(
segments: List[dict], words: Sequence["Word"], turns: Sequence[tuple],
) -> List[dict]:
"""Split each segment that spans >1 speaker at the word-level boundary.
``turns`` is a normalised list of ``(start, end, speaker_label)``. Single-
speaker segments are passed through untouched. Pieces keep the segment's
outer start/end (preserving any onset-snap) and use word times for interior
boundaries, so the pieces exactly cover the original span.
"""
if not turns or not words:
return segments
ordered = sorted(words, key=lambda w: (w.start, w.end))
out: List[dict] = []
for seg in segments:
s0, s1 = seg["start"], seg["end"]
seg_words = [w for w in ordered if min(w.end, s1) - max(w.start, s0) > 1e-6]
if len(seg_words) < 2:
out.append(seg)
continue
labels = _fill_and_smooth([_word_speaker(w, turns) for w in seg_words])
if len({l for l in labels if l is not None}) <= 1:
out.append(seg) # single speaker (or unknown) → byte-for-byte unchanged
continue
runs: List[tuple] = []
for w, label in zip(seg_words, labels):
if runs and runs[-1][0] == label:
runs[-1][1].append(w)
else:
runs.append((label, [w]))
n_runs = len(runs)
piece_no = 0
for k, (label, ws) in enumerate(runs):
text = _clean(" ".join(w.text for w in ws))
if not text:
continue
piece = dict(seg)
piece["text"] = text
piece["start"] = s0 if k == 0 else ws[0].start
piece["end"] = s1 if k == n_runs - 1 else ws[-1].end
if label:
piece["speaker_id"] = label
if piece_no > 0:
piece["id"] = f"{seg.get('id', 'seg')}-{piece_no}"
if "text_original" in piece:
piece["text_original"] = text
elif "text_original" in piece:
piece["text_original"] = text
out.append(piece)
piece_no += 1
return out
def _diar_speaker_label(raw) -> str:
"""``SPEAKER_00`` → ``Speaker 1`` (mirrors assign_speakers_from_diarization)."""
try:
return f"Speaker {int(str(raw).split('_')[-1]) + 1}"
except (ValueError, AttributeError):
return str(raw)
def resplit_segments_by_diarization(
segments: List[dict], words: Sequence["Word"], diarization,
) -> List[dict]:
"""Speaker-aware re-split using a pyannote diarization result (#486)."""
turns = [
(turn.start, turn.end, _diar_speaker_label(spk))
for turn, _, spk in diarization.itertracks(yield_label=True)
]
return _resplit_core(segments, words, turns)
def resplit_segments_by_turns(
segments: List[dict], words: Sequence["Word"], turns: Sequence[dict],
) -> List[dict]:
"""Speaker-aware re-split using inline ASR speaker turns (FunASR cam++).
``speaker`` is used verbatim (FunASR already labels ``"Speaker N"``), matching
:func:`assign_speakers_from_turns`."""
norm = [
(t["start"], t["end"], t["speaker"])
for t in (turns or [])
if t.get("speaker") is not None
and t.get("start") is not None
and t.get("end") is not None
]
return _resplit_core(segments, words, norm)
+106 -4
View File
@@ -108,6 +108,107 @@ def clear_hf_token() -> None:
conn.execute("DELETE FROM settings WHERE key = ?", (_TOKEN_KEY,))
# ── Generic encrypted secrets (LLM provider API keys, future tokens) ───────
# The HF token got the first bespoke encrypted row; the LLM-providers feature
# needs the *same* at-rest protection for a dozen provider keys. Rather than
# copy the Fernet dance per provider, expose generic secret helpers. Rows are
# namespaced with the ``secret.`` prefix so a misrouted ``get_text`` on a
# secret key returns opaque ciphertext (defence in depth), and so plaintext
# ``settings`` rows can never collide with a secret. Same InvalidToken →
# None degrade as the HF path (install moved across machines → fall back to
# env), same per-install key.
_SECRET_PREFIX = "secret."
def _secret_key_name(name: str) -> str:
if not name or not isinstance(name, str):
raise ValueError(f"secret name must be a non-empty string, got {name!r}")
if name == _TOKEN_KEY or name.startswith(_SECRET_PREFIX):
raise ValueError(f"invalid secret name {name!r}")
return f"{_SECRET_PREFIX}{name}"
def get_secret(name: str) -> Optional[str]:
"""Return a decrypted secret (e.g. an LLM provider API key), or None.
Mirrors :func:`get_hf_token`: on decrypt failure (install migrated across
machines) or any SQLite error, log and return None so callers fall back to
env / provider defaults instead of crashing.
"""
from core.db import db_conn
key = _secret_key_name(name)
try:
with db_conn() as conn:
row = conn.execute(
"SELECT value FROM settings WHERE key = ?", (key,)
).fetchone()
if row is None or not row[0]:
return None
try:
from cryptography.fernet import InvalidToken
except ImportError: # pragma: no cover — dep should always be present
logger.error("cryptography unavailable; cannot decrypt secret %s", name)
return None
try:
return _fernet().decrypt(row[0].encode("ascii")).decode("utf-8")
except InvalidToken:
logger.warning(
"Stored secret %r failed to decrypt (install moved across "
"machines or salt tampered) — falling back to env/default.", name,
)
return None
except Exception:
logger.exception("settings_store.get_secret(%s): SQLite read failed", name)
return None
def set_secret(name: str, value: str) -> None:
"""Persist an encrypted secret. Empty value clears the row."""
if not value:
clear_secret(name)
return
from core.db import db_conn
key = _secret_key_name(name)
blob = _fernet().encrypt(value.encode("utf-8")).decode("ascii")
with db_conn() as conn:
conn.execute(
"INSERT OR REPLACE INTO settings(key, value, updated_at) "
"VALUES (?, ?, ?)",
(key, blob, time.time()),
)
def clear_secret(name: str) -> None:
"""Remove a secret row (salt row preserved, like clear_hf_token)."""
from core.db import db_conn
key = _secret_key_name(name)
with db_conn() as conn:
conn.execute("DELETE FROM settings WHERE key = ?", (key,))
def list_secret_names() -> list[str]:
"""Return the bare names of all stored secrets (no values, no ciphertext).
Lets the LLM-providers settings API report *which* providers have a key
configured without ever decrypting or returning the key material.
"""
from core.db import db_conn
try:
with db_conn() as conn:
rows = conn.execute(
"SELECT key FROM settings WHERE key LIKE ?",
(f"{_SECRET_PREFIX}%",),
).fetchall()
return [r[0][len(_SECRET_PREFIX):] for r in rows if r and r[0]]
except Exception:
logger.exception("settings_store.list_secret_names: SQLite read failed")
return []
# ── Non-secret text settings ──────────────────────────────────────────────
# Plan 01-02 Task 4 (INST-12): the Performance panel needs to persist a
# boolean toggle (`perf.torch_compile_disabled`). It is NOT a secret — no
@@ -128,7 +229,8 @@ def get_text(key: str, default: Optional[str] = None) -> Optional[str]:
looking like opaque bytes callers MUST use `get_hf_token()` for
secrets and only ever pass non-secret keys to `get_text()`.
"""
if key == _TOKEN_KEY: # defence in depth — never let a misrouted call leak ciphertext
if key == _TOKEN_KEY or key.startswith(_SECRET_PREFIX):
# defence in depth — never let a misrouted call leak ciphertext
return default
from core.db import db_conn
@@ -150,10 +252,10 @@ def set_text(key: str, value: str) -> None:
Use for non-secret config only. For tokens, use `set_hf_token()`.
"""
if key == _TOKEN_KEY:
if key == _TOKEN_KEY or key.startswith(_SECRET_PREFIX):
raise ValueError(
"set_text refuses to write to the encrypted hf_token row; "
"use set_hf_token() for secrets"
"set_text refuses to write to an encrypted secret row; "
"use set_hf_token()/set_secret() for secrets"
)
from core.db import db_conn
+334
View File
@@ -0,0 +1,334 @@
"""
sherpa-onnx live-dictation ASR backend.
Adds the k2-fsa/sherpa-onnx ONNX runtime as a *dictation* engine alongside the
existing Whisper/NeMo family without touching any of them. The whole point of
this engine is **live, faster-than-real-time dictation on CPU**:
STREAMING models (OnlineRecognizer) emit partial text frame-by-frame as the
user speaks, finalising on sherpa's built-in endpoint (silence) detection.
OFFLINE models (OfflineRecognizer) re-transcribe a growing buffer on a short
cadence so the user still sees live partials, finalising on EOF/silence.
CPU provider only (strict cross-platform-default parity rule): identical
behaviour on macOS arm64+x86_64, Windows x64, Linux. No CUDA dependency.
Model weights are the small int8 ONNX checkpoints published under
``csukuangfj/`` on HuggingFace; they download on first use through the same HF
cache the rest of the app uses (``snapshot_download``). Exact asset filenames
were verified against the live HF repo trees (see ``_MODELS`` below) the
streaming zipformer repos use the plain ``encoder-epoch-99-avg-1.int8.onnx``
naming, NOT a ``-chunk-16-left-64`` variant.
"""
from __future__ import annotations
import logging
import os
from dataclasses import dataclass, field
logger = logging.getLogger("omnivoice.asr.sherpa")
# CPU only — strict cross-platform default-parity rule. Overridable for
# power users on a verified GPU build, but the default never diverges.
_PROVIDER = os.environ.get("OMNIVOICE_SHERPA_ASR_PROVIDER", "cpu")
_NUM_THREADS = int(os.environ.get("OMNIVOICE_SHERPA_ASR_THREADS", "2"))
def _endpoint_rules() -> tuple[float, float]:
"""Trailing-silence endpoint rules (seconds) for streaming recognizers.
Wispr-Flow-speed defaults (dictation v2): rule2 commits ~0.6s after speech
stops, rule1 flushes after 1.0s of trailing non-speech down from the
upstream 2.4/1.2, which made every committed sentence feel laggy. Read at
call time so the env overrides apply without a restart.
"""
def _f(env: str, default: float) -> float:
try:
return float(os.environ.get(env, "") or default)
except (TypeError, ValueError):
return default
return (_f("OMNIVOICE_DICTATION_ENDPOINT_R1", 1.0),
_f("OMNIVOICE_DICTATION_ENDPOINT_R2", 0.6))
@dataclass(frozen=True)
class SherpaModelSpec:
"""One downloadable sherpa-onnx dictation model.
``files`` maps a logical role (encoder/decoder/joiner/tokens) to the EXACT
asset filename in the HF repo. ``kind`` selects the recognizer factory:
``offline-transducer`` | ``offline-whisper`` | ``online-transducer`` |
``online-paraformer``. ``tag`` is the frontend-facing "offline"/"streaming".
"""
id: str
repo_id: str
label: str
tag: str # "offline" | "streaming"
kind: str # recognizer factory selector
size_gb: float
languages: str
files: dict[str, str]
recommended: bool = False
model_type: str = "" # offline transducer only (nemo_transducer)
extra: dict = field(default_factory=dict)
@property
def streaming(self) -> bool:
return self.tag == "streaming"
# ── The 7 models (HF repo ids under csukuangfj/, filenames VERIFIED against the
# live HF /api/models/<repo>/tree/main on 2026-06-25; int8 variants pinned).
_MODELS: dict[str, SherpaModelSpec] = {
"sherpa-parakeet-tdt-v3": SherpaModelSpec(
id="sherpa-parakeet-tdt-v3",
repo_id="csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8",
label="Parakeet TDT v3",
tag="offline",
kind="offline-transducer",
size_gb=0.18,
languages="25 European languages",
recommended=True,
model_type="nemo_transducer",
files={
"encoder": "encoder.int8.onnx",
"decoder": "decoder.int8.onnx",
"joiner": "joiner.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-parakeet-tdt-v2": SherpaModelSpec(
id="sherpa-parakeet-tdt-v2",
repo_id="csukuangfj/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8",
label="Parakeet TDT v2",
tag="offline",
kind="offline-transducer",
size_gb=0.17,
languages="English",
model_type="nemo_transducer",
files={
"encoder": "encoder.int8.onnx",
"decoder": "decoder.int8.onnx",
"joiner": "joiner.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-zipformer-bilingual-zh-en": SherpaModelSpec(
id="sherpa-zipformer-bilingual-zh-en",
repo_id="csukuangfj/sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20",
label="Zipformer Bilingual",
tag="streaming",
kind="online-transducer",
size_gb=0.13,
languages="Chinese + English",
files={
"encoder": "encoder-epoch-99-avg-1.int8.onnx",
"decoder": "decoder-epoch-99-avg-1.int8.onnx",
"joiner": "joiner-epoch-99-avg-1.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-paraformer-bilingual-zh-en": SherpaModelSpec(
id="sherpa-paraformer-bilingual-zh-en",
repo_id="csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en",
label="Paraformer Bilingual",
tag="streaming",
kind="online-paraformer",
size_gb=0.115,
languages="Chinese + English",
files={
"encoder": "encoder.int8.onnx",
"decoder": "decoder.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-zipformer-en-20m": SherpaModelSpec(
id="sherpa-zipformer-en-20m",
repo_id="csukuangfj/sherpa-onnx-streaming-zipformer-en-20M-2023-02-17",
label="Zipformer Streaming EN",
tag="streaming",
kind="online-transducer",
size_gb=0.128,
languages="English",
files={
"encoder": "encoder-epoch-99-avg-1.int8.onnx",
"decoder": "decoder-epoch-99-avg-1.int8.onnx",
"joiner": "joiner-epoch-99-avg-1.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-zipformer-zh-14m": SherpaModelSpec(
id="sherpa-zipformer-zh-14m",
repo_id="csukuangfj/sherpa-onnx-streaming-zipformer-zh-14M-2023-02-23",
label="Zipformer Streaming ZH",
tag="streaming",
kind="online-transducer",
size_gb=0.074,
languages="Chinese",
files={
"encoder": "encoder-epoch-99-avg-1.int8.onnx",
"decoder": "decoder-epoch-99-avg-1.int8.onnx",
"joiner": "joiner-epoch-99-avg-1.int8.onnx",
"tokens": "tokens.txt",
},
),
"sherpa-whisper-tiny": SherpaModelSpec(
id="sherpa-whisper-tiny",
repo_id="csukuangfj/sherpa-onnx-whisper-tiny",
label="Whisper Tiny",
tag="offline",
kind="offline-whisper",
size_gb=0.116,
languages="90+ languages (auto-detect)",
files={
"encoder": "tiny-encoder.int8.onnx",
"decoder": "tiny-decoder.int8.onnx",
"tokens": "tiny-tokens.txt",
},
),
}
DEFAULT_MODEL_ID = "sherpa-parakeet-tdt-v3"
# repo_id → model id, so the model-store list (keyed by repo_id) can be
# enriched with the dictation metadata, and so capture can map either key.
_REPO_TO_ID: dict[str, str] = {m.repo_id: mid for mid, m in _MODELS.items()}
def list_specs() -> list[SherpaModelSpec]:
return list(_MODELS.values())
def get_spec(model_id: str) -> SherpaModelSpec | None:
"""Look up a spec by its dictation id OR its HF repo_id."""
if model_id in _MODELS:
return _MODELS[model_id]
if model_id in _REPO_TO_ID:
return _MODELS[_REPO_TO_ID[model_id]]
return None
def is_sherpa_model(model_id: str | None) -> bool:
return bool(model_id) and get_spec(model_id) is not None
def sherpa_available() -> tuple[bool, str]:
try:
import sherpa_onnx # noqa: F401
return True, "ready"
except ImportError as e:
return False, f"sherpa-onnx not installed: {e}. Install with: uv add sherpa-onnx"
def _resolve_model_dir(spec: SherpaModelSpec, *, download: bool = True) -> str:
"""Return the local directory containing this model's ONNX assets.
Tries the HF cache offline first (``local_files_only=True``); on a miss,
downloads on first use (like every other engine) unless ``download=False``.
Restricts the fetch to the exact int8 assets we pin via ``allow_patterns``
so we never pull the bundled fp32 weights or test wavs.
"""
from huggingface_hub import snapshot_download
wanted = list(spec.files.values())
try:
return snapshot_download(
repo_id=spec.repo_id,
local_files_only=True,
allow_patterns=wanted,
)
except Exception:
if not download:
raise
logger.info("sherpa dictation: downloading %s on first use", spec.repo_id)
return snapshot_download(repo_id=spec.repo_id, allow_patterns=wanted)
def is_installed(spec: SherpaModelSpec) -> bool:
"""True if every pinned asset is already present in the HF cache."""
try:
d = _resolve_model_dir(spec, download=False)
except Exception:
return False
return all(os.path.isfile(os.path.join(d, f)) for f in spec.files.values())
# ── Recognizers ──────────────────────────────────────────────────────────────
def build_offline_recognizer(spec: SherpaModelSpec, *, download: bool = True):
"""Construct an ``OfflineRecognizer`` for an offline transducer/whisper model."""
import sherpa_onnx
d = _resolve_model_dir(spec, download=download)
def p(role: str) -> str:
return os.path.join(d, spec.files[role])
if spec.kind == "offline-transducer":
return sherpa_onnx.OfflineRecognizer.from_transducer(
encoder=p("encoder"),
decoder=p("decoder"),
joiner=p("joiner"),
tokens=p("tokens"),
num_threads=_NUM_THREADS,
provider=_PROVIDER,
decoding_method="greedy_search",
model_type=spec.model_type or "nemo_transducer",
)
if spec.kind == "offline-whisper":
return sherpa_onnx.OfflineRecognizer.from_whisper(
encoder=p("encoder"),
decoder=p("decoder"),
tokens=p("tokens"),
num_threads=_NUM_THREADS,
provider=_PROVIDER,
language="", # auto-detect
task="transcribe",
)
raise ValueError(f"{spec.id} is not an offline model (kind={spec.kind})")
def build_online_recognizer(spec: SherpaModelSpec, *, download: bool = True):
"""Construct an ``OnlineRecognizer`` (true streaming) with endpoint detection.
Endpoint (silence) detection drives the live "final" boundary: sherpa
commits a sentence after trailing silence so we can flush a ``final`` and
reset the stream for the next utterance all within one WS session.
"""
import sherpa_onnx
d = _resolve_model_dir(spec, download=download)
rule1, rule2 = _endpoint_rules()
def p(role: str) -> str:
return os.path.join(d, spec.files[role])
if spec.kind == "online-transducer":
return sherpa_onnx.OnlineRecognizer.from_transducer(
tokens=p("tokens"),
encoder=p("encoder"),
decoder=p("decoder"),
joiner=p("joiner"),
num_threads=_NUM_THREADS,
provider=_PROVIDER,
decoding_method="greedy_search",
enable_endpoint_detection=True,
rule1_min_trailing_silence=rule1,
rule2_min_trailing_silence=rule2,
rule3_min_utterance_length=20,
)
if spec.kind == "online-paraformer":
return sherpa_onnx.OnlineRecognizer.from_paraformer(
tokens=p("tokens"),
encoder=p("encoder"),
decoder=p("decoder"),
num_threads=_NUM_THREADS,
provider=_PROVIDER,
decoding_method="greedy_search",
enable_endpoint_detection=True,
rule1_min_trailing_silence=rule1,
rule2_min_trailing_silence=rule2,
rule3_min_utterance_length=20,
)
raise ValueError(f"{spec.id} is not a streaming model (kind={spec.kind})")
+155 -13
View File
@@ -42,11 +42,25 @@ IDEAL_REF_DURATION_S = 8.0 # target window — long enough for prosody, short e
# is the empirical floor below which our zero-shot clone gets unstable.
MIN_SEGMENT_REF_DURATION_S = 3.0
# Clone-purity guards (speaker-hint fix): a per-speaker reference cut from
# mislabeled or boundary-adjacent audio mixes two people's voices and the
# resulting clone sounds "made up".
# * A slice below MIN_SLICE_DURATION_S is too short to be a reliable
# single-speaker sample (and diarization boundary jitter dominates it).
# * A slice whose edges come within ADJACENT_TURN_GUARD_S of a *different*
# speaker's turn risks bleeding that speaker's audio across the imprecise
# boundary — deprioritized (scoring preference, not a hard filter, so
# extraction still succeeds on dense dialogue).
MIN_SLICE_DURATION_S = 1.5
ADJACENT_TURN_GUARD_S = 0.3
def extract_speaker_clones(
vocals_path: str,
segments: list[dict],
out_dir: str,
*,
labels_source: str | None = None,
) -> dict[str, dict]:
"""Build a per-speaker reference sample from `vocals_path` + `segments`.
@@ -63,7 +77,20 @@ def extract_speaker_clones(
Speakers whose segments total < MIN_REF_DURATION_S are skipped we'd
rather fall back to the default TTS voice than ship a bad clone.
``labels_source`` records where the ``speaker_id`` labels came from
(``"pyannote"`` | ``"turns"`` | ``"heuristic"``; ``None`` = unknown,
treated as trusted for backward compatibility). ``"heuristic"`` labels
are silence-gap *estimates*, not voice identity a reference cut from
them routinely concatenates two people's audio, so extraction is skipped
entirely (the caller warns the user and falls back to the default voice).
"""
if labels_source == "heuristic":
logger.info(
"speaker_clone: skipping auto-clone extraction — speaker labels "
"are gap-based heuristic estimates, not voice identity"
)
return {}
if not vocals_path or not os.path.exists(vocals_path):
logger.info("speaker_clone: no vocals track at %s; skipping", vocals_path)
return {}
@@ -88,7 +115,12 @@ def extract_speaker_clones(
out: dict[str, dict] = {}
for speaker_id, items in by_speaker.items():
chosen = _pick_reference_slices(items)
chosen = _pick_reference_slices(
items,
speaker_id=speaker_id,
all_segments=segments,
labels_source=labels_source,
)
if not chosen:
logger.info(
"speaker_clone: %s has <%ss of usable audio; will fall back to default voice",
@@ -191,34 +223,144 @@ def extract_segment_refs(
return out
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
"""Re-transcribe a written reference clip and return that transcript.
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
the ASR segment's OWN text field, on the assumption that the segment's
timestamps and its transcribed text agree. They routinely don't — Whisper
(and friends) frequently drift on segment boundaries: a trailing word
audible in `[start, end]` but missing from `text`, or vice versa. When the
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
down and the clone can speak the mismatched reference text itself instead
of the target-language text it was given to synthesize (issue #1004).
Re-transcribing the *actual written clip* guarantees the pair matches by
construction the model doesn't care whether the original ASR text was
right, only that ref_text is what's really in ref_audio. `asr_backend` is
the caller's already-loaded active backend (duck-typed:
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
`{"text": ...}`); the model is already warm, so this costs one more short
transcribe call, not a fresh load. Falls back to `fallback_text` never
raises so a re-transcribe failure is a strict no-op, never a regression
from the original (matching) behavior.
"""
if asr_backend is None:
return fallback_text
try:
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
text = " ".join(
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
).strip()
return text or fallback_text
except Exception as e:
logger.warning(
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
ref_audio_path, e,
)
return fallback_text
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
or per-segment `seg_clones` from `extract_segment_refs`) into the single
executor round-trip the caller submits to the GPU pool, rather than one
dispatch per reference. Mutates and returns `clones` for a convenient
call-and-reassign at the call site.
"""
for entry in clones.values():
entry["ref_text"] = refine_ref_text(
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
)
return clones
# ── Internals ───────────────────────────────────────────────────────────────
def _pick_reference_slices(items: list[tuple[int, dict]]) -> list[tuple[int, dict]]:
def _adjacent_to_other_speaker(
seg: dict, speaker_id: str, all_segments: list[dict] | None
) -> bool:
"""True when `seg`'s edges come within ADJACENT_TURN_GUARD_S of (or
overlap) a segment attributed to a *different* speaker a boundary where
imprecise diarization timestamps risk bleeding the other voice into the
reference slice."""
if not all_segments:
return False
s0 = float(seg.get("start", 0.0))
s1 = float(seg.get("end", 0.0))
for other in all_segments:
if other is seg:
continue
if (other.get("speaker_id") or "Speaker 1") == speaker_id:
continue
o0 = float(other.get("start", 0.0))
o1 = float(other.get("end", 0.0))
# Signed gap between the two spans; negative = overlap.
if max(o0 - s1, s0 - o1) < ADJACENT_TURN_GUARD_S:
return True
return False
def _pick_reference_slices(
items: list[tuple[int, dict]],
*,
speaker_id: str | None = None,
all_segments: list[dict] | None = None,
labels_source: str | None = None,
) -> list[tuple[int, dict]]:
"""Select the subset of a speaker's segments to use as reference audio.
Strategy: take the single longest segment; if it's short, accumulate the
next longest ones in original order until we clear IDEAL_REF_DURATION_S.
Cap at MAX_REF_DURATION_S. Return [] if we can't reach MIN_REF_DURATION_S.
Strategy: rank candidates clean-first (not temporally adjacent to a
different speaker's turn — see ``_adjacent_to_other_speaker``), longest
first within each tier, and accumulate until IDEAL_REF_DURATION_S is
cleared. Adjacency is a scoring preference, NOT a hard filter on dense
dialogue where every slice borders another speaker, extraction still
succeeds using the adjacent ones. Two hard guards protect clone purity:
* slices shorter than MIN_SLICE_DURATION_S are rejected outright
(boundary jitter dominates them, so they're the likeliest to carry a
second speaker's audio);
* ``labels_source="heuristic"`` returns [] gap-based labels are not
voice identity, so no slice of them is safe to clone from.
Cap at MAX_REF_DURATION_S. Return [] if we can't reach
MIN_REF_DURATION_S. When ``all_segments``/``speaker_id`` are not
provided (legacy callers), adjacency scoring degrades to duration-only
the pre-guard behavior.
"""
if not items:
return []
if labels_source == "heuristic":
return []
if speaker_id is None:
speaker_id = items[0][1].get("speaker_id") or "Speaker 1"
# Longest-first candidates. Keep original indices so we can preserve order.
by_dur = sorted(
def _dur(pair) -> float:
return max(0.0, float(pair[1].get("end", 0.0)) - float(pair[1].get("start", 0.0)))
# Rank: clean (non-adjacent) before adjacent, longest first within each
# tier. Keep original indices so we can restore transcript order below.
ranked = sorted(
items,
key=lambda pair: (pair[1].get("end", 0.0) - pair[1].get("start", 0.0)),
reverse=True,
key=lambda pair: (
_adjacent_to_other_speaker(pair[1], speaker_id, all_segments),
-_dur(pair),
),
)
picked: list[tuple[int, dict]] = []
total = 0.0
for idx, seg in by_dur:
dur = max(0.0, float(seg.get("end", 0.0)) - float(seg.get("start", 0.0)))
if dur <= 0.0:
for idx, seg in ranked:
dur = _dur((idx, seg))
if dur < MIN_SLICE_DURATION_S:
continue
if total + dur > MAX_REF_DURATION_S and picked:
break
# Ranking is no longer duration-monotonic, so a later (shorter or
# adjacent) slice may still fit — skip, don't stop.
continue
picked.append((idx, seg))
total += dur
if total >= IDEAL_REF_DURATION_S:
+151 -10
View File
@@ -18,9 +18,17 @@ import logging
from typing import Iterable, Optional
from services.llm_backend import get_active_llm_backend, OffBackend
# Shared LLM-output divergence guard (length window + target-script +
# critique-echo). Lives in translator; translator never imports this module,
# so there is no import cycle.
from services.translator import refine_output_ok
logger = logging.getLogger("omnivoice.speech_rate")
# LLM Skills registry id — Settings → LLM Skills can disable the slot-fit
# LLM pass or route it to a specific provider. Disabled == the no-llm path.
_SKILL_ID = "slot_fitting"
# Per-language read-speed estimates (chars/sec at natural pace, counting
# Python `len()` codepoints — not phonemes or graphemes). These are
# rough; real speakers vary wildly. Numbers below come from a mix of
@@ -87,9 +95,15 @@ _EXPAND_PROMPT = """\
You are a dubbing writer. The user will give you a translated line + the exact
time slot it must fit. The current line is TOO SHORT add natural filler or
gently flesh out the thought while keeping the meaning the same. Aim for a
reading duration that matches the slot.
reading duration that matches the slot. Never invent new information, names,
or dialogue that is not already in the line; do not more than double the line.
Reply with ONLY the new line. No quotes, no commentary."""
# Below this predicted rate ratio a line can never honestly fill its slot —
# any LLM "expansion" that far would be fabricated dialogue. Skip the expand
# pass entirely and keep the short line (slot-aware TTS absorbs the silence).
_MIN_EXPANDABLE_RATIO = 0.15
def adjust_for_slot(
text: str,
@@ -97,16 +111,47 @@ def adjust_for_slot(
slot_seconds: float,
target_lang: str,
source_text: Optional[str] = None,
strict: bool = False,
) -> dict:
"""Return `{text, rate_ratio, attempts, error?}`.
Falls back to the input text if the LLM is off or the loop gives up.
"""
initial_ratio = rate_ratio(text, slot_seconds, target_lang)
if TOL_LOW <= initial_ratio <= TOL_HIGH:
return {"text": text, "rate_ratio": initial_ratio, "attempts": 0}
llm = get_active_llm_backend()
``strict`` (Autofit mode) changes exactly one thing: the accepted upper
bound is 1.0 instead of ``TOL_HIGH`` the line must fit *within* the
slot, never overrun it so the target-language reading time can't exceed
the segment and push the video timing out. Lines under ``TOL_LOW`` still
go through the LLM expand pass in strict mode too (same as loose mode);
padding is bounded by the divergence guard below, and a line under
``_MIN_EXPANDABLE_RATIO`` is never expanded at all it could only "fill"
the slot with fabricated dialogue, so it stays short. Best-effort: after
``MAX_ATTEMPTS`` it returns the closest candidate seen, so a stubborn line
degrades gracefully.
Every LLM reply is validated with ``translator.refine_output_ok`` against
the ORIGINAL input ``text`` (not the previous candidate divergence
compounds across attempts otherwise). A reply that fails the guard is
discarded: the attempt is burned, ``current``/``best`` stay put, and if
nothing valid ever came back the input text is returned with
``error="fit-diverged"`` a hallucinating model can no longer invent the
dub line (v0.3.9 field report).
"""
tol_high = 1.0 if strict else TOL_HIGH
initial_ratio = rate_ratio(text, slot_seconds, target_lang)
if TOL_LOW <= initial_ratio <= tol_high:
return {"text": text, "rate_ratio": initial_ratio, "attempts": 0}
if initial_ratio < _MIN_EXPANDABLE_RATIO:
return {
"text": text,
"rate_ratio": initial_ratio,
"attempts": 0,
"error": "fit-skip-short",
}
from services import llm_skills
# `active=` forwards this module's (monkeypatch-able) name so the
# no-override path is byte-identical to the pre-skills behavior.
llm = llm_skills.skill_backend(_SKILL_ID, active=lambda: get_active_llm_backend())
if isinstance(llm, OffBackend):
return {
"text": text,
@@ -117,9 +162,10 @@ def adjust_for_slot(
current = text
best = (current, initial_ratio)
diverged = False
for attempt in range(1, MAX_ATTEMPTS + 1):
r = rate_ratio(current, slot_seconds, target_lang)
if TOL_LOW <= r <= TOL_HIGH:
if TOL_LOW <= r <= tol_high:
return {"text": current, "rate_ratio": r, "attempts": attempt - 1}
system = _TRIM_PROMPT if r > 1.0 else _EXPAND_PROMPT
@@ -134,23 +180,43 @@ def adjust_for_slot(
user_lines.append(f"Source line (for meaning): {source_text}")
try:
next_text = llm.chat(system=system, user="\n".join(user_lines))
next_text = llm.chat(
system=system, user="\n".join(user_lines),
temperature=0.2, # pinned like the Fast path — default 1.0 drifts/invents
)
except Exception as e:
logger.warning("speech-rate attempt %d failed: %s", attempt, e)
return {"text": best[0], "rate_ratio": best[1], "attempts": attempt - 1, "error": str(e)}
if next_text and next_text.strip():
current = next_text.strip()
candidate = next_text.strip()
# Divergence guard — validate against the ORIGINAL text, not
# `current`: each accepted reply becomes the next prompt's input,
# so per-step checks would let drift compound across attempts.
ok, reason = refine_output_ok(text, candidate, target_lang)
if not ok:
diverged = True
logger.warning(
"speech-rate attempt %d rejected (%s) — discarding candidate",
attempt, reason,
)
continue # attempt burned; current/best untouched
current = candidate
new_r = rate_ratio(current, slot_seconds, target_lang)
# Keep the best candidate seen so far in case we exhaust retries.
if abs(new_r - 1.0) < abs(best[1] - 1.0):
best = (current, new_r)
return {
out = {
"text": best[0],
"rate_ratio": best[1],
"attempts": MAX_ATTEMPTS,
}
# Every usable reply diverged and the input text survived unchanged —
# surface it on the row (rate_error in dub_translate, like fit-budget).
if diverged and best[0] == text:
out["error"] = "fit-diverged"
return out
def adjust_many(pairs: Iterable[tuple[str, float, str, Optional[str]]]) -> list[dict]:
@@ -162,3 +228,78 @@ def adjust_many(pairs: Iterable[tuple[str, float, str, Optional[str]]]) -> list[
adjust_for_slot(t, slot_seconds=s, target_lang=tl, source_text=src)
for (t, s, tl, src) in pairs
]
async def adjust_for_slot_many(
items: Iterable[tuple],
*,
executor=None,
concurrency: Optional[int] = None,
deadline: Optional[float] = None,
loop=None,
) -> dict:
"""Fan `adjust_for_slot` out across many segments concurrently, bounded by a
shared wall-clock ``deadline``.
``items``: iterable of ``(key, text, slot_seconds, target_lang,
source_text_or_None, strict)``. Returns ``{key: adjust_for_slot_result}``.
Why this exists: the Autofit fit pass used to run one `adjust_for_slot` per
segment *sequentially* and *outside* any budget, so a 50-segment dub against
a slow/rate-limited LLM spun ~50×(per-call timeout) unbounded. Here every
segment runs on the executor under a bounded ``asyncio.Semaphore``, and any
segment still running when the shared ``deadline`` passes degrades to a
no-fit result (input text kept, predicted ``rate_ratio``, ``error`` =
``"fit-budget"``) instead of hanging the translate. ``deadline`` is an
absolute ``loop.time()``; ``None`` disables the bound (run to completion).
"""
import asyncio
import os
loop = loop or asyncio.get_running_loop()
items = list(items)
if not items:
return {}
sem = asyncio.Semaphore(concurrency or int(os.environ.get("OMNIVOICE_LLM_CONCURRENCY", "6")))
async def _one(key, text, slot, tgt, src, strict):
async with sem:
res = await loop.run_in_executor(
executor,
lambda: adjust_for_slot(
text, slot_seconds=slot, target_lang=tgt,
source_text=src, strict=strict,
),
)
return key, res
def _degraded(text, slot, tgt) -> dict:
return {
"text": text,
"rate_ratio": rate_ratio(text, slot, tgt),
"attempts": 0,
"error": "fit-budget",
}
tasks = [asyncio.ensure_future(_one(*it)) for it in items]
if deadline is None:
pairs_out = await asyncio.gather(*tasks)
return dict(pairs_out)
timeout = max(0.0, deadline - loop.time())
done, _pending = await asyncio.wait(tasks, timeout=timeout)
out: dict = {}
for task, it in zip(tasks, items):
key, text, slot, tgt = it[0], it[1], it[2], it[3]
if task in done and not task.cancelled():
try:
k, res = task.result()
out[k] = res
continue
except Exception as e: # noqa: BLE001 — one slow seg must not sink the pass
logger.warning("fit segment %s failed: %s", key, e)
else:
task.cancel() # stop awaiting; the executor thread is abandoned (#730 pattern)
out[key] = _degraded(text, slot, tgt)
return out
+471
View File
@@ -0,0 +1,471 @@
"""Storage usage report for Settings → Storage.
Computes, for everything the app owns on disk:
* per-volume totals (total / used / free, grouped by ``st_dev`` so two
roots on the same disk are reported once),
* per-category directory sizes the HF model cache (with the largest
model dirs), the app data dir (broken into voices / outputs / dub_jobs /
batch / preview / database / logs / other subtotals), the per-engine
venvs under ``backend/engines/*/.venv`` (+ the app venv), and any
``omnivoice*`` entries in the OS temp dir,
* server-side ``warnings`` (low disk, volume pressure, unreadable paths)
so every client renders the same guidance.
Directory walks are **bounded**: each top-level category gets a deadline
(default 10 s) and returns a partial total (``complete: false`` + an
``unreadable`` warning with ``reason: "timeout"``) when it expires. Results
are cached in-process for 5 minutes; ``refresh`` bypasses the cache. The API
layer runs the whole build in a worker thread so the event loop never blocks.
"""
from __future__ import annotations
import glob
import os
import shutil
import sys
import tempfile
import threading
import time
from pathlib import Path
CACHE_TTL_SECONDS = 300.0
CATEGORY_TIMEOUT_SECONDS = 10.0
TOP_MODEL_COUNT = 10
VOLUME_PRESSURE_PERCENT = 90.0
DEFAULT_MIN_FREE_GB = 10 # callers pass setup.wizard.MIN_FREE_GB — this is the standalone fallback
# DATA_DIR children we know by name (core.config constants + routers that
# write there). Anything else lands in the "other" subtotal so the numbers
# always add up to the real on-disk footprint.
_DATA_CHILD_DIRS = ("voices", "outputs", "dub_jobs", "batch", "preview")
_DB_PREFIX = "omnivoice.db" # omnivoice.db + -wal / -shm / -journal
_LOG_FILES = ("crash_log.txt", "error_journal.jsonl")
_LOG_PREFIX = "omnivoice.log" # rolling log + rotations
_GB = 1024 ** 3
def default_engines_dir() -> str:
"""``backend/engines`` — where per-engine venvs live (`<id>/.venv`)."""
return str(Path(__file__).resolve().parents[1] / "engines")
def default_app_venv() -> str | None:
"""The venv this backend runs from, when it is one (None for system python)."""
if sys.prefix != getattr(sys, "base_prefix", sys.prefix):
return sys.prefix
return None
def _existing_ancestor(path: str) -> str:
"""Deepest existing ancestor of ``path`` (for disk_usage on missing dirs)."""
p = os.path.abspath(path)
while p and not os.path.exists(p):
parent = os.path.dirname(p)
if parent == p:
break
p = parent
return p
def _mount_point(path: str) -> str:
"""Mount point of the volume holding ``path`` (best-effort, cheap)."""
p = _existing_ancestor(path)
try:
while p and not os.path.ismount(p):
parent = os.path.dirname(p)
if parent == p:
break
p = parent
except OSError:
pass
return p or os.path.abspath(os.sep)
def _dir_size(path: str, deadline: float) -> tuple[int, bool, str | None]:
"""du-style size of ``path``: ``(bytes, complete, first_unreadable_path)``.
Never follows symlinks (lstat + walk default), never raises. Stops early
and reports ``complete=False`` once ``deadline`` (time.monotonic) passes.
"""
err_path: str | None = None
def _onerror(e: OSError) -> None:
nonlocal err_path
if err_path is None:
err_path = getattr(e, "filename", None) or path
try:
if not os.path.exists(path):
return 0, True, None
if not os.path.isdir(path):
return os.lstat(path).st_size, True, None
except OSError:
return 0, True, path
total = 0
complete = True
for root, _dirs, files in os.walk(path, onerror=_onerror):
if time.monotonic() > deadline:
complete = False
break
for name in files:
fp = os.path.join(root, name)
try:
total += os.lstat(fp).st_size
except OSError:
if err_path is None:
err_path = fp
return total, complete, err_path
def _sum_files(paths: list[str]) -> int:
total = 0
for p in paths:
try:
total += os.lstat(p).st_size
except OSError:
pass
return total
def _hf_model_dirs(cache_dir: str) -> list[str]:
"""`models--org--name` dirs in the cache root and its `hub/` child.
HF_HUB_CACHE points straight at the hub dir; HF_HOME needs `/hub`
appended scanning both covers either env resolution.
"""
out: list[str] = []
for base in (cache_dir, os.path.join(cache_dir, "hub")):
try:
with os.scandir(base) as it:
out.extend(
e.path for e in it
if e.name.startswith("models--") and e.is_dir(follow_symlinks=False)
)
except OSError:
continue
return out
def _model_display_name(dir_name: str) -> str:
return dir_name.removeprefix("models--").replace("--", "/")
def build_report(
*,
data_dir: str,
hf_cache_dir: str,
engines_dir: str | None = None,
app_venv: str | None = None,
temp_root: str | None = None,
min_free_gb: float = DEFAULT_MIN_FREE_GB,
category_timeout: float = CATEGORY_TIMEOUT_SECONDS,
) -> dict:
"""Build the full storage report (synchronous; call from a worker thread)."""
engines_dir = engines_dir if engines_dir is not None else default_engines_dir()
temp_root = temp_root if temp_root is not None else tempfile.gettempdir()
warnings: list[dict] = []
categories: list[dict] = []
def _warn_unreadable(category_id: str, path: str, reason: str) -> None:
warnings.append({
"kind": "unreadable",
"severity": "warning",
"category_id": category_id,
"path": path,
"reason": reason,
})
def _finish(category_id: str, cat: dict, complete: bool, err_path: str | None) -> None:
cat["complete"] = complete
if not complete:
_warn_unreadable(category_id, cat["path"], "timeout")
if err_path is not None:
_warn_unreadable(category_id, err_path, "permission")
# ── 1. HF model cache (+ top model dirs) ───────────────────────────────
deadline = time.monotonic() + category_timeout
hf_total = 0
hf_complete = True
hf_err: str | None = None
models: list[dict] = []
model_dirs = set(_hf_model_dirs(hf_cache_dir))
seen: set[str] = set()
for mdir in sorted(model_dirs):
size, ok, err = _dir_size(mdir, deadline)
hf_total += size
hf_complete = hf_complete and ok
hf_err = hf_err or err
models.append({"name": _model_display_name(os.path.basename(mdir)), "bytes": size})
seen.add(os.path.realpath(mdir))
# Non-model remainder of the cache (datasets, xet chunks, token file, …):
# walk the top-level entries that aren't model dirs so the category total
# reflects the whole cache, not just models.
try:
with os.scandir(hf_cache_dir) as it:
entries = list(it)
except OSError:
entries = []
if os.path.exists(hf_cache_dir):
hf_err = hf_err or hf_cache_dir
for e in entries:
if os.path.realpath(e.path) in seen:
continue
if e.name == "hub":
# hub/ holds the model dirs (already counted) + misc; count the rest.
try:
with os.scandir(e.path) as hub_it:
for h in hub_it:
if os.path.realpath(h.path) in seen:
continue
size, ok, err = _dir_size(h.path, deadline)
hf_total += size
hf_complete = hf_complete and ok
hf_err = hf_err or err
except OSError:
hf_err = hf_err or e.path
continue
size, ok, err = _dir_size(e.path, deadline)
hf_total += size
hf_complete = hf_complete and ok
hf_err = hf_err or err
models.sort(key=lambda m: m["bytes"], reverse=True)
hf_cat = {
"id": "hf_cache",
"path": hf_cache_dir,
"exists": os.path.isdir(hf_cache_dir),
"bytes": hf_total,
"items": models[:TOP_MODEL_COUNT],
}
_finish("hf_cache", hf_cat, hf_complete, hf_err)
categories.append(hf_cat)
# ── 2. App data dir, broken into subtotals ─────────────────────────────
deadline = time.monotonic() + category_timeout
data_complete = True
data_err: str | None = None
children: list[dict] = []
claimed: set[str] = set()
for name in _DATA_CHILD_DIRS:
p = os.path.join(data_dir, name)
size, ok, err = _dir_size(p, deadline)
data_complete = data_complete and ok
data_err = data_err or err
claimed.add(name)
children.append({"id": name, "path": p, "bytes": size, "complete": ok})
db_files = sorted(glob.glob(os.path.join(glob.escape(data_dir), _DB_PREFIX + "*")))
claimed.update(os.path.basename(p) for p in db_files)
children.append({
"id": "database",
"path": os.path.join(data_dir, _DB_PREFIX),
"bytes": _sum_files(db_files),
"complete": True,
})
log_files = sorted(glob.glob(os.path.join(glob.escape(data_dir), _LOG_PREFIX + "*")))
log_files += [os.path.join(data_dir, n) for n in _LOG_FILES]
claimed.update(os.path.basename(p) for p in log_files)
children.append({
"id": "logs",
"path": data_dir,
"bytes": _sum_files(log_files),
"complete": True,
})
other_bytes = 0
try:
with os.scandir(data_dir) as it:
for e in it:
if e.name in claimed:
continue
if e.is_dir(follow_symlinks=False):
size, ok, err = _dir_size(e.path, deadline)
other_bytes += size
data_complete = data_complete and ok
data_err = data_err or err
else:
try:
other_bytes += e.stat(follow_symlinks=False).st_size
except OSError:
data_err = data_err or e.path
except OSError:
if os.path.exists(data_dir):
data_err = data_err or data_dir
children.append({"id": "other", "path": data_dir, "bytes": other_bytes, "complete": True})
data_cat = {
"id": "data",
"path": data_dir,
"exists": os.path.isdir(data_dir),
"bytes": sum(c["bytes"] for c in children),
"children": children,
}
_finish("data", data_cat, data_complete, data_err)
categories.append(data_cat)
# ── 3. Engine venvs (+ the app venv) ───────────────────────────────────
deadline = time.monotonic() + category_timeout
venv_total = 0
venv_complete = True
venv_err: str | None = None
venv_items: list[dict] = []
try:
with os.scandir(engines_dir) as it:
engine_dirs = sorted(e.path for e in it if e.is_dir(follow_symlinks=False))
except OSError:
engine_dirs = []
for edir in engine_dirs:
venv_dir = os.path.join(edir, ".venv")
if not os.path.isdir(venv_dir):
continue
size, ok, err = _dir_size(venv_dir, deadline)
venv_total += size
venv_complete = venv_complete and ok
venv_err = venv_err or err
venv_items.append({"name": os.path.basename(edir), "bytes": size})
if app_venv:
size, ok, err = _dir_size(app_venv, deadline)
venv_total += size
venv_complete = venv_complete and ok
venv_err = venv_err or err
venv_items.append({"name": "app", "bytes": size})
venv_items.sort(key=lambda m: m["bytes"], reverse=True)
venv_cat = {
"id": "engine_venvs",
"path": engines_dir,
"exists": os.path.isdir(engines_dir),
"bytes": venv_total,
"items": venv_items,
}
_finish("engine_venvs", venv_cat, venv_complete, venv_err)
categories.append(venv_cat)
# ── 4. Temp/working files the app owns (omnivoice* in the OS temp dir) ─
deadline = time.monotonic() + category_timeout
tmp_total = 0
tmp_complete = True
tmp_err: str | None = None
for p in sorted(glob.glob(os.path.join(glob.escape(temp_root), "omnivoice*"))):
size, ok, err = _dir_size(p, deadline)
tmp_total += size
tmp_complete = tmp_complete and ok
tmp_err = tmp_err or err
tmp_cat = {
"id": "temp",
"path": temp_root,
"exists": os.path.isdir(temp_root),
"bytes": tmp_total,
"items": [],
}
_finish("temp", tmp_cat, tmp_complete, tmp_err)
categories.append(tmp_cat)
# ── Volumes: group category roots by device, disk_usage once each ──────
roots = {"hf_cache": hf_cache_dir, "data": data_dir, "engine_venvs": engines_dir, "temp": temp_root}
by_dev: dict[object, dict] = {}
for cid, root in roots.items():
anchor = _existing_ancestor(root)
try:
dev: object = os.stat(anchor).st_dev
except OSError:
dev = anchor
if dev not in by_dev:
try:
usage = shutil.disk_usage(anchor)
except OSError:
continue
by_dev[dev] = {
"path": _mount_point(anchor),
"total_bytes": usage.total,
"used_bytes": usage.used,
"free_bytes": usage.free,
"used_percent": round(usage.used / usage.total * 100.0, 1) if usage.total else 0.0,
"roots": [],
}
by_dev[dev]["roots"].append(cid)
volumes = list(by_dev.values())
# ── Server-side warnings ────────────────────────────────────────────────
for v in volumes:
free_gb = v["free_bytes"] / _GB
base = {
"path": v["path"],
"free_gb": round(free_gb, 1),
"min_free_gb": min_free_gb,
"roots": v["roots"],
}
if free_gb < min_free_gb:
warnings.append({"kind": "low_disk", "severity": "critical", **base})
elif free_gb < 2 * min_free_gb:
warnings.append({"kind": "low_disk", "severity": "low", **base})
if v["used_percent"] > VOLUME_PRESSURE_PERCENT and ({"hf_cache", "data"} & set(v["roots"])):
warnings.append({
"kind": "volume_pressure",
"severity": "warning",
"path": v["path"],
"used_percent": v["used_percent"],
"roots": v["roots"],
})
# Order: critical first, then the rest in computed order (stable sort).
warnings.sort(key=lambda w: 0 if w["severity"] == "critical" else 1)
return {
"generated_at": time.time(),
"min_free_gb": min_free_gb,
"volumes": volumes,
"categories": categories,
"warnings": warnings,
}
# ── In-process cache (5-minute TTL, refresh bypasses) ──────────────────────
_cache_lock = threading.Lock()
_cache: dict = {"key": None, "ts": 0.0, "report": None}
def get_report(
*,
data_dir: str,
hf_cache_dir: str,
engines_dir: str | None = None,
app_venv: str | None = None,
temp_root: str | None = None,
min_free_gb: float = DEFAULT_MIN_FREE_GB,
category_timeout: float = CATEGORY_TIMEOUT_SECONDS,
refresh: bool = False,
ttl: float = CACHE_TTL_SECONDS,
) -> dict:
"""Cached ``build_report``. ``refresh=True`` forces a rescan."""
key = (data_dir, hf_cache_dir, engines_dir, app_venv, temp_root, min_free_gb)
if not refresh:
with _cache_lock:
fresh = (
_cache["report"] is not None
and _cache["key"] == key
and (time.monotonic() - _cache["ts"]) < ttl
)
if fresh:
return {**_cache["report"], "cached": True}
report = build_report(
data_dir=data_dir,
hf_cache_dir=hf_cache_dir,
engines_dir=engines_dir,
app_venv=app_venv,
temp_root=temp_root,
min_free_gb=min_free_gb,
category_timeout=category_timeout,
)
with _cache_lock:
_cache.update(key=key, ts=time.monotonic(), report=report)
return {**report, "cached": False}
def clear_cache() -> None:
"""Testing hook — drop the in-process cache."""
with _cache_lock:
_cache.update(key=None, ts=0.0, report=None)
+4
View File
@@ -132,6 +132,10 @@ class IsolatedFasterWhisperBackend(SubprocessASRBackend):
id = "faster-whisper-isolated"
display_name = "Faster-Whisper (crash-isolated subprocess)"
# Same engine as FasterWhisperBackend, so the same device support — the
# sidecar picks cuda/cpu itself via `_device()`. Without this the registry
# default ("cpu",) would dishonestly report cpu_only routing on CUDA hosts.
gpu_compat = ("cuda", "cpu")
@classmethod
def is_available(cls) -> tuple[bool, str]:
+79
View File
@@ -0,0 +1,79 @@
"""
Deterministic polish for dictation finals (dictation v2).
Every ``final`` that leaves ``/ws/transcribe`` passes through
:func:`polish_text` so pasted dictation reads like typed text:
* leading capital -- Latin scripts only (CJK/Cyrillic/etc. untouched),
* terminal punctuation -- a period is appended unless the text already
ends with sentence-terminal punctuation (incl. the CJK fullwidth forms),
* doubled spaces collapsed, leading/trailing whitespace stripped.
Purely rule-based -- no model, no locale detection, no network -- so it is
byte-for-byte reproducible and idempotent (``polish(polish(x)) == polish(x)``).
CJK codepoints below are ``\\u``-escaped on purpose: this is functional
punctuation handling (allowed), and the escapes keep this file outside the
literal-CJK scan in ``tests/test_no_hardcoded_cjk.py`` without growing its
allowlist.
"""
from __future__ import annotations
import re
# Sentence-terminal punctuation that already "closes" a final -- Latin plus
# the CJK fullwidth forms (U+3002 ideographic full stop, U+FF01 !, U+FF1F ?)
# and ellipsis. A trailing closing quote/bracket after one of these still
# counts as terminated ("He said \"hi.\"").
_TERMINAL = ".!?\u2026\u3002\uff01\uff1f"
_CLOSERS = "\"'\u201d\u2019\u00bb\u203a)]}\u300d\u300f\uff09\u3011"
# A dangling clause separator at the very end (ASR often stops mid-breath on
# a comma) is swapped for a stop instead of stacking ",." punctuation.
# Latin , ; : plus the CJK forms U+3001 U+FF0C U+FF1B U+FF1A.
_DANGLING = ",;:\u3001\uff0c\uff1b\uff1a"
# CJK codepoints (kana, unified ideographs, compatibility + halfwidth forms)
# -- used to pick the fullwidth stop U+3002 over "." for CJK sentences.
_CJK = re.compile(
"[\u3040-\u30ff\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff\uff66-\uff9f]"
)
_MULTISPACE = re.compile(r"[ \t]{2,}")
def _is_latin_lower(ch: str) -> bool:
"""Lowercase letter in a Latin block (ASCII, Latin-1, Latin Extended-A/B).
Capitalization is meaningless (CJK) or presumptuous (Cyrillic, Greek --
the model's casing is trusted) outside Latin scripts.
"""
return ch.islower() and ord(ch) <= 0x024F
def polish_text(text: str) -> str:
"""Normalise one dictation final. Empty/whitespace-only input -> ``""``."""
if not text:
return ""
out = _MULTISPACE.sub(" ", text).strip()
if not out:
return ""
# Leading capital (Latin scripts only).
if _is_latin_lower(out[0]):
out = out[0].upper() + out[1:]
# Already terminated -- possibly behind a closing quote/bracket?
body = out.rstrip(_CLOSERS)
if body and body[-1] in _TERMINAL:
return out
# Swap a dangling comma/colon for the stop instead of stacking ",.".
if out[-1] in _DANGLING:
out = out[:-1].rstrip()
if not out:
return ""
# Script-matched stop: fullwidth U+3002 when the sentence ends in CJK.
out += "\u3002" if _CJK.search(out[-1]) else "."
return out
+63 -5
View File
@@ -92,9 +92,11 @@ REGISTRY: dict[str, dict] = {
"category": "llm",
"needs_key": True,
"notes": (
"Any OpenAI-compatible endpoint: GPT-4/5 (OpenAI), Claude (via OpenRouter), "
"Gemini (OpenAI-compat mode), DeepSeek, Qwen, Ollama, LM Studio. "
"Set TRANSLATE_BASE_URL + TRANSLATE_API_KEY + TRANSLATE_MODEL."
"Uses the LLM provider you configure in Settings → LLM Providers "
"(route it via the 'Dub translation' skill in Settings → LLM Skills): "
"GPT (OpenAI), Claude (via OpenRouter), Gemini, DeepSeek, Qwen, "
"Ollama, LM Studio. Power-user env override: TRANSLATE_BASE_URL + "
"TRANSLATE_API_KEY + TRANSLATE_MODEL."
),
},
}
@@ -120,16 +122,62 @@ def _probe(entry: dict) -> tuple[bool, str]:
return False, f"import {mod!r} failed: {e}"
def install_command(engine: "str | dict | None") -> str | None:
"""The exact shell command that makes this engine importable, or None.
Single source of truth for the install string. BOTH the proactive Install
affordance in the Engine selector (via list_engines' ``install_command``
field) AND the translate-time 400 error (dub_translate.py) read from here,
so the command a user is told to run can never drift between the two
surfaces. Returns None when the engine needs no separate install either
it's unknown or its dependency is a core dep already pinned in
``pyproject.toml`` (e.g. NLLB transformers), in which case a
``uv pip install`` line would be misleading.
"""
entry = engine if isinstance(engine, dict) else REGISTRY.get(engine) if engine else None
pkg = entry.get("pip_package") if entry else None
return f"uv pip install {pkg}" if pkg else None
def _llm_configured() -> tuple[bool, "str | None"]:
"""Whether the LLM translation engine has something to call, and via what.
Resolution mirrors the translate-time path in dub_translate.py: the
"dub_translation" LLM skill (per-skill override active provider from
Settings LLM Providers) first, then the TRANSLATE_* env override. Lets
the Engine dropdown say "ready via <provider>" / "needs setup" up front
instead of a per-segment failure after the user clicks Translate.
"""
try:
from services import llm_skills
res = llm_skills.resolve_skill("dub_translation")
if res.ready and res.provider is not None:
return True, res.provider.display_name
except Exception: # noqa: BLE001 — a probe must never break list_engines()
logger.debug("dub_translation skill probe failed", exc_info=True)
if os.environ.get("TRANSLATE_BASE_URL") or os.environ.get("TRANSLATE_API_KEY"):
return True, "env"
return False, None
def list_engines() -> list[dict]:
"""Return a UI-ready list with per-engine availability stamped in."""
out = []
for e in REGISTRY.values():
installed, reason = _probe(e)
out.append({
entry = {
**e,
"installed": installed,
"availability_reason": reason,
})
"install_command": install_command(e),
}
# LLM engines additionally need a provider/key — surface configured-ness
# so the UI can distinguish "importable" from "actually ready to call".
if e.get("category") == "llm":
configured, via = _llm_configured()
entry["configured"] = configured
entry["configured_via"] = via
out.append(entry)
return out
@@ -177,6 +225,16 @@ async def run_pip(args: list[str], timeout: float = 600.0) -> tuple[int, str]:
"""
base = _installer_cmd()
using_uv = base[:1] == ["uv"]
# Pin `uv pip` to the interpreter the backend ACTUALLY runs under. The desktop
# spawns `<venv>/bin/python -m uvicorn` WITHOUT exporting VIRTUAL_ENV, so bare
# `uv pip install` finds no venv and 500s with "No virtual environment found"
# (#529/#527) — and the `--system` branch below never fires, because the
# running interpreter genuinely IS in a venv (uv just can't auto-discover it).
# `--python sys.executable` targets the same interpreter _probe()/is_installed()
# import from, and takes precedence when both flags are present, so the Docker
# `--system` path is unaffected.
if using_uv and args and args[0] in ("install", "uninstall") and "--python" not in args:
args = [args[0], "--python", sys.executable, *args[1:]]
if using_uv and not _in_virtualenv() and args and args[0] in ("install", "uninstall") and "--system" not in args:
args = [args[0], "--system", *args[1:]]
cmd = base + args
+196 -36
View File
@@ -59,8 +59,9 @@ _ADAPT_PROMPT = """\
You are a cinematic dubbing writer. Rewrite the literal translation using the
editor's critique so it sounds natural, in-character, and fits the speaker's
time slot. Keep meaning faithful but prefer native idiom over word-for-word
accuracy. The output MUST be written in the same target language and script
as the literal translation never switch language or transliterate.
accuracy. Never introduce facts, names, or dialogue that are not present in
the source line. The output MUST be written in the same target language and
script as the literal translation never switch language or transliterate.
Reply ONLY with the adapted translation no quotes, no headers, no code
fences, no commentary."""
@@ -93,28 +94,138 @@ def _looks_like_target_script(text: str, code: str, threshold: float = 0.5) -> b
return (inside / len(letters)) >= threshold
def _llm_client():
"""Lazy-build the OpenAI-compatible client. Returns None if no key + no local base_url."""
# ── Divergence guard (shared with speech_rate's Autofit fit pass) ────────────
# For every Latin-script target `_looks_like_target_script` passes ANY text
# unconditionally (no `_SCRIPT_RANGES` entry), so it was the only — and for
# es/de/fr/… a no-op — gate on the ADAPT/fit LLM output. These checks close
# that gap for the whole class: runaway length (hallucinated dialogue,
# refusals, commentary) and the REFLECT critique echoed back as the "line".
_SHORT_REF_CHARS = 20 # below this, a length *ratio* is meaningless
_SHORT_REF_ABS_SLACK = 120 # …use an absolute cap instead: ref + this many chars
def _refine_ratio_bounds() -> tuple[float, float]:
"""Accepted ``len(candidate)/len(reference)`` window for LLM refine output.
Anything outside is treated as divergence and the caller degrades to its
input text. Defaults [0.4, 2.5]; env-tunable like the cinematic budget."""
try:
from openai import OpenAI
except ImportError:
logger.warning("openai package not installed — cinematic mode unavailable.")
return None
base_url = os.environ.get("TRANSLATE_BASE_URL")
api_key = (
os.environ.get("TRANSLATE_API_KEY")
or os.environ.get("OPENAI_API_KEY")
or ("local" if base_url else None) # local providers often accept any key
)
if not api_key:
return None
kw = {"api_key": api_key}
if base_url:
kw["base_url"] = base_url
return OpenAI(**kw)
lo = float(os.environ.get("OMNIVOICE_REFINE_RATIO_MIN", "0.4"))
except ValueError:
lo = 0.4
try:
hi = float(os.environ.get("OMNIVOICE_REFINE_RATIO_MAX", "2.5"))
except ValueError:
hi = 2.5
return lo, hi
def _norm_overlap_text(s: str) -> str:
return " ".join(s.lower().split())
def _echoes_critique(candidate: str, critique: str) -> bool:
"""True when the "adaptation" is really the REFLECT critique leaking through.
Deterministic on purpose (no fuzzy matching): exact match after
case/whitespace normalization; containment the full critique inside the
candidate always counts, the candidate inside the critique only when it
covers most of it (critiques legitimately quote short phrases from the
line); or >0.8 token-set overlap.
"""
c = _norm_overlap_text(candidate)
k = _norm_overlap_text(critique)
if not c or not k:
return False
if c == k:
return True
if k in c: # critique embedded in the output
return True
if c in k and len(c) >= 0.6 * len(k): # output ≈ a big chunk of the critique
return True
ct, kt = set(c.split()), set(k.split())
union = ct | kt
return bool(union) and len(ct & kt) / len(union) > 0.8
def refine_output_ok(
reference: str,
candidate: str,
target_lang: str,
*,
critique: str | None = None,
max_ratio: float | None = None,
) -> tuple[bool, str | None]:
"""Sanity-check one LLM refine output against the text it was rewriting.
Shared by the Cinematic ADAPT step here and by ``speech_rate``'s Autofit
fit pass (speech_rate imports this; translator never imports speech_rate,
so there is no cycle). Returns ``(ok, reason)`` ``reason`` is ``None``
when ok, otherwise a short machine-readable tag for logs/error mapping.
Checks, in order:
script candidate must look like the target language's script
(``_looks_like_target_script``; Latin-script targets pass, as before);
length ``len(candidate)/len(reference)`` must sit inside
[``OMNIVOICE_REFINE_RATIO_MIN``, ``OMNIVOICE_REFINE_RATIO_MAX``]
(default 0.42.5; ``max_ratio`` overrides the upper bound). References
shorter than ~20 chars use an absolute cap (reference + 120 chars)
instead a two-word line legitimately doubles or halves;
critique echo the candidate must not be the critique itself.
"""
cand = (candidate or "").strip()
ref = (reference or "").strip()
if not cand:
return False, "empty"
if not _looks_like_target_script(cand, target_lang):
return False, f"wrong-script:{target_lang}"
lo, hi = _refine_ratio_bounds()
if max_ratio is not None:
hi = max_ratio
if ref:
if len(ref) < _SHORT_REF_CHARS:
if len(cand) > len(ref) + _SHORT_REF_ABS_SLACK:
return False, f"length-abs:{len(cand)}>{len(ref)}+{_SHORT_REF_ABS_SLACK}"
else:
ratio = len(cand) / len(ref)
if not (lo <= ratio <= hi):
return False, f"length-ratio:{ratio:.2f}"
if critique and _echoes_critique(cand, critique):
return False, "critique-echo"
return True, None
# The LLM Skills registry entry this pipeline resolves through — lets the
# user disable Cinematic/Autofit's LLM use or route it to a specific provider
# (Settings → LLM Skills) independently of the other LLM features.
_SKILL_ID = "cinematic_translation"
def _llm_client():
"""Lazy-build the OpenAI-compatible client for the Cinematic skill.
Resolves through the LLM Skills registry: per-skill provider override
global active provider (Settings LLM Providers). The registry's
``custom`` provider still maps ``TRANSLATE_BASE_URL``/``TRANSLATE_API_KEY``,
so legacy env setups keep working. Returns None if the skill is disabled
or no provider is configured the callers' Fast-fallback path.
The registry builds the client with ``max_retries=0`` (see
``llm_skills.resolve_skill_client``) so a 429 + long Retry-After can't make
one call sleep+retry past the cinematic wall-clock budget from inside a
single request. The pass-level budget (``cinematic_refine_many``) and the
per-call timeout stay the only bounds.
"""
from services import llm_skills
handle = llm_skills.resolve_skill_client(_SKILL_ID)
return handle.client if handle is not None else None
def _llm_model() -> str:
from services import llm_providers, llm_skills
p = llm_skills.effective_provider(_SKILL_ID)
if p is not None:
return llm_providers.resolve_model(p)
return os.environ.get("TRANSLATE_MODEL", "gpt-4o-mini")
@@ -125,6 +236,16 @@ def _llm_timeout() -> float:
return 45.0
def _cinematic_budget() -> float:
"""Overall wall-clock cap for a whole cinematic/autofit refine pass (seconds).
Unfinished segments degrade to their literal (Fast) translation once hit, so
a slow provider can't hang the translate. Default 180s; <=0 disables."""
try:
return float(os.environ.get("OMNIVOICE_CINEMATIC_BUDGET_S", "180"))
except ValueError:
return 180.0
def _glossary_text(glossary: Iterable[dict] | None) -> str:
"""Format the project glossary as a preamble for the LLM prompts.
@@ -154,6 +275,7 @@ def _chat(client, *, system: str, user: str) -> str:
res = client.chat.completions.create(
model=_llm_model(),
timeout=_llm_timeout(),
temperature=0.2, # pinned like the Fast path — default 1.0 drifts/invents
messages=[
{"role": "system", "content": system},
{"role": "user", "content": user},
@@ -255,21 +377,28 @@ def cinematic_refine_sync(
}
final = (adapted or "").strip() or literal_text
# Refuse adaptations that drifted off the target script (e.g. local LLM
# rewrote a Devanagari line in Latin/German). Caller still gets the
# critique so the UI can show what happened, but the live text falls
# back to the literal translation rather than corrupting the dub.
if final is not literal_text and not _looks_like_target_script(final, target_lang):
logger.warning(
"cinematic adapt produced wrong-script output for %s — falling back to literal",
target_lang,
)
return {
"text": literal_text,
"literal": literal_text,
"critique": critique,
"error": f"adapt-wrong-script:{target_lang}",
}
# Refuse adaptations that diverged from the line they were rewriting:
# wrong script (e.g. a local LLM rewrote a Devanagari line in
# Latin/German), runaway length (hallucinated dialogue, refusals,
# commentary — the script check alone passes ANY text for Latin-script
# targets), or the critique echoed back as the "adaptation". Caller still
# gets the critique so the UI can show what happened, but the live text
# falls back to the literal translation rather than corrupting the dub.
if final is not literal_text:
ok, reason = refine_output_ok(literal_text, final, target_lang, critique=critique)
if not ok:
logger.warning(
"cinematic adapt diverged for %s (%s) — falling back to literal",
target_lang, reason,
)
wrong_script = (reason or "").startswith("wrong-script")
return {
"text": literal_text,
"literal": literal_text,
"critique": critique,
"error": (f"adapt-wrong-script:{target_lang}" if wrong_script
else "adapt-diverged"),
}
return {
"text": final,
"literal": literal_text,
@@ -320,4 +449,35 @@ async def cinematic_refine_many(
)
return {"id": seg_id, **res}
return await asyncio.gather(*(_one(sid, src, lit) for sid, src, lit in pairs))
# Overall wall-clock budget for the whole pass. Per-call timeout + bounded
# concurrency already cap it, but a slow/rate-limited provider on a large dub
# can still stall the "Translating…" spinner for minutes. Bound it: segments
# that finish in time keep their cinematic refine; any still-running segment
# degrades to its literal (Fast) translation so the translate ALWAYS returns
# within the budget instead of hanging. 0/negative disables the bound.
budget = _cinematic_budget()
tasks = [asyncio.ensure_future(_one(sid, src, lit)) for sid, src, lit in pairs]
if budget <= 0:
return await asyncio.gather(*tasks)
done, pending = await asyncio.wait(tasks, timeout=budget)
if pending:
logger.warning(
"Cinematic pass hit its %.0fs budget with %d/%d segment(s) unfinished "
"— falling back to the literal translation for those (slow LLM "
"provider?). Raise OMNIVOICE_CINEMATIC_BUDGET_S or pick a faster "
"provider.", budget, len(pending), len(tasks),
)
out: list[dict] = []
for task, (sid, _src, lit) in zip(tasks, pairs):
if task in done and not task.cancelled():
try:
out.append(task.result())
continue
except Exception as e: # noqa: BLE001 — never let one seg sink the pass
logger.warning("cinematic segment %s failed: %s", sid, e)
else:
task.cancel() # stop awaiting; the executor thread is abandoned (#730 pattern)
out.append({"id": sid, "text": lit, "literal": lit, "critique": "",
"error": "cinematic-budget"})
return out
+410 -12
View File
@@ -55,6 +55,59 @@ def _mask_hf_tokens(value):
return _HF_TOKEN_MASK_RE.sub(_HF_TOKEN_MASK, value)
# ── HF Hub closed-client recovery (#880) ────────────────────────────────────
#
# huggingface_hub ≥1.x shares ONE global httpx client across every download.
# If anything closes it mid-lifecycle, every later hub call — e.g. an engine's
# first-use model download inside the generate path — dies with httpx's
# "Cannot send a request, as the client has been closed". The client is
# recoverable: ``close_session()`` drops it and the next hub call builds a
# fresh one, so the correct handling is a single targeted retry, not a
# user-facing failure.
def _is_closed_client_error(e) -> bool:
"""True iff ``e`` (or anything in its __cause__/__context__ chain) is
httpx's closed-client lifecycle error. Cycle-safe."""
seen, stack = set(), [e]
while stack:
exc = stack.pop()
if exc is None or id(exc) in seen:
continue
seen.add(id(exc))
low = str(exc).lower()
if "client has been closed" in low or "cannot send a request" in low:
return True
stack.append(exc.__cause__)
stack.append(exc.__context__)
return False
def _retry_once_with_fresh_hf_client(loader, what: str):
"""Run ``loader()`` — a model constructor that may download from the HF
Hub on first use. On the specific closed-client failure above, reset the
hub's shared client and retry exactly ONCE. Any other failure (and a
repeat closed-client failure) propagates untouched, where the generation
error classifier labels it as a network problem (#880)."""
try:
return loader()
except Exception as e:
if not _is_closed_client_error(e):
raise
logger.warning(
"%s: HF Hub httpx client was closed mid-download (%s); "
"retrying once with a fresh client.", what, e,
)
try:
from huggingface_hub.utils import close_session
close_session()
except Exception: # pragma: no cover — hub too old / API renamed
logger.warning(
"%s: couldn't reset the HF Hub client; retrying anyway.", what,
)
return loader()
# ── Protocol ────────────────────────────────────────────────────────────────
@@ -89,13 +142,40 @@ class TTSBackend(ABC):
#: (e.g. "young female, warm tone, British accent") without reference audio.
supports_voice_design: bool = False
def ensure_ready(self) -> None:
"""Load model weights now (blocking), so callers can separate the
LOAD budget from the GENERATE budget (#1033/#1037 class).
Every adapter lazily loads inside ``generate()`` via a private
``_ensure_loaded()`` which meant a cold first call spent its whole
``OMNIVOICE_GENERATE_TIMEOUT_S`` window (default 300s) downloading /
loading weights and got killed with a misleading "too heavy for the
available compute" error (measured in the wild on a fresh install:
multi-GB checkpoint download, 0% GPU util, #1014). Routes call this
first under the model-load budget (``OMNIVOICE_MODEL_LOAD_TIMEOUT``,
default 1200s), then start the generate clock on an already-warm
engine. Default implementation dispatches to the adapter's own
``_ensure_loaded`` when present; engines without lazy state no-op.
Must be called on the GPU pool (it's blocking), same as generate.
"""
loader = getattr(self, "_ensure_loaded", None)
if callable(loader):
loader()
#: Whether this engine already emits mastered, studio-grade audio and should
#: therefore skip the shared apply_mastering() chain (Compressor + Reverb,
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
#: 48 kHz) set this True so their clean output isn't pumped/reverbed. Loudness
#: 48 kHz) set this True so their clean output isn't pumped. Loudness
#: normalisation is applied regardless — it's a benign peak scale.
applies_own_mastering: bool = False
#: Whether this engine can clone an arbitrary voice from reference audio
#: (`ref_audio=`), as opposed to only offering a fixed set of preset
#: voices. Default True — most engines clone. Dub/batch gate on this
#: (issue #312 class) before committing to a job that needs it, instead
#: of silently falling back to OmniVoice or mis-cloning per segment.
supports_cloning: bool = True
#: GPU/accelerator targets the engine can run on. Surfaced via the
#: Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) so users can
#: tell at a glance which engines will use their hardware. Defaults to
@@ -550,6 +630,7 @@ class KittenTTSBackend(TTSBackend):
display_name = "KittenTTS (English, 8 preset voices, CPU realtime)"
# KittenTTS ships as an ONNX CPU graph; no CUDA/MPS path today.
gpu_compat = ("cpu",)
supports_cloning = False # fixed preset voices only; ref_audio is ignored
PRESET_VOICES = [
"expr-voice-2-m", "expr-voice-2-f",
@@ -587,7 +668,13 @@ class KittenTTSBackend(TTSBackend):
"OMNIVOICE_KITTENTTS_MODEL", "KittenML/kitten-tts-mini-0.8"
)
logger.info("Loading KittenTTS from %s", checkpoint)
self._model = KittenTTS(checkpoint)
# #880: the first-use load downloads ~80 MB from the HF Hub inside the
# generate path; if the hub's shared httpx client was closed
# mid-lifecycle, retry once with a fresh client instead of failing
# the whole generation.
self._model = _retry_once_with_fresh_hf_client(
lambda: KittenTTS(checkpoint), what="KittenTTS"
)
def generate(self, text: str, **kw) -> torch.Tensor:
import numpy as np
@@ -624,6 +711,54 @@ class KittenTTSBackend(TTSBackend):
# ── MLX-Audio (mac-ARM engine multiplexer) ──────────────────────────────────
# #977: Kokoro's own ALIASES table (mlx_audio.tts.models.kokoro.pipeline) only
# recognizes ISO-ish tokens ("en", "es", "fr-fr", "pt-br", …) — it has no idea
# what a full language name is. OmniVoice's `language` kwarg is normally a
# full display name from frontend/src/languages.json (e.g. "Dutch",
# "Spanish"), forwarded verbatim by the frontend and by
# `OmniVoiceBackend.generate()`. Translate the subset Kokoro actually
# supports to the ISO token its own ALIASES expects; a caller that already
# passes an ISO code (or one of Kokoro's own single-letter codes) is
# resolved unchanged by `resolve_kokoro_lang_code()` below.
_KOKORO_ISO_BY_FULL_NAME = {
"english": "en",
"spanish": "es",
"french": "fr",
"hindi": "hi",
"italian": "it",
"portuguese": "pt",
"japanese": "ja",
"chinese": "zh",
}
def resolve_kokoro_lang_code(language: str) -> str:
"""Map a full language name / ISO code to Kokoro's single-letter
`lang_code`, against the AUTHORITATIVE table read from the installed
mlx-audio package (never a hardcoded guess the vendored table is the
only source of truth and can change across mlx-audio versions).
Raises ``ValueError`` for anything Kokoro doesn't support, naming what
it *does* support instead of forwarding a bogus code into Kokoro's
`assert lang_code in LANG_CODES`, which crashes with an unreadable
``(lang_code, LANG_CODES)`` tuple/dict repr (#977).
"""
from mlx_audio.tts.models.kokoro.pipeline import ALIASES, LANG_CODES
key = language.strip().lower()
iso = _KOKORO_ISO_BY_FULL_NAME.get(key, key)
code = ALIASES.get(iso, iso)
if code not in LANG_CODES:
supported = ", ".join(sorted(name.title() for name in _KOKORO_ISO_BY_FULL_NAME))
raise ValueError(
f"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) doesn't "
f"support language={language!r}. Kokoro supports: {supported}. "
f"Pick one of those, leave language as 'Auto', or switch to a "
f"multilingual engine (e.g. OmniVoice) for other languages."
)
return code
class MLXAudioBackend(TTSBackend):
"""Blaizzy/mlx-audio — Apple-Silicon-only wrapper over 14+ TTS engines
(Kokoro, CSM, Dia, Qwen3-TTS, Chatterbox, MeloTTS, OuteTTS, Spark,
@@ -662,7 +797,16 @@ class MLXAudioBackend(TTSBackend):
def __init__(self):
self._model = None
self._sr = 24000 # most mlx-audio engines emit 24 kHz mono
key = os.environ.get("OMNIVOICE_MLX_AUDIO_MODEL", self.DEFAULT_MODEL_KEY)
# Env var > persisted UI choice (#981 — Settings → Engines curated-
# model picker) > default. Mirrors active_backend_id()'s resolution
# order exactly so power-users can still pin a model without the UI
# silently undoing it.
from core import prefs
key = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=self.DEFAULT_MODEL_KEY,
)
# Accept either a curated key ("kokoro") or a full HF repo id
# ("mlx-community/Kokoro-82M-bf16") — flexibility for power users.
self._model_id = self.CURATED_MODELS.get(key, key)
@@ -700,6 +844,18 @@ class MLXAudioBackend(TTSBackend):
# silently ignores languages it doesn't know.
return ["multi"]
@property
def supports_cloning(self) -> bool:
"""Model-dependent — this adapter multiplexes 7+ curated models and
only some take a reference-audio speaker prompt. `generate()` passes
`ref_audio` through when present (~kwargs below) but silently retries
without it on a TypeError, so an engine picked for cloning that's
actually running Kokoro/Qwen3-TTS/etc. would clone nothing. Of the
curated set, only CSM (`mlx-community/csm-1b-8bit`) is confirmed to
accept a reference prompt default False for every other model,
curated or user-supplied, until positively confirmed."""
return self._model_id == self.CURATED_MODELS.get("csm")
def _ensure_loaded(self):
if self._model is not None:
return
@@ -713,6 +869,7 @@ class MLXAudioBackend(TTSBackend):
voice = kw.get("voice")
ref_audio = kw.get("ref_audio")
ref_text = kw.get("ref_text")
language = kw.get("language")
speed = float(kw.get("speed", 1.0))
@@ -723,7 +880,31 @@ class MLXAudioBackend(TTSBackend):
kwargs = {"text": text, "speed": speed}
if voice: kwargs["voice"] = voice
if ref_audio: kwargs["ref_audio"] = ref_audio
if language: kwargs["lang_code"] = language[:2].lower()
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
# AND ref_text are present — with ref_text missing, its context list
# stays empty and indexing into it raises an opaque
# "IndexError: list index out of range" deep inside mlx-audio,
# instead of ever attempting the clone. Community-diagnosed (#1012).
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
if language and language != "Auto":
if self._model_id == self.CURATED_MODELS.get("kokoro"):
# Kokoro's vendored pipeline hard-asserts `lang_code` against
# its own single-letter table — a bogus code crashes with an
# unreadable AssertionError instead of failing cleanly
# (#977). Resolve against the authoritative installed table
# instead of guessing via `language[:2]`.
kwargs["lang_code"] = resolve_kokoro_lang_code(language)
else:
# `lang_code`-as-2-letter-truncation is Kokoro's own
# convention, not mlx-audio's in general — other curated
# models either ignore unrecognized kwargs (CSM/Dia/OuteTTS
# accept **kwargs and drop it) or expect something else
# entirely (Qwen3-TTS's own docstring: "lang_code: Language
# code (auto, chinese, english, etc.)" — a full name, not a
# 2-letter code). Kokoro's strict validation doesn't apply to
# them, so don't reject a language that's valid for whatever
# model is actually active.
kwargs["lang_code"] = language[:2].lower()
pieces = []
try:
@@ -1052,6 +1233,7 @@ class SherpaOnnxBackend(TTSBackend):
# Sherpa-ONNX uses the onnxruntime providers — CPU is the universal
# baseline; CUDA provider is available on Linux/Windows installs.
gpu_compat = ("cuda", "cpu")
supports_cloning = False # VITS speaker-id only; no ref_audio support
def __init__(self):
self._tts = None
@@ -1061,13 +1243,34 @@ class SherpaOnnxBackend(TTSBackend):
def is_available(cls) -> tuple[bool, str]:
try:
import sherpa_onnx # noqa: F401
return True, "ready"
except ImportError as e:
return False, (
f"sherpa-onnx not installed: {e}. "
"Install with: pip install sherpa-onnx. "
"Download models from https://github.com/k2-fsa/sherpa-onnx/releases"
)
# #919: sherpa-onnx ships no bundled default model — it can only
# synthesize once OMNIVOICE_SHERPA_MODEL points at a downloaded model
# directory. Gate on it here (like the other path-configured opt-in
# engines: Confucius4/dots/MOSS) so the picker marks it unavailable-
# with-a-reason instead of letting a user select it, generate, and hit
# a config error that used to be mislabeled as out-of-memory.
model_dir = os.environ.get("OMNIVOICE_SHERPA_MODEL", "").strip()
if not model_dir:
return False, (
"OMNIVOICE_SHERPA_MODEL not set. Point it to a sherpa-onnx TTS "
"model directory (containing model.onnx + tokens.txt), then "
"restart OmniVoice. Download models from "
"https://github.com/k2-fsa/sherpa-onnx/releases"
)
if not os.path.isfile(os.path.join(model_dir, "model.onnx")):
return False, (
f"No model.onnx in OMNIVOICE_SHERPA_MODEL ({model_dir}). Point "
"it at a sherpa-onnx TTS model directory containing model.onnx "
"+ tokens.txt. Download models from "
"https://github.com/k2-fsa/sherpa-onnx/releases"
)
return True, "ready"
@property
def sample_rate(self) -> int:
@@ -1151,6 +1354,19 @@ _LAZY_REGISTRY: dict[str, tuple[str, str]] = {
# this module for TTSBackend). The class is resolved on first
# attribute access via the LazyRegistry below.
"supertonic3": ("engines.supertonic3", "Supertonic3Backend"),
# Issue #498: MOSS-TTS-v1.5 (8B) and dots.tts (2B) — both opt-in,
# subprocess-isolated with their own venv because each pins a
# transformers version that conflicts with the parent's >=5.3
# (MOSS == 5.0.0, dots.tts == 4.57.0). Same dedicated-venv pattern as
# IndexTTS2. Lazy for the same import-cycle reason as the entries above.
"moss-tts-v15": ("engines.moss_tts_v15", "MossTTSV15Backend"),
"dots-tts": ("engines.dots_tts", "DotsTTSBackend"),
# Issue #590: Confucius4-TTS (netease-youdao) — LLM-based, 14-language
# cross-lingual zero-shot cloning, Apache-2.0. Opt-in + subprocess-isolated
# (own Python 3.10 venv) like the entries above. Validated end-to-end
# 2026-07-02 (CPU, Apple Silicon; 22.05 kHz output). Gated behind
# OMNIVOICE_CONFUCIUS4_TTS_DIR so it's inert until enabled.
"confucius4-tts": ("engines.confucius4", "Confucius4Backend"),
}
@@ -1187,7 +1403,13 @@ class _LazyRegistry(dict):
# effect on every list_backends() call — we keep iteration light
# and let the caller's __getitem__ trigger the import.
seen: set[str] = set()
for k in dict.__iter__(self):
# Snapshot the live keys before yielding. A concurrent thread's lazy
# __getitem__ inserts into self (self[key] = cls), and list_backends()
# runs in a FastAPI threadpool — so holding a *live* dict iterator open
# across the per-engine is_available() probes would raise
# "dictionary changed size during iteration". list() consumes the
# iterator atomically under the GIL, closing that window.
for k in list(dict.__iter__(self)):
seen.add(k)
yield k
for k in _LAZY_REGISTRY:
@@ -1244,6 +1466,41 @@ _INSTALL_HINTS: dict[str, str] = {
"sherpa-onnx": "pip install sherpa-onnx (universal ONNX runtime, WASM-ready)",
"omnivoice-gguf":"Bundled — runs the C++ omnivoice-tts binary in bin/. Quants download lazily from Serveurperso/OmniVoice-GGUF on first generate.",
"supertonic3": "uv sync --extra supertonic (CPU-only ONNX, 31 langs, ~400 MB model on first use; OpenRAIL-M model license)",
"moss-tts-v15": "git clone OpenMOSS/MOSS-TTS + set OMNIVOICE_MOSS_TTS_V15_DIR (own venv, transformers==5.0; 8B, ~16 GB weights; CUDA/CPU, no MPS; Apache-2.0)",
"dots-tts": "git clone rednote-hilab/dots.tts + set OMNIVOICE_DOTS_TTS_DIR (own venv, transformers==4.57; 2B, ~9 GB weights; CUDA/CPU, Linux/macOS only — no Windows; Apache-2.0)",
"confucius4-tts":"git clone netease-youdao/Confucius4-TTS + set OMNIVOICE_CONFUCIUS4_TTS_DIR (own Python 3.10 venv; 14-lang cross-lingual zero-shot clone; ~5 GB weights auto-download; CUDA/CPU, no MPS; Apache-2.0)",
}
# Copy-paste-ready setup line for opt-in engines gated behind a filesystem-path
# env var (issue #498 / #590). The install_hint tells users a var exists; this
# is the *exact* `export VAR=...` line to run, so they don't have to reconstruct
# it from the docs. Surfaced verbatim in the Compat Matrix's "Why unavailable?"
# disclosure with a Copy button. Single-sourced here so it can't drift from the
# var each engine's is_available() actually reads. bash/zsh form (the dominant
# clone-and-run workflow for these engines; dots.tts is *nix-only anyway).
_SETUP_SNIPPETS: dict[str, str] = {
"indextts2": "export OMNIVOICE_INDEXTTS_DIR=/path/to/index-tts",
"moss-tts-v15": "export OMNIVOICE_MOSS_TTS_V15_DIR=/path/to/MOSS-TTS",
"dots-tts": "export OMNIVOICE_DOTS_TTS_DIR=/path/to/dots.tts",
"confucius4-tts": "export OMNIVOICE_CONFUCIUS4_TTS_DIR=/path/to/Confucius4-TTS",
# #919: sherpa-onnx gates on a downloaded model dir (model.onnx + tokens.txt).
"sherpa-onnx": "export OMNIVOICE_SHERPA_MODEL=/path/to/sherpa-onnx-model",
}
# Short, readable labels for mlx-audio's curated models (#981) — surfaced in
# the Settings → Engines model picker so users see more than a bare key.
# Single-sourced here rather than on MLXAudioBackend.CURATED_MODELS itself so
# the class dict stays a plain key → repo-id map (what __init__ needs).
_MLX_AUDIO_MODEL_LABELS: dict[str, str] = {
"kokoro": "Kokoro (default, fast)",
"csm": "CSM (voice cloning)",
"qwen3-tts": "Qwen3-TTS (voice design)",
"dia": "Dia",
"chatterbox": "Chatterbox",
"melotts": "MeloTTS (lightweight)",
"outetts": "OuteTTS",
}
@@ -1258,6 +1515,7 @@ def list_backends() -> list[dict]:
"available": bool,
"reason": Optional[str], # message when not available
"install_hint": Optional[str],
"setup_snippet": Optional[str], # exact `export VAR=...` for path-gated opt-in engines
"last_error": Optional[str], # cached most-recent failure
"isolation_mode": "in-process" | "subprocess",
"gpu_compat": list[str], # subset of {cuda, rocm, mps, xpu, cpu}
@@ -1321,12 +1579,30 @@ def list_backends() -> list[dict]:
"available": ok,
"reason": None if ok else _mask_hf_tokens(msg),
"install_hint": _INSTALL_HINTS.get(bid),
# Exact `export VAR=...` line for path-gated opt-in engines, or None.
"setup_snippet": _SETUP_SNIPPETS.get(bid),
"last_error": _LAST_ERRORS.get(bid),
"isolation_mode": isolation,
"gpu_compat": list(gpu_compat),
# effective_device / routing_status / routing_reason (scrubbed):
**routing_fields(gpu_compat, caps),
})
# #981: mlx-audio multiplexes 7+ curated models behind one backend id
# — surface the roster + the currently-active pick so Settings can
# render a model picker instead of always defaulting to Kokoro.
# mlx-audio ONLY; every other backend loads a single fixed model.
if bid == "mlx-audio":
from core import prefs
active_model = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=cls.DEFAULT_MODEL_KEY,
)
out[-1]["curated_models"] = [
{"key": key, "label": _MLX_AUDIO_MODEL_LABELS.get(key, key), "repo_id": repo_id}
for key, repo_id in cls.CURATED_MODELS.items()
]
out[-1]["active_model_id"] = active_model
return out
@@ -1336,6 +1612,29 @@ def get_backend_class(backend_id: str) -> type[TTSBackend]:
return _REGISTRY[backend_id]
def cloning_capable_engine_ids() -> list[str]:
"""Engine ids that support reference-audio voice cloning — used to build
an actionable error when the active engine can't (dub/batch gating).
Iterates the same registry ``list_backends()`` uses, via ``.items()`` so
lazy entries resolve through ``_LazyRegistry``'s snapshot-safe iteration
(see ``_LazyRegistry.__iter__``) exactly like every other registry scan
in this module.
A class-level ``getattr`` on a *property* returns the descriptor object
itself (always truthy) rather than its computed value so a
model-dependent adapter like ``MLXAudioBackend`` (only some of its 7+
curated models can clone) would always show up here regardless of which
model is actually configured. Excluded rather than falsely recommended:
``isinstance(..., bool)`` is False for a descriptor, True for a plain
class attribute.
"""
return [
bid for bid, cls in _REGISTRY.items()
if isinstance((v := getattr(cls, "supports_cloning", True)), bool) and v
]
def active_routing() -> dict | None:
"""Routing verdict for the currently-active TTS engine, or ``None`` if it
can't be determined (no engine / probe failure).
@@ -1404,15 +1703,21 @@ def active_backend_id() -> str:
# call the outgoing engine's unload() before switching.
_active_instance: "TTSBackend | None" = None
_active_instance_id: "str | None" = None
# mlx-audio multiplexes 7+ curated models behind one backend id — a model-only
# switch (same "mlx-audio" id, different curated model) must also invalidate
# the cache, or picking a different model in Settings has no effect until the
# app restarts (#981). Only meaningful when _active_instance_id == "mlx-audio".
_active_mlx_model_key: "str | None" = None
def reset_active_backend() -> None:
"""Unload + clear the cached active backend. For app shutdown and tests.
Idempotent and best-effort a raising unload() never propagates."""
global _active_instance, _active_instance_id
global _active_instance, _active_instance_id, _active_mlx_model_key
inst = _active_instance
_active_instance = None
_active_instance_id = None
_active_mlx_model_key = None
if inst is not None:
try:
inst.unload()
@@ -1430,13 +1735,32 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
``model=`` (caller already holds a loaded model), we return a fresh view
over the shared singleton rather than caching it but a switch *away from*
a different engine still triggers that engine's unload().
For mlx-audio specifically, the backend id alone doesn't capture *which*
curated model is loaded (#981) — so we also track the resolved model key
and treat a model-only change as a switch, reusing the exact same
unload-and-reconstruct path as an id switch.
"""
global _active_instance, _active_instance_id
global _active_instance, _active_instance_id, _active_mlx_model_key
bid = active_backend_id()
# Switching engines: release the outgoing one first. Best-effort so a bad
# unload() can never block the switch.
if _active_instance is not None and _active_instance_id != bid:
mlx_model_key = None
if bid == "mlx-audio":
from core import prefs
mlx_model_key = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=MLXAudioBackend.DEFAULT_MODEL_KEY,
)
# Switching engines (or, for mlx-audio, switching curated models): release
# the outgoing one first. Best-effort so a bad unload() can never block
# the switch.
switching = _active_instance is not None and (
_active_instance_id != bid
or (bid == "mlx-audio" and mlx_model_key != _active_mlx_model_key)
)
if switching:
try:
_active_instance.unload()
except Exception as exc: # noqa: BLE001
@@ -1444,6 +1768,7 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
type(_active_instance).__name__, exc)
_active_instance = None
_active_instance_id = None
_active_mlx_model_key = None
cls = get_backend_class(bid)
if cls is OmniVoiceBackend and model is not None:
@@ -1455,9 +1780,82 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
if _active_instance is None or _active_instance_id != bid:
_active_instance = OmniVoiceBackend(model=model) if cls is OmniVoiceBackend else cls()
_active_instance_id = bid
_active_mlx_model_key = mlx_model_key
return _active_instance
# ── Shared generation-time engine resolution (issue #312 class) ───────────
#
# dub_generate.py and batch.py used to call services.model_manager.get_model()
# directly, hardcoding OmniVoice regardless of the engine selected in
# Settings → Engines — a SILENT fallback: pick VoxCPM2, dub anyway with
# OmniVoice, no error. This is the single resolution path both routers now
# call instead, mirroring generation.py's /generate resolution (engine id →
# is_available() → routing gate) plus a voice-cloning capability gate that
# /generate doesn't need (OmniVoice's native path always clones).
async def resolve_generation_backend(
*, require_cloning: bool = False, cloning_purpose: str = "dubbing",
) -> TTSBackend:
"""Resolve + validate the active TTS engine for a generation call.
Returns the live backend instance (:func:`get_active_tts_backend`)
cached, and properly unload()ed on an engine switch. Raises ``ValueError``
with an actionable message (never silently falls back to OmniVoice) when:
* the configured engine id is unknown (bad env var / stale pref),
* the engine reports itself unavailable (``is_available()``),
* the engine needs an accelerator this host lacks and has no CPU path
(``routing_status == "unavailable"``),
* ``require_cloning`` is True and the resolved backend can't clone
from reference audio (``supports_cloning`` False) checked on the
live *instance*, not the class, so a model-dependent adapter like
MLX-Audio (Kokoro vs. CSM) is judged by what's actually loaded.
"""
engine_id = active_backend_id()
try:
backend_cls = get_backend_class(engine_id)
except ValueError as e:
raise ValueError(
f"Active TTS engine '{engine_id}' is not a recognized backend ({e}). "
"Check Settings → Engines or the OMNIVOICE_TTS_BACKEND env var."
) from e
try:
ok, msg = backend_cls.is_available()
except Exception as exc: # noqa: BLE001 — surface as an actionable ValueError
ok, msg = False, f"{type(exc).__name__}: {exc}"
if not ok:
raise ValueError(f"TTS engine '{engine_id}' is not available: {_mask_hf_tokens(msg)}")
from core.device_caps import detect_host_caps
from services.engine_routing import resolve_routing
routing = resolve_routing(getattr(backend_cls, "gpu_compat", ("cpu",)), detect_host_caps())
if routing["routing_status"] == "unavailable":
raise ValueError(routing["routing_reason"])
_model = None
if backend_cls is OmniVoiceBackend:
# OmniVoice needs its model pre-loaded before construction: called
# from an async context, OmniVoiceBackend._ensure_loaded() refuses to
# bootstrap its own event loop (see its docstring) — same reason
# generation.py's /generate special-cases this backend.
from services.model_manager import get_model
_model = await get_model()
backend = get_active_tts_backend(model=_model)
if require_cloning and not getattr(backend, "supports_cloning", True):
raise ValueError(
f"The active TTS engine '{engine_id}' doesn't support voice cloning, "
f"so {cloning_purpose} can't preserve speaker voices. Switch to one "
f"of: {', '.join(cloning_capable_engine_ids())} in Settings → "
"Engines, or use OmniVoice for this job."
)
return backend
# ── PEP 562 lazy attribute re-export ───────────────────────────────────────
#
# Allows ``from services.tts_backend import IndexTTS2Backend`` to keep
+77 -2
View File
@@ -25,7 +25,17 @@ sys.modules["core.config"] = _config
whisperx = pytest.importorskip("whisperx")
from services.asr_backend import WhisperXBackend # noqa: E402
from services.asr_backend import ( # noqa: E402
WhisperXBackend,
_is_compute_type_error,
)
# The exact ValueError CTranslate2 raises at model construction on a GPU
# without efficient fp16 (older Maxwell/Pascal, GTX 16xx) or a cuDNN mismatch.
_FP16_ERR = (
"Requested float16 compute type, but the target device or backend do not "
"support efficient float16 computation"
)
def test_cuda_oom_falls_back_to_cpu(monkeypatch):
@@ -52,8 +62,13 @@ def test_cuda_oom_falls_back_to_cpu(monkeypatch):
def test_non_oom_runtime_error_still_raises(monkeypatch):
msg = "some other failure"
# A generic non-OOM, non-compute-type RuntimeError must still propagate —
# the new compute_type fallback must NOT swallow it.
assert _is_compute_type_error(msg) is False
def fake_load_model(name, device, compute_type, **kw):
raise RuntimeError("some other failure") # not an OOM → must propagate
raise RuntimeError(msg) # not an OOM, not compute-type → must propagate
monkeypatch.setattr(whisperx, "load_model", fake_load_model)
@@ -62,3 +77,63 @@ def test_non_oom_runtime_error_still_raises(monkeypatch):
be._allow_vad_pickle_globals = lambda: None
with pytest.raises(RuntimeError, match="some other failure"):
be._ensure_asr()
def test_float16_unsupported_falls_back_to_int8(monkeypatch):
"""#551: a GPU without efficient fp16 raises a ValueError at load for both
float16 AND int8_float16; the backend must degrade to int8 on the SAME
device (cuda) without raising not fall to CPU and not crash."""
calls = []
def fake_load_model(name, device, compute_type, **kw):
calls.append((device, compute_type))
if device == "cuda" and compute_type in ("float16", "int8_float16"):
raise ValueError(_FP16_ERR)
return object() # cuda int8 succeeds
monkeypatch.setattr(whisperx, "load_model", fake_load_model)
be = WhisperXBackend()
be._device, be._compute_type = "cuda", "float16"
be._allow_vad_pickle_globals = lambda: None
be._ensure_asr()
assert be._asr is not None # recovered, no raise
assert be._device == "cuda" and be._compute_type == "int8" # same device, int8
assert calls == [("cuda", "float16"), ("cuda", "int8_float16"), ("cuda", "int8")]
def test_faster_whisper_float16_unsupported_falls_back_to_int8(monkeypatch):
"""Mirror for FasterWhisperBackend: float16 + int8_float16 raise the fp16
ValueError, int8 succeeds loads on (cuda, int8) without raising."""
import services.asr_backend as asr_backend
from services.asr_backend import FasterWhisperBackend
calls = []
class FakeWhisperModel:
def __init__(self, name, device, compute_type, **kw):
calls.append((device, compute_type))
if device == "cuda" and compute_type in ("float16", "int8_float16"):
raise ValueError(_FP16_ERR)
# cuda int8 succeeds
fake_fw = types.ModuleType("faster_whisper")
fake_fw.WhisperModel = FakeWhisperModel
monkeypatch.setitem(sys.modules, "faster_whisper", fake_fw)
# Force the CUDA starting point regardless of the CI host's hardware by
# making torch.cuda.is_available() return True inside _ensure_model.
fake_torch = types.ModuleType("torch")
fake_torch.cuda = types.SimpleNamespace(
is_available=lambda: True, empty_cache=lambda: None
)
monkeypatch.setitem(sys.modules, "torch", fake_torch)
be = FasterWhisperBackend()
be._ensure_model()
assert be._model is not None # recovered, no raise
assert be._device == "cuda" and be._compute_type == "int8"
assert calls == [("cuda", "float16"), ("cuda", "int8_float16"), ("cuda", "int8")]
@@ -0,0 +1,84 @@
"""speechbrain LazyModule cross-platform guard (#630/#611/#647).
speechbrain 1.x suppresses optional-integration imports (k2_fsa, numba, ) that
are triggered merely by introspection from the stdlib `inspect` module. Its
guard checked `filename.endswith("/inspect.py")` a hardcoded POSIX separator
so on Windows (backslash paths) the guard MISSED and a stray access to the
`speechbrain.k2_integration` redirect actually imported the (absent) k2 package,
raising `ImportError: Lazy import of LazyModule(...k2_fsa...) failed` that aborted
WhisperX transcription with zero segments.
`_harden_speechbrain_lazy_imports()` re-implements `ensure_module` with an
`os.path.basename` check so the guard fires on every platform. These tests fake
the importer frame (both Windows- and POSIX-style `inspect.py` paths, plus a
real-caller path) so they pin the behaviour regardless of the host OS.
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
importutils = pytest.importorskip(
"speechbrain.utils.importutils",
reason="speechbrain not installed in this environment",
)
from services.asr_backend import _harden_speechbrain_lazy_imports # noqa: E402
class _FakeFrameInfo:
def __init__(self, filename):
self.filename = filename
def _bogus_lazy_module():
# A LazyModule whose target can never import — so we can observe whether the
# inspect.py guard fired (AttributeError) or the import was attempted (ImportError).
return importutils.LazyModule(
"omnivoice_nonexistent_zzz",
"omnivoice_nonexistent_zzz_target",
None,
)
@pytest.mark.parametrize(
"inspect_path",
[
r"C:\Python311\Lib\inspect.py", # Windows — the case the old guard missed
"/usr/lib/python3.11/inspect.py", # POSIX — already worked, must keep working
],
)
def test_guard_fires_for_inspect_frame_on_any_separator(monkeypatch, inspect_path):
_harden_speechbrain_lazy_imports()
lm = _bogus_lazy_module()
monkeypatch.setattr(
importutils.inspect, "getframeinfo",
lambda *_a, **_k: _FakeFrameInfo(inspect_path),
)
# Guard must treat an inspect.py-triggered access as "attribute absent"
# (AttributeError) rather than attempting the doomed import (ImportError).
with pytest.raises(AttributeError):
lm.ensure_module(0)
def test_real_caller_still_surfaces_import_error(monkeypatch):
"""A genuine access from real user code (not inspect.py) with the target
missing must still raise ImportError we only suppress inspect-triggered
spurious imports, never legitimate failures."""
_harden_speechbrain_lazy_imports()
lm = _bogus_lazy_module()
monkeypatch.setattr(
importutils.inspect, "getframeinfo",
lambda *_a, **_k: _FakeFrameInfo(r"C:\Users\me\app\real_caller.py"),
)
with pytest.raises(ImportError):
lm.ensure_module(0)
def test_patch_is_idempotent():
_harden_speechbrain_lazy_imports()
first = importutils.LazyModule.ensure_module
_harden_speechbrain_lazy_imports()
assert importutils.LazyModule.ensure_module is first
assert getattr(importutils.LazyModule, "_omnivoice_xplat_guard", False) is True
@@ -0,0 +1,238 @@
"""Whole-file ASR transcribe must be wall-clock bounded (TamKieu / Vietnam report).
The chunked dub pipeline already bounds each chunk, but the whole-file paths
(dub QC re-transcribe, dictation, OpenAI-compat) ran unbounded a slow/stuck
transcribe (e.g. large-v3 on a VRAM-starved GPU) hung the request *and* held a
GPU-pool worker, surfacing in the UI as the misleading "can't reach the local
backend". `run_transcribe_guarded` bounds them and raises `ASRTimeoutError` with
actionable guidance. These tests pin the timeout path, the pass-through path, and
that the error message tells the user what to do.
"""
import asyncio
import os
import sys
import time
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from services import asr_backend # noqa: E402
from services.asr_backend import ( # noqa: E402
ASRTimeoutError,
ASR_TRANSCRIBE_TIMEOUT_S,
reset_pool_after_wedge,
run_transcribe_guarded,
)
from concurrent.futures import ThreadPoolExecutor # noqa: E402
@pytest.fixture(autouse=True)
def _fresh_timeout_streak(monkeypatch):
"""The consecutive-timeout streak (#730 residual B) is process-global
session state; zero it per test so ordering can't leak recommendations,
and pin the active engine so a dev box's prefs can't flip the hint."""
monkeypatch.setattr(asr_backend, "_timeout_streak", 0)
monkeypatch.setattr(asr_backend, "active_backend_id", lambda: "whisperx")
def test_default_timeout_is_env_overridable(monkeypatch):
# The constant is read at import; just assert it's a sane positive default.
assert ASR_TRANSCRIBE_TIMEOUT_S > 0
def test_slow_transcribe_raises_actionable_timeout():
pool = ThreadPoolExecutor(max_workers=1)
def _hang():
time.sleep(5) # would block far past our tiny timeout
return "never"
async def _go():
with pytest.raises(ASRTimeoutError) as ei:
await run_transcribe_guarded(pool, _hang, what="QC", timeout=0.2)
msg = str(ei.value)
# Message must reassure (backend alive) + give concrete remedies.
assert "backend is running" in msg
assert "Settings → Models" in msg
assert "CPU" in msg
asyncio.run(_go())
pool.shutdown(wait=False)
def test_fast_transcribe_passes_through():
pool = ThreadPoolExecutor(max_workers=1)
def _quick():
return {"segments": [{"text": "hi"}]}, "whisperx"
async def _go():
out = await run_transcribe_guarded(pool, _quick, what="Dictation", timeout=5.0)
assert out == ({"segments": [{"text": "hi"}]}, "whisperx")
asyncio.run(_go())
pool.shutdown(wait=True)
def test_timeout_error_is_a_timeouterror_subclass():
# Routers that catch broad TimeoutError (openai_compat) must also catch ours.
assert issubclass(ASRTimeoutError, TimeoutError)
def test_timeout_resets_a_resilient_pool_to_restore_capacity():
# #730: a wedged transcribe holds its GPU-pool worker forever; with a 1-2
# worker pool that starves TTS generate and surfaces as "can't reach
# backend". On timeout, run_transcribe_guarded must reset() a pool that
# supports it (the real _ResilientGpuPool) so the next submit gets a fresh
# worker — capacity restored without an app restart.
class _FakePool(ThreadPoolExecutor):
def __init__(self):
super().__init__(max_workers=1)
self.reset_calls = 0
def reset(self):
self.reset_calls += 1
pool = _FakePool()
def _hang():
time.sleep(5)
return "never"
async def _go():
with pytest.raises(ASRTimeoutError):
await run_transcribe_guarded(pool, _hang, what="Dub", timeout=0.2)
asyncio.run(_go())
assert pool.reset_calls == 1
pool.shutdown(wait=False)
def test_timeout_without_reset_capable_pool_does_not_crash():
# A plain ThreadPoolExecutor (no reset) must still bound + raise cleanly —
# the reset() is best-effort, never required.
pool = ThreadPoolExecutor(max_workers=1)
def _hang():
time.sleep(5)
return "never"
async def _go():
with pytest.raises(ASRTimeoutError):
await run_transcribe_guarded(pool, _hang, what="QC", timeout=0.2)
asyncio.run(_go())
pool.shutdown(wait=False)
# ── Residual B on #730: consecutive timeouts recommend the isolated engine ──
def _hang_forever():
time.sleep(5)
return "never"
async def _timeout_once(pool, timeout=0.1) -> str:
with pytest.raises(ASRTimeoutError) as ei:
await run_transcribe_guarded(pool, _hang_forever, what="Dub", timeout=timeout)
return str(ei.value)
def test_second_consecutive_timeout_recommends_isolated_engine():
"""When guarded timeouts hit twice in a row in one session, pool resets
clearly aren't recovering the hang — the error the user sees must name the
crash-isolated escape-hatch engine (and make clear we never auto-switch)."""
pool = ThreadPoolExecutor(max_workers=2)
async def _go():
first = await _timeout_once(pool)
assert "faster-whisper-isolated" not in first # one timeout ≠ a pattern
second = await _timeout_once(pool)
assert "faster-whisper-isolated" in second
assert "Settings → Engines" in second
assert "never switches engines automatically" in second
asyncio.run(_go())
pool.shutdown(wait=False)
def test_successful_transcribe_resets_the_timeout_streak():
"""'Consecutive' must mean consecutive: a transcribe that completes between
two timeouts proves the pool recovered, so the recommendation must not fire."""
pool = ThreadPoolExecutor(max_workers=3)
async def _go():
await _timeout_once(pool)
out = await run_transcribe_guarded(pool, lambda: "ok", what="Dub", timeout=5.0)
assert out == "ok"
second = await _timeout_once(pool)
assert "faster-whisper-isolated" not in second
asyncio.run(_go())
pool.shutdown(wait=False)
def test_no_recommendation_when_already_on_isolated_engine(monkeypatch):
"""Recommending the isolated engine to a user already running it is noise —
the base message's smaller-model/CPU guidance is all that's left."""
monkeypatch.setattr(
asr_backend, "active_backend_id", lambda: "faster-whisper-isolated"
)
pool = ThreadPoolExecutor(max_workers=2)
async def _go():
await _timeout_once(pool)
second = await _timeout_once(pool)
assert "faster-whisper-isolated) in Settings" not in second
assert "never switches engines automatically" not in second
asyncio.run(_go())
pool.shutdown(wait=False)
def test_timeout_env_name_is_parameterized():
"""The chunked dub path passes its own knob; the message must name IT, not
the whole-file env var (actionable errors point at the right dial)."""
pool = ThreadPoolExecutor(max_workers=1)
async def _go():
with pytest.raises(ASRTimeoutError) as ei:
await run_transcribe_guarded(
pool, _hang_forever, what="Dub chunk 1/3", timeout=0.1,
timeout_env="OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S",
)
msg = str(ei.value)
assert "OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S" in msg
assert "OMNIVOICE_ASR_TRANSCRIBE_TIMEOUT_S" not in msg
asyncio.run(_go())
pool.shutdown(wait=False)
def test_reset_pool_after_wedge_is_shared_and_best_effort():
"""One reset mechanism for every transcribe path (#730 residual A): it
resets a reset-capable pool, no-ops a plain executor, and never raises."""
class _Pool:
resets = 0
def reset(self):
self.resets += 1
p = _Pool()
assert reset_pool_after_wedge(p, what="Dub chunk 1/2") is True
assert p.resets == 1
plain = ThreadPoolExecutor(max_workers=1)
try:
assert reset_pool_after_wedge(plain) is False
finally:
plain.shutdown(wait=False)
class _Broken:
def reset(self):
raise RuntimeError("reset blew up")
assert reset_pool_after_wedge(_Broken()) is False # must not raise
Regular → Executable
View File
+792 -18
View File
File diff suppressed because it is too large Load Diff
+352
View File
@@ -0,0 +1,352 @@
# Migration — Per-Component `.css` → Tailwind v4 Utilities
**Status:** Plan (not yet executed) · **Drafted:** 2026-06-30 · **Type:** Incremental styling migration, no intended visual change
**Owner stance:** leans toward a full migration but values not breaking the UI · **This plan's recommendation:** *bounded* migration (utilities for layout/spacing/typography everywhere; keep CSS for the hard stuff). See §8.
## Why
`frontend/src` carries **74 `.css` files / 16,615 lines** of global, BEM-ish CSS
(`dub-col`, `models-row__role`, `readiness-checklist__title`, …). Tailwind v4 is
**already wired** — `src/index.css` imports `tailwindcss/theme.css` +
`tailwindcss/utilities.css`, has an `@theme` block, and `vite.config.js` runs
`@tailwindcss/vite`. So the runtime cost of utilities is already paid; we are just
not using them. Editing a layout today means hunting a class across a 989-line
file and a JSX `className`. Utilities put the layout where it's read — in the JSX —
and shrink the per-component CSS to only what utilities can't express.
This is **not** a redesign. Every step must render pixel-identical. The honest
blocker is that **there are zero visual-regression tests** — the prior page
refactors (see `docs/maintenance-pages-modularization.md`) verified "no change"
by diffing `className` strings, and *that trick is useless here because the whole
point is that class names change*. Closing that gap is the first real task (§4),
not an afterthought.
## Current state (measured 2026-06-30)
| Metric | Value |
|--------|------:|
| `.css` files | 74 |
| Total CSS lines | 16,615 |
| `var(--…)` token references across CSS | ~3,200 |
| Files using `display:flex` | 64 |
| Files using `display:grid` / `grid-template` | 27 |
| Files using `transition:` | 46 |
| Files using `box-shadow` | 43 |
| Files using `@media` | 25 |
| Files using `linear/radial-gradient` | 22 |
| Files using `@keyframes` (73 blocks total) | 30 |
| Files using `animation:` | 34 |
| Files using `backdrop-filter`/glass blur | 11 |
| Files using `::before`/`::after` | 11 |
| Files using `:has()` | 3 |
| Files using `!important` | 14 |
Biggest files (conversion ROI ranked by layout density, not raw size):
`index.css` 2532 · `FirstRunSetup.css` 1020 · `DubTab.css` 989 ·
`VoiceGallery.css` 541 · `StoriesEditor.css` 525 · `LogsFooter.css` 507 ·
`Settings.css` 469 · `CloneDesignTab.css` 458 · `settings/primitives/primitives.css` 368.
The token system (do **not** redesign it):
- `src/ui/tokens.css` (157 lines, ~82 custom props): the declared "single source of
truth" — colors, a 4px spacing scale (`--space-0..9`), radius, fonts, type scale,
weights, shadows, motion, z-index, focus ring, glass blur. Imported via `src/ui/index.js`.
- `src/ui/themes.css` (188 lines): per-theme overrides of the semantic color tokens,
keyed on `[data-theme="midnight|nord|solarized|…"]` on `<html>`. Default (no attribute)
= Gruvbox Dark.
- `src/index.css` `@theme { … }`: maps a subset of tokens into Tailwind's theme
namespace (`--color-*`, `--radius-*`, `--font-*`) so utilities like `bg-bg`,
`text-fg`, `rounded-lg`, `font-mono` exist. **It hardcodes hex literals that
duplicate `tokens.css`** — the known drift bug (see §2).
Load order today: `index.css` (`@theme``theme` layer, lowest priority) is
imported in `App.jsx`; `tokens.css` + `themes.css` are **unlayered** `:root` /
`[data-theme]` rules imported via `ui/index.js`. Because unlayered CSS outranks
`@layer theme`, **`tokens.css` already wins for the default values and theming
already works** — the `@theme` hex literals are effectively a *losing duplicate*
that exists only so Tailwind knows the utility names. That is precisely why they
drift silently: nothing at runtime reads them, so a stale value never shows up.
## Strategy (the shape of the whole thing)
1. **Incremental, component-by-component — never big-bang.** One component (or one
small cluster) per PR. Each PR is independently shippable and CI-green. A
half-migrated component is fine; a half-migrated *codebase* is the steady state
for months and that's acceptable.
2. **Utilities-first for the mechanical 80%:** flexbox, grid, gap, padding/margin,
width/height, `text-*`/`font-*`, `rounded-*`, `border`, simple `bg-*`/`text-*`
color, `hidden`, `truncate`, basic `hover:`/`focus:` color states. These map 1:1
to utilities and are where the line-count win lives.
3. **Keep `.css` for the hard 20%:** glassmorphism (layered gradients +
`backdrop-filter`), `::before`/`::after`, `@keyframes`, `:has()` and other complex
combinators, `[data-theme]`-specific rules, and anything with `!important`
fighting specificity. Utilities don't express these cleanly and forcing them
(arbitrary-value soup, `[&::before]:…`) trades readable CSS for unreadable JSX.
4. **One source of truth via the token bridge (§2):** utilities reference the same
CSS vars the remaining `.css` reads, so a value lives in exactly one place and
`data-theme` switching keeps working for both.
5. **No file is "done" until it's deleted or demonstrably minimal.** Success is
measured in CSS LOC removed and `.css` files deleted, not files "touched."
## 2. Token-bridge prerequisite (P0 — gates everything)
The migration is only safe if a utility and the leftover CSS in the same component
resolve a token to the *same* value, including after a theme switch. Today the
`@theme` literals duplicate `tokens.css`; once components start mixing `bg-bg`
(utility) with `background: var(--color-bg)` (CSS), any drift becomes a visible,
theme-dependent bug. Fix the source-of-truth **before** converting anything.
**Recommended fix — Solution A (lowest churn, no rename):** Make `@theme` the
single declared home for the **already-overlapping** groups only — colors, radius,
fonts — and **delete those default declarations from `tokens.css`** (leave a
one-line pointer comment). Everything else in `tokens.css` (spacing, type scale,
weights, shadows, motion, z-index, focus ring, glass blur) stays put.
Why this is correct and safe:
- Tailwind needs the keys present in `@theme` to generate the utility names
(`--color-fg``text-fg`/`bg-fg`; `--radius-lg``rounded-lg`; `--font-mono`
`font-mono`). Keeping the keys there is non-negotiable.
- `@theme` emits `:root { --color-fg: … }` into the low-priority `theme` layer.
`themes.css` `[data-theme]` rules are unlayered and still outrank it, so
**theme switching is unchanged** — verify with a quick manual cycle through all
themes after the edit.
- Removing the duplicate `:root` color/radius/font lines from `tokens.css` leaves
exactly one literal per value. All ~3,200 existing `var(--…)` references keep
resolving (the var still exists on `:root`, now sourced from `@theme`).
**Guard against recurrence (required, per the "fix the class" rule):** add
`frontend/src/__tests__/theme-token-parity.test.js` (vitest, no browser) that
parses `index.css` `@theme` + `tokens.css` + `themes.css` and asserts:
(a) no token key is declared with a literal in **both** `@theme` and `tokens.css`
(catches re-introduced duplication), and (b) every `@theme` color key is overridden
by every `[data-theme]` block in `themes.css` (catches a theme that forgot a color).
This test is the thing that makes the de-dup *stay* de-duped.
**Rejected alternative — Solution B (purist):** rename source tokens to a private
namespace (`--ov-color-fg`) and bridge with `@theme inline { --color-fg:
var(--ov-color-fg) }`. This honors "`tokens.css` is the source" literally and is
the textbook Tailwind pattern, **but** it forces renaming all ~3,200 `var(--color-*)`
references across 74 files in one shot — a massive, high-risk diff that violates
"low-risk, incremental." Not worth it. (`@theme inline` referencing the *same* name
is circular and is not an option.)
**Optionally, later:** add `--spacing` to `@theme` so `p-*`/`gap-*`/`m-*` map onto
the existing 4px scale (`--space-1 = 2px``--space-9 = 44px`). Tailwind's default
spacing is a 0.25rem multiplier; OmniVoice's scale is custom, so without this,
`gap-3``var(--space-3)`. Two choices, decide in P0:
- **Map to the scale:** set `--spacing: 2px` won't reproduce the non-linear steps;
instead define explicit `--spacing-1..9` in `@theme` mirroring `--space-1..9`,
and use `gap-2`/`p-5` etc. Cleanest for readers, but utility numbers won't match
Tailwind defaults — document it.
- **Use arbitrary values bridged to the var:** `gap-[var(--space-3)]`,
`p-[var(--space-5)]`. Zero ambiguity, slightly noisier JSX, guarantees identical
pixels. **Recommended for P1P2** (safest for "no visual change"); revisit named
spacing once confidence is high.
## 3. What converts cleanly vs. what stays CSS
**Converts cleanly → utilities** (concrete, from real files):
- `ReadinessChecklist.css` `.readiness-checklist { display:flex; flex-direction:column;
gap:var(--space-3); padding:var(--space-5); border:1px solid var(--color-border);
border-radius:var(--radius-lg); font-size:var(--text-sm); }`
→ `className="flex flex-col gap-[var(--space-3)] p-[var(--space-5)] border
border-border rounded-lg text-sm"` (or mapped `text-sm` if the type scale is
bridged). The `backdrop-filter` line on the same selector **stays in CSS** (see below).
- `.readiness-checklist__title { font-weight:var(--weight-semibold);
color:var(--color-fg); display:flex; align-items:center; gap:var(--space-3); }`
`font-semibold text-fg flex items-center gap-[var(--space-3)]`.
- Generic layout rows/cols (`dub-col`, `models-row`) — flex/grid/gap/padding → utilities.
**Stays in `.css`** (criteria + real examples):
- **Glassmorphism / layered backgrounds.** `Panel.css` `.ui-panel--glass` stacks two
`radial-gradient`s + a `linear-gradient` + `backdrop-filter: var(--glass-blur-md)`.
Leave entirely in CSS. (11 files use glass blur.)
- **Pseudo-elements.** `Panel.css` `.ui-panel--glass::before` (top hairline gradient);
`DubTab.css` `.dub-stepper__step::before` (connector line). 11 files. Stay.
- **Keyframes + animations.** 73 `@keyframes` blocks across 30 files
(`@keyframes mesh/spin/pulse/shimmer` in `index.css`; `dub-pulse`,
`dub-stepper-spin`, `dub-skel-shimmer` in `DubTab.css`). Keep the `@keyframes` and
the `animation:` shorthand in CSS; a `className="animate-…"` only helps if you
register the animation in `@theme`, which isn't worth it for one-off effects.
- **`:has()` and complex combinators** (3 files), **`[data-theme]`-specific rules**
(all of `themes.css` + scattered overrides), **`!important` blocks** (14 files,
e.g. `DubTab.css` `.dub-footer-panel::before { display:none !important; }`).
- **Media queries** (25 files): convertible to `sm:`/`md:`/`lg:` **only** if the
breakpoints match Tailwind's; OmniVoice's are custom, so leave responsive blocks in
CSS unless a component's breakpoints are first added to `@theme`. Low priority.
Rule of thumb for a reviewer: *if a declaration reads a single token and sets one
box/text/flex property, it's a utility; if it composes multiple values, targets a
pseudo-element/state combinator, or animates, it stays.*
## 4. Risk mitigation — the no-visual-test gap (the gating risk)
This is the make-or-break item. Be honest: **without a visual baseline, "no change"
is unverifiable**, and `className`-diffing (what the page refactors relied on) cannot
work when class names are the thing changing. Two layers, do both:
**(a) Establish a screenshot baseline before touching components (part of P0).**
Add Playwright component/page screenshots for the surfaces being migrated. The repo
already references Playwright tooling in its docs stack; wire a minimal
`tests/visual/` that boots the Vite app (or Storybook-less direct route renders) and
captures per-component PNGs at a fixed viewport for **the default theme + one dark +
one light theme** (catches token-bridge regressions specifically). Commit baselines.
Each migration PR runs `playwright test --update-snapshots=none` and **fails on any
pixel diff above a tiny threshold**. This converts "did it change?" from a human
guess into a CI gate. Capture baselines *first*, on `main`, so they reflect
pre-migration truth.
- Scope realistically: snapshotting all 74 surfaces up front is its own project.
Snapshot **per phase, just-in-time** — before P1 leaf work, baseline the leaf
components; before P3, baseline the big pages. Baselines for a component land in
the same PR that prepares to migrate it (separate from the conversion PR so the
baseline diff is reviewable on its own).
**(b) A per-component manual checklist** (belt-and-suspenders, and the fallback for
surfaces that are hard to screenshot deterministically — anything with animation,
canvas/waveform, or live backend data):
1. Default theme: side-by-side before/after at the same viewport.
2. Cycle every `[data-theme]` — confirm colors still swap (token-bridge check).
3. Hover/focus/active/disabled states on interactive elements.
4. The component's `@keyframes`/animation still runs.
5. `prefers-reduced-motion` path unaffected (e.g. `#root` launch animation).
6. No console warnings; `bun run build` + `bun run lint` clean.
If neither (a) nor (b) is in place for a surface, **do not migrate it** — defer it to
the "leave as CSS" bucket rather than fly blind.
## 5. Phasing
Each phase = one or more independently shippable, CI-green PRs. Ordered
leaf-inward so blast radius grows only as confidence does.
### P0 — Token bridge + tooling + visual baseline (no component conversions)
- De-dup `@theme``tokens.css` (§2 Solution A) + the parity test.
- Decide + document the spacing approach (arbitrary-value bridge recommended).
- Add `prettier-plugin-tailwindcss` (or confirm oxlint/oxfmt class-sort) and wire
class sorting (§6).
- Update `CONTRIBUTING.md` (§6 — currently says *"Vanilla CSS … no Tailwind"*, which
now contradicts reality and **must** change in this same PR per the docs-sync rule).
- Stand up `tests/visual/` Playwright harness (no per-component baselines yet — just
the runner + theme matrix).
- **Effort:** ~12 days. **Success:** parity test green; theme switch verified across
all themes; CI gains a class-sort check; zero pixels changed (this PR ships no
component edits).
### P1 — Leaf / presentational components (lowest risk)
Targets: small `ui/` primitives and stateless components where CSS is mostly
flex/grid/spacing/type — e.g. `Badge`, `UpdateStatusChip`, `NetworkToggle`,
`ReadinessChecklist`, `ReadinessChecklist`, `DemoPresetGrid`, `KeyboardCheatsheet`,
`MultiLangPicker`. Skip glass-heavy ones for now.
- Per component: baseline screenshot PR → conversion PR. Convert layout/spacing/type
to utilities; keep any glass/`::before`/animation lines in a now-tiny `.css`; delete
the `.css` entirely if nothing remains and remove its import.
- **Effort:** ~35 days across ~1015 components. **Success:** ~10 `.css` files deleted
or reduced >70%; visual diffs clean; a repeatable per-component recipe proven.
### P2 — Panels & mid-size components
Targets: `settings/*Panel.css`, `Sidebar`, `NotificationPanel`, `CastingView`,
`ExportModal`, `EngineCompatibilityMatrix`, `donate/Postcard`, etc. More state,
some glass — convert the layout skeleton, leave glass/pseudo/animation.
- **Effort:** ~11.5 weeks. **Success:** settings panels are thin utility JSX + a
shared `primitives.css` for the glass/control look; CSS LOC down materially.
### P3 — Big pages
Targets in ROI order: `DubTab` (989), `VoiceGallery` (541), `StoriesEditor` (525),
`LogsFooter` (507), `Settings` (469), `CloneDesignTab` (458), `FirstRunSetup` (1020).
These pair naturally with the already-planned page modularization
(`docs/maintenance-pages-modularization.md`) — **sequence the modularization first**,
then migrate the smaller extracted components (P3 becomes "P1 again" on the pieces).
Convert layout/spacing; the pipeline steppers, overlays, gradients, and keyframes
stay as CSS.
- **Effort:** ~23 weeks. **Success:** each page's `.css` drops to the
glass/animation/pseudo residue; biggest single LOC reductions land here.
### P4 — Retire `index.css` globals last
`index.css` (2532 lines) is foundation: `@theme`, `@keyframes`, `::selection`, root
rendering, base resets, and shared global classes. Convert only the **global utility
classes** that components reuse into real utilities or component-scoped CSS; **keep**
the `@theme`, keyframes, resets, and `::selection`. Do this last because everything
depends on it.
- **Effort:** ~1 week. **Success:** `index.css` shrinks to foundation only; no
orphaned global classes.
## 6. Tooling
- **Class sorting / formatting.** The repo lints with **oxlint** (`bun run lint`,
gate) and an advisory ESLint for hooks. For Tailwind class ordering, add
**`prettier-plugin-tailwindcss`** (canonical, understands `@theme`) wired to run on
`*.jsx`, *or* adopt oxfmt's Tailwind class-sorting if the team prefers a single
formatter. Either way the goal is deterministic class order so diffs stay readable
and merge-clean.
- **Regression prevention.** Add an oxlint/convention guard so new components don't
reintroduce sprawling CSS: a soft rule (warn-only first, per "keep main green") that
flags new `.css` files over a small line budget for components that should be
utility-first, and the §2 parity test as a hard gate on token drift.
- **CONTRIBUTING update (required).** `CONTRIBUTING.md` currently states *"CSS:
Vanilla CSS in component-level files — no Tailwind."* That is now false. Replace it
with the utilities-first standard: *layout/spacing/typography/simple color via
Tailwind utilities; component `.css` only for glass, pseudo-elements, keyframes,
`:has()`, `[data-theme]` rules, and `!important` overrides; tokens live in
`tokens.css`/`@theme`, never hardcoded.* Per the docs-sync hard rule this lands in
the **same PR** as P0.
- **No new build infra**`@tailwindcss/vite` already does everything; no PostCSS
config, no Tailwind config file (v4 is CSS-first via `@theme`).
## 7. Non-goals / when to stop
- **No 100% conversion target.** ~20% of the CSS (the 11 glass files, 30 keyframe
files, 11 pseudo-element files, 3 `:has()` files, 14 `!important` files, custom-
breakpoint media queries) is **genuinely better as CSS** and should stay. Forcing it
into arbitrary-value utilities makes JSX unreadable for zero benefit.
- **No token-system redesign.** `tokens.css`/`themes.css` and the `data-theme` model
stay as-is (only the §2 de-dup).
- **No visual redesign.** Pixel-identical is the contract; restyling is a separate task.
- **No `.jsx` → `.tsx`**, no engine/backend/Tauri/Python surface, no version bump,
no dependency change beyond the dev-only formatter plugin + Playwright (frontend-only).
- **Stop conditions for an individual file:** if after pulling out layout/spacing the
remaining CSS is all glass/animation/pseudo, it's *done* — don't chase the last 10%.
- **Hands off** `BootstrapSplash.css`, `WaveformPlayer.css`/`SegmentTrack.css`
(canvas-adjacent), and other animation/`::before`-dominated files unless a clear
layout win exists.
## 8. Effort + recommendation
**Total rough effort:** ~57 focused weeks for P0P4 at the *bounded* scope below,
spread across many small PRs (it parallelizes and pauses cleanly — it never has to be
one big push).
**Recommendation — bounded migration, not 100%.** The owner leans full-migration and
prizes not breaking things; those two goals partly conflict, and the honest call is:
- **Do** convert layout/spacing/typography/simple color **everywhere** — that's the
real maintainability win, it's where ~80% of the 16.6k lines live, and it's the
low-risk part.
- **Keep ~1525% as CSS** (glass, keyframes, pseudo-elements, `:has()`,
`[data-theme]`, `!important`, custom-breakpoint media). Converting these buys
unreadable JSX and *raises* visual-regression risk on exactly the components where
diffs are hardest to verify.
- **Gate on the visual baseline (§4).** This is the single most important decision: if
the Playwright screenshot harness doesn't ship in P0, do **not** start P1 — without
it the "won't break the UI" requirement is unmet by construction. The token-bridge
de-dup (§2) is the other hard prerequisite; both are cheap and both are P0.
A realistic end state: ~60 `.css` files deleted or reduced >70%, perhaps ~1012k of
the 16.6k CSS lines removed, the rest a deliberate, documented residue of effects
utilities can't express. That delivers nearly all the maintainability benefit of a
"full" migration at a fraction of the regression risk.
## Constraints honored
- **Keep main green** — every phase is an independently CI-green PR; lint/format and
parity-test guards are warn-first where they'd otherwise churn.
- **Docs-sync** — the `CONTRIBUTING.md` rewrite lands in the same PR as P0.
- **No versioning/Docker/Tauri/Python impact** — frontend-only; dev-dependency-only
tooling additions; no `package.json` *version* bump (a devDependency add still
requires regenerating root `bun.lock` and confirming `bun install --frozen-lockfile`
per the Docker-green rule).
- **Local-first / cross-platform parity** — pure styling; no behavior, no platform
divergence.
+19 -5
View File
@@ -14,6 +14,15 @@ download UI couldn't show real bytes/speed. Until a proper Xet progress hook
lands, the app forces the **classic LFS path**, which streams through the
standard progress reporter and gives accurate downloaded/remaining/speed.
To keep that path **fast** despite Xet being off, the app runs a built-in
**multi-connection (segmented) downloader on by default** — it fetches each file
over parallel byte-ranges (IDM/uGet style), so the legacy-LFS path is no longer
single-stream. It reports real live speed/ETA and **falls back to the normal
download on any error**, so it can never compromise a correct install. Adding a
free Hugging Face token (first-run setup, or Settings → Credentials) makes this
faster still — authenticated downloads get higher rate limits and fewer stalls.
To force the old single-stream path, set `OMNIVOICE_SEGMENTED_DOWNLOAD=0`.
State is reported at **Settings → About** / `GET /system/info`:
- `fast_download.xet_installed``hf_xet` present (true)
@@ -54,12 +63,13 @@ When a download starts you'll see, in order:
## Advanced / opt-in tuning
All of these default **off** and apply to every platform identically. Set them
as environment variables (or via **Settings → API keys / environment**).
These apply to every platform identically. Set them as environment variables (or
via **Settings → API keys / environment**). The segmented accelerator is **on by
default** (set its var to `0` to disable); the rest default **off**.
| Setting | Env var | Effect |
|---|---|---|
| Segmented accelerator | `OMNIVOICE_SEGMENTED_DOWNLOAD=1` | Multi-connection downloader (parallel byte-ranges) for the legacy-LFS path — restores parallel speed **and** shows live byte speed/ETA. Falls back to the normal download on any error; files land in the standard cache. Best paired with Xet disabled (the default). |
| Segmented accelerator | `OMNIVOICE_SEGMENTED_DOWNLOAD=0` | **On by default** (see above). Set to `0` to force the old single-stream legacy-LFS download instead of the parallel byte-range one. |
| Max parallel files | `OMNIVOICE_DOWNLOAD_MAX_WORKERS` (default 8) | Files fetched at once. Xet already parallelises *within* a file, so raising this rarely helps and uses more memory. |
| High-performance mode | `HF_XET_HIGH_PERFORMANCE=1` | Maximum throughput. Needs lots of RAM and bandwidth — can **hurt** low-RAM machines. Leave off unless you have headroom. |
| Spinning-disk (HDD) | `HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=1` | Sequential writes; avoids parallel-write thrash on HDDs. Leave off on SSD/NVMe. |
@@ -72,8 +82,12 @@ If `huggingface.co` is slow or blocked, point the client at a mirror:
HF_ENDPOINT=https://hf-mirror.com
```
Set it as an environment variable (or in **Settings → environment**) before
downloading. Caveats:
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets
included), or as an environment variable before launching. On first run, the
setup wizard offers the same mirror quick-pick right on the system-check
screen when the endpoint is unreachable — the network check is a warning, not
a blocker, so an offline or firewalled machine can still finish setup once
models are available (mirror, or manual download below). Caveats:
- A mirror serves the **classic** download path, **not Xet** — you lose
chunk-dedup and Xet's parallel fetch, but you gain reachability. On the
+156
View File
@@ -0,0 +1,156 @@
# Translation engines (Dub tab)
OmniVoice dubs in two steps: **transcribe → translate → speak**. The *translate*
step is pluggable — pick the engine in the Dub tab's **Engine** dropdown. Two
engines are **built in** and always available offline; the rest need a small
optional Python package.
| Engine | Category | Needs a package? | Key needed? |
|--------|----------|------------------|-------------|
| **Argos** (Local, Fast) | offline | `argostranslate` (bundled) | no |
| **NLLB-200** (Local, Heavy) | offline | none (uses core `transformers`) | no |
| Google Translate (Free) | online | `deep_translator` | no |
| DeepL | online | `deep_translator` | yes (`DEEPL_API_KEY`) |
| Microsoft Translator | online | `deep_translator` | yes (`MICROSOFT_API_KEY`) |
| MyMemory | online | `deep_translator` | no |
| LLM (OpenAI-compatible) | llm | `openai` | usually yes |
If you pick an engine whose package isn't importable yet, the Engine label shows
a **highlighted Install affordance**, and — if you try to translate anyway — the
backend returns a single, actionable error telling you exactly what to install
(the install command is single-sourced, so the button and the error never
disagree).
## Installing optional translation engines (from-source vs packaged build)
How you add an engine depends on **how you installed OmniVoice**.
### From-source / dev install (one-click)
If you cloned the repo and run OmniVoice from source (`uv sync` + the dev
launcher) or via Docker, the app can install engines for you:
1. In the Dub tab, open the translation settings and pick the engine you want
(e.g. **Google Translate**) from the **Engine** dropdown.
2. A highlighted **Install** button appears next to the *Engine* label. Click it.
3. OmniVoice runs the install into the **same** Python environment the backend
is using (`uv pip install <package> --python <backend-interpreter>`), then
re-probes. When it reports *"restart the backend to load it"*, restart so the
freshly-installed module is importable.
You can also install by hand into the backend venv:
```
uv pip install deep_translator # Google / DeepL / Microsoft / MyMemory
uv pip install argostranslate # Argos (already bundled; rarely needed)
uv pip install openai # LLM (OpenAI-compatible) provider
```
Then restart the backend.
### Packaged / installer build (read-only — use the popover)
The signed desktop installers (`.dmg`, `.msi`, AppImage, `.deb`) ship a
**read-only, code-signed Python environment**. Installing extra packages into it
would break the signature, so **in-app install is intentionally disabled** on
these builds. Selecting an uninstalled engine there shows a highlighted button
that opens a small popover with everything you need:
- **The exact command** to run (with a copy-to-clipboard button) if you *do*
have a from-source checkout somewhere and want the online engines there.
- **Switch to Argos (bundled, offline)** — one click. Argos and NLLB are always
importable in every build, so this is the guaranteed escape hatch: you can
keep dubbing immediately, fully offline, no install required.
- A link back to this page.
**Recommendation for packaged builds:** just use **Argos** (fast, offline) or
**NLLB-200** (heavier, higher quality, offline). They need nothing installed and
never leave your machine. Reach for the online engines only from a from-source
install where you can add their package.
## Translation quality: Fast, Autofit, Cinematic
The **Quality** control in the Dub tab (and Settings → Translation) picks how the
translation is produced:
- **Fast** — a direct one-shot translation from the selected engine (Argos, NLLB,
Google, …). No LLM, no timing awareness.
- **Cinematic** — an LLM refines the literal translation (reflect → adapt) for
natural, in-context phrasing.
- **Autofit** — Cinematic **plus** a strict fit-to-time pass: the LLM rewrites
each line so its target-language reading time fits **within** the segment's
slot (never overruns it). This keeps the video timing intact and avoids the
stressed audio time-stretch you get when a translation is too long for its
slot. Fit is per-language pronunciation-speed aware.
Cinematic and Autofit **require an LLM** (below). If none is configured, they
fall back to Fast with a notice.
## LLM Providers (for Cinematic / Autofit)
**Settings → System → LLM Providers** is the one place to set up the LLM. Pick a
provider, paste its API key, choose a model, **Test** it, and "use for
translation." Supported: OpenAI, OpenRouter, Groq, Cerebras, Google AI (Gemini),
Mistral, Cohere, NVIDIA, GitHub Models, Cloudflare, Hugging Face, SambaNova,
SiliconFlow, **local Ollama / LM Studio** (offline, no key), and a **Custom**
OpenAI-compatible endpoint.
Keys entered here are stored **encrypted** on your machine and never returned to
the UI. For a fully offline setup, pick **Ollama** (`ollama pull llama3.1`) or
**LM Studio** — nothing leaves the machine. Power users can still override any
provider via environment variables (e.g. `GROQ_API_KEY`, or the legacy
`TRANSLATE_BASE_URL` / `TRANSLATE_API_KEY` / `TRANSLATE_MODEL`, which map to the
**Custom** provider).
### Pinning the active provider with `LLM_DEFAULT_PROVIDER`
By default the LLM used for Cinematic/Autofit is the one you mark "use for
translation" in **Settings → LLM Providers**. To force a specific provider
regardless of that stored selection — handy for headless/CI/Docker runs or a
shared machine — set the `LLM_DEFAULT_PROVIDER` environment variable to a
provider id before launching the backend:
```
LLM_DEFAULT_PROVIDER=groq # or openai, openrouter, cerebras, ollama, custom, …
```
Resolution order for the active provider is: `LLM_DEFAULT_PROVIDER` (env) →
your saved selection → the first provider that has a key → none. The id must be
one OmniVoice knows (the ids shown in **Settings → LLM Providers**); an unknown
value is ignored and resolution falls through to your saved selection. While
this env var is set it wins over the in-app picker, so if the UI selection
appears to have "no effect," check whether `LLM_DEFAULT_PROVIDER` is exported.
## LLM Skills (per-feature routing)
**Settings → System → LLM Skills** lists every LLM-powered feature — Cinematic &
Autofit translation, speech-rate slot fitting, glossary auto-extract, direction
parsing, and dictation cleanup — and lets you toggle each one or route it to a
specific provider instead of the global active one. That way sensitive work
(e.g. dictation cleanup) can stay on a local Ollama/LM Studio model while
heavier jobs use a remote provider. A disabled skill degrades exactly like
having no LLM configured: Cinematic/Autofit falls back to Fast, dictation
cleanup passes the raw transcript through, direction parsing uses the keyword
heuristic. Everything defaults to enabled + "use active provider", so existing
setups behave unchanged.
## API keys (online MT engines)
The non-LLM online engines need a key, set as an environment variable before
launching the backend (or in **Settings → Credentials**):
- **DeepL:** `DEEPL_API_KEY` (optionally `DEEPL_BASE_URL` for a self-hosted /
pro endpoint).
- **Microsoft Translator:** `MICROSOFT_API_KEY` (optionally `MICROSOFT_BASE_URL`).
## Troubleshooting
- **"The 'google' translation engine needs the optional deep_translator Python
package…"** — the package isn't installed. On a from-source install, click the
Install button (or run the command above) and restart. On a packaged build,
switch to Argos/NLLB via the popover.
- **Install button does nothing / says "disabled in packaged builds"** — you're
on a signed installer build (expected). Use Argos/NLLB, or add the package in a
from-source checkout.
- **Installed it but still "needs install"** — restart the backend so Python
picks up the newly-installed module.
+90
View File
@@ -0,0 +1,90 @@
# Confucius4-TTS (opt-in engine)
> **Status: validated end-to-end (2026-07-02).** The integration (engine
> registration, dedicated-venv bootstrap, sidecar wire protocol, opt-in gating)
> is done, the sidecar's pure logic is unit-tested
> (`tests/test_confucius4_sidecar.py`), and a live synthesis run on Apple
> Silicon (CPU) produced audible cloned speech — confirming the model API and
> the true output sample rate of **22 050 Hz**. CUDA is the recommended
> hardware; CPU works but is slow (~17× realtime — roughly 100 s for 6 s of
> audio). MPS also runs but is *slower* than CPU (~64× realtime), so the
> sidecar deliberately never selects it. The engine is gated behind
> `OMNIVOICE_CONFUCIUS4_TTS_DIR`, so it's completely inert until you opt in —
> it can't affect the default install on any platform.
[Confucius4-TTS](https://github.com/netease-youdao/Confucius4-TTS) (netease-youdao)
is an LLM-based multilingual / cross-lingual zero-shot voice-cloning TTS.
- **14 languages**: Chinese, English, Japanese, Korean, German, French, Spanish,
Indonesian, Italian, Thai, Portuguese, Russian, Malay, Vietnamese.
- **Unconstrained cloning** — no reference transcript required.
- **Cross-lingual voice transfer** — keep one voice across languages.
- **License:** Apache-2.0. **Hardware:** NVIDIA GPU (CUDA 12.6) recommended;
CPU validated on Apple Silicon but ~17× realtime. Output: 22 050 Hz mono.
Like IndexTTS-2 / MOSS-TTS-v1.5 / dots.tts, it runs in its **own subprocess venv**
so its dependency stack never touches the default OmniVoice interpreter.
## Install
```bash
git clone https://github.com/netease-youdao/Confucius4-TTS.git
cd Confucius4-TTS
uv venv --python 3.10
uv pip install -r requirements.txt
```
> Upstream ships **no `pyproject.toml`/`setup.py`**, so there is nothing to
> `pip install -e` — don't try; it fails. The OmniVoice sidecar puts the clone
> on `sys.path` itself (the same thing upstream's `example.py` does).
**Model weights — all fetched automatically from HuggingFace on first
synthesis (~5 GB total, cached in `$HF_HUB_CACHE`):**
- `netease-youdao/Confucius4-TTS``t2s_model.safetensors` + `s2a_model.pt`
(the tokenizer + `wav2vec2bert_stats.pt` already ship in the clone's
`checkpoints/`).
- `facebook/w2v-bert-2.0` — semantic feature extractor (~2.3 GB).
- `funasr/campplus` — speaker-style encoder (small).
- `nvidia/bigvgan_v2_22khz_80band_256x` — vocoder (BigVGAN and CAMPPlus
*code* is vendored in the clone's `external/`; no Amphion install needed).
Set your `HF_TOKEN` (Settings → Credentials) if you hit rate limits.
Then point OmniVoice at the clone and restart:
- **macOS/Linux:** `export OMNIVOICE_CONFUCIUS4_TTS_DIR=/path/to/Confucius4-TTS`
- **Windows (PowerShell):** `[Environment]::SetEnvironmentVariable("OMNIVOICE_CONFUCIUS4_TTS_DIR","C:\path\to\Confucius4-TTS","User")`
Select **Confucius4-TTS** in Settings → Engines. The first synthesize triggers
the weight downloads above, then generates.
### Optional overrides
- `OMNIVOICE_CONFUCIUS4_CONFIG` — path to `inference_config.yaml` if it isn't at
`<clone>/config/inference_config.yaml`.
## Validation record (2026-07-02, Apple Silicon M-series, CPU)
The sidecar (`backend/engines/confucius4/main.py`) uses:
```python
from confuciustts.cli.inference import ConfuciusTTS
model = ConfuciusTTS(config_path=..., device="cuda") # or "cpu"
audio = model.generate(text=..., lang="en", prompt_wav="ref.wav") # → tensor
sr = model.sample_rate # 22050
```
- ✅ **Live end-to-end run**: English zero-shot clone from a 9.5 s reference —
6.06 s of audible speech (peak 0.85) in 102 s on CPU. `model.sample_rate`
returned **22 050**, matching `target_sample_rate` in
`config/inference_config.yaml`; `CONFUCIUS_SAMPLE_RATE` /
`_DEFAULT_SAMPLE_RATE` are pinned to it (regression-tested).
- ✅ **Not pip-installable upstream** — discovered live; the bootstrap now skips
the editable install unless upstream ships packaging, and both the import
probe and the sidecar resolve `confuciustts` via the clone on `sys.path`.
- ✅ **MPS probed and rejected**: runs, but ~4× slower than CPU (Metal op
fallbacks) — the sidecar selects CUDA when available, else CPU, never MPS.
- ✅ **Sidecar logic unit-tested** (`tests/test_confucius4_sidecar.py`):
language normalization, tensor→PCM (mono/stereo/clip), config-path
resolution, clone sys.path injection, wire framing, synthesize dispatch.
+12 -4
View File
@@ -1,10 +1,11 @@
# Engine venvs & disk usage
Most engines run in-process in OmniVoice's main environment. A few
(**IndexTTS2**, and any engine whose dependencies conflict with the parent's
`torch`/`transformers` pins) run in a **dedicated sidecar venv** so their pins
can't break the rest of the app. Those sidecars are where disk adds up — this
page explains why, and how the on-disk cost is kept down.
(**IndexTTS2**, **MOSS-TTS-v1.5**, **dots.tts**, and any engine whose
dependencies conflict with the parent's `torch`/`transformers` pins) run in a
**dedicated sidecar venv** so their pins can't break the rest of the app. Those
sidecars are where disk adds up — this page explains why, and how the on-disk
cost is kept down.
## Why a sidecar needs its own venv
@@ -48,6 +49,13 @@ the parent whenever the engine allows it.** When it doesn't (IndexTTS2's
`transformers<5` forces an older torch line), the second copy is the
unavoidable price of isolation — not a bug.
The opt-in #498 engines illustrate both sides: **dots.tts** pins
`torch==2.8.0` — the **same** build the parent constrains to — so it shares
almost all of torch with the main venv and only its `transformers==4.57` +
model deps are new. **MOSS-TTS-v1.5** pins `torch==2.9.1+cu128`, a **different**
build, so it pays a full extra multi-GB torch copy on CUDA hosts (the price of
running an 8B model whose stack pins `transformers==5.0`).
> On Linux, the `nvidia-*` CUDA packages are separate wheels, so even across
> *different* torch versions any `nvidia-*` whose pinned version happens to
> match is still shared. On Windows the CUDA DLLs live inside the one torch

Some files were not shown because too many files have changed in this diff Show More