Compare commits

..
34 Commits
Author SHA1 Message Date
a721c1fdcf release: freeze v0.3.14 — version bump, lockfiles, changelog (#1027)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 09:12:24 +05:30
80f10289fe feat(asr): ASR engines get the same Settings picker TTS has (env var still wins) (#1026)
Settings → Engines now stacks one pinned Engine Compatibility Matrix per
family (TTS, ASR, LLM) instead of a single TTS-titled table with the other
families tucked behind a low-discoverability tab. The backend select/prefs
path (family="asr" → prefs.asr_backend, env > prefs > auto-detect) already
worked but was unexercised and undocumented — it's now locked by API and
resolution-order tests, and README + the openai-compat-asr doc stop
promising a picker that didn't exist / denying one that now does.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 04:43:16 +05:30
a0ad314736 docs: CLAUDE.md refresh — de-rot versions, compress shipped stack research, replace the dead GSD gate (#1025)
Three classes of staleness that actively misled agent sessions:

- The Project section hardcoded "latest stable v0.3.5 / main at v0.3.6"
  — eight releases behind. Now points at the sources of truth
  (frontend/package.json, the Releases page) and documents the current
  AUTO_VERSION_BUMP-off holding behavior instead of a version literal
  that rots every release.
- ~165 lines of May-2026 stack research for five capabilities that have
  ALL since shipped (HF-token panel, prefilled-URL bug reporting, uv
  mirror fallback, Supertonic-3, in-repo docs). Compressed to the
  durable don'ts it established (no telemetry endpoints, no app-side
  GitHub tokens, no setx, no MkDocs, no hf_transfer) plus a pointer to
  prefer what's already pinned.
- The GSD Workflow Enforcement gate referenced /gsd-quick//gsd-debug/
  /gsd-execute-phase skills that exist nowhere in this environment; the
  owner explicitly chose direct edits over restoring them (2026-07-08).
  It cost a real mid-task detour when a subagent correctly refused to
  work under an unsatisfiable rule. Replaced with the owner decision
  and the working conventions that actually bind (merge gating,
  check-the-PR-queue-first).

244 → 83 lines. GSD section markers preserved so the generating tool
can still find its blocks.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 04:25:14 +05:30
1808a373a1 fix(linux): AppRun workaround detection reads the BUNDLED WebKitGTK version, not the host's (#961 follow-up) (#1024)
The launcher decided whether to export WEBKIT_DISABLE_COMPOSITING_MODE
by asking the host's pkg-config — but LD_LIBRARY_PATH makes the
BUNDLED libwebkit2gtk the one that actually runs, so on any machine
where the two diverge the detection read the wrong number. This was
the second bug identified during #961's investigation (the reporter
built from source, so their dev packages answered pkg-config with a
healthy version while the shipped bundle ran an older lib) and was
explicitly deferred in #1007 as not-safely-fixable at runtime.

The fix makes it knowable by construction instead: inject-apprun.sh
runs at bundle time ON the build host whose libwebkit2gtk gets
bundled, so it stamps that version into .bundled-webkitgtk-version
inside the AppDir. AppRun reads the stamp first and only falls back to
host pkg-config for bundles predating it. Empty/unreadable stamp fails
safe (workaround on), same philosophy as the missing-pkg-config path.

Tests: 3 new cases in AppRun.test.sh — marker-beats-host in both
directions (broken-marker/healthy-host and the #961 inversion,
healthy-marker/broken-host) plus empty-marker fail-safe. Also wires
AppRun.test.sh into pytest (tests/test_apprun_launcher.py) — it was
previously run by NO CI job, so the launcher could regress silently.

Also documents Windows install-to-another-drive behavior in
docs/install/windows.md (#938): local drives work via the wizard's
directory picker, mapped network drives are a Windows Installer
limitation, and the data directory moves independently of the app.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 03:53:46 +05:30
67789fb31c release: freeze v0.3.13 — version bump, lockfiles, changelog (#1023)
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:52:25 +05:30
da4bef8e42 docs: correct troubleshooting §16 — mic bug was a missing entitlement, not an upstream limitation; changelog for #1016/#1020/#1021 (#1022)
troubleshooting.md §16 claimed the macOS microphone-permission bug was
an unresolved upstream Tauri/wry limitation with no available fix.
That was wrong: @MahdiHedhli read the wry/tauri sources more carefully
and found the real cause — Tauri's Hardened Runtime default blocks mic
hardware access without com.apple.security.device.audio-input in the
bundle's entitlements, which also explains why TCC never listed the
app. Their fix (#1016) is merged; §16 now documents the real mechanism,
credits the correction, and keeps the record-elsewhere workaround for
users on ≤0.3.12 builds.

Also brings CHANGELOG [Unreleased] current for the three merges that
lacked entries: #1016 (mic fix), #1020 (shutdown wait 3s→20s), #1021
(CI flaky-trio root cause + guard).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:27:00 +05:30
0076d0067e test(ci): root-cause and neutralize the flaky trio — a leaked torch fp16 default dtype (#1021)
test_effects_chain / test_generation_audio_guard / test_persona_bundle
failed intermittently on CI (never locally) with identical signatures
across three unrelated PRs today (#1002, #1019, #1016) — costing a
full CI cycle per occurrence and repeatedly muddying merge decisions.

Root cause, confirmed by local reproduction: a leaked
torch.set_default_dtype(torch.float16) from some earlier test in the
CI-Linux ordering. The smoking gun was test_generation_audio_guard's
observed 0.0999755859375 — exactly float16(0.1), i.e.
torch.tensor([0.1, …]) built under a leaked fp16 default. Reproducing
with a simulated polluter locally produced the trio's exact failures:
Pedalboard refuses fp16 audio outright ("only supports 32-bit and
64-bit floating point") and silently returns unmodified audio for
every preset, so test_effects_chain's preset outputs compare
identical; and the fp16 tensor value breaks the sanitize approx-check.

Fix: an autouse conftest guard (same philosophy as the existing
LLM-state isolation guard, #878) that checks torch's default dtype
after every test, resets any leak to float32, and emits a UserWarning
naming the offending test's nodeid — so the actual CI-only polluter
identifies itself in the next CI log instead of being chased blind.
Regression test: a deliberate-leak pair proving reset-between-tests.

Fail-before/pass-after verified: with the guard stashed, a simulated
polluter + the trio reproduced 2/3 failures locally with the exact CI
signatures; with the guard active, 73/73 pass and the warning names
the polluter.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:19:07 +05:30
Mahdi Hedhli faf34348c8 fix(macos): add microphone/camera entitlements so TCC ever sees a request (#1016)
Root cause of #1013 (macOS: "Microphone access denied" but OmniVoice never
appears in Privacy & Security → Microphone to enable it):

Tauri's macOS bundle config defaults `hardenedRuntime` to true, and Hardened
Runtime blocks camera/microphone hardware access unless the matching
entitlement is present — regardless of Info.plist's NSMicrophoneUsageDescription
(that only supplies the *prompt text*, it isn't itself the grant) and
regardless of wry's own WKUIDelegate already granting the request at the
WebKit/JS layer (WryWebViewUIDelegate::request_media_capture_permission
unconditionally calls WKPermissionDecision::Grant — confirmed by reading
wry 0.55.1's source; that part was never the problem). With Hardened Runtime
on and zero entitlements, TCC never registers a request at all, which is
exactly the reported symptom: nothing to enable because the OS never saw a
legitimately-entitled process ask. This also explains the workaround in
#1013 and its comments (launching the raw binary from Terminal works, but
as Terminal's identity, not the app's) — Terminal is a properly entitled,
hardened-runtime process; the ad-hoc/unentitled app binary isn't.

Adds src-tauri/entitlements.plist (com.apple.security.device.audio-input,
plus com.apple.security.device.camera matching the forward-looking
NSCameraUsageDescription already in Info.plist) and wires it in via
tauri.conf.json's bundle.macOS.entitlements. Also corrects the stale
"nothing to do here" module comment in lib.rs that documented the
incomplete assumption this bug falsified.

Verified: built a debug .app (`tauri build --debug --bundles app`) and
diffed `codesign -dv --entitlements -` before/after this change — the
entitlements dictionary goes from absent to containing exactly the two
keys added here, alongside the runtime (Hardened Runtime) flag that was
already on. `cargo test` — 60 passed, 0 failed.
2026-07-09 02:15:28 +05:30
69ce697ee5 fix(backend): shutdown wait bound 3s→20s — post-merge review finding on #1002; absorb #1015's design-path test (#1020)
Greptile's review of the merged #1002 flagged a real residual gap: a
cold transformers import alone can exceed the 3s shutdown wait, and
cancelling the asyncio task doesn't stop the underlying OS thread —
so quitting during an unusually slow preload could still let shutdown
report "done" while that thread was alive, the exact #1000 class with
lower odds. Python cannot forcibly kill a running thread, so no finite
bound eliminates this outright; 20s shrinks the window from "any
preload" to "an unusually slow cold-import," the practical ceiling
before a long shutdown becomes its own complaint. New source-level
contract test pins the production bound at ≥15s so a future edit
can't quietly shrink it back without deliberate consideration.

Also absorbs the one test case from community PR #1015 (superseded by
the earlier-merged #1017, which duplicated it — my fault for not
checking the PR queue) that the merged version lacked: the
design/instruct path with no ref kwargs at all stays untouched by the
ref_text forwarding fix.

Co-authored-by: mergetest <test@local>
Co-authored-by: MahdiHedhli <noreply@github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:10:06 +05:30
93a3cb260a fix(ui): dub editor play button no longer sticks disabled; remove donate heart from nav rail (#1019)
WaveformTimeline's play button (disabled={!ready}) stayed permanently
disabled whenever the initial WaveSurfer decode failed and the
component fell back to loading pre-computed peaks. The waveform still
rendered fine from those peaks (nothing looked visibly broken), but
`ready` was only ever flipped by the 'ready' event re-firing on that
recovery load — which this component's own error-handling never
actually confirmed, just assumed. Each of the three fallback ws.load()
calls now explicitly confirms readiness once it settles (via .then()/
.catch(), or the existing synchronous-throw catch), instead of hoping
the event fires again.

Regression test: WaveformTimeline.readyFallback.test.js — a
source-level contract guard (driving a real decode-failure/recovery
sequence through jsdom is brittle, same house pattern as the sibling
WaveformTimeline.unlock.test.js) asserting every fallback load in the
error handler is followed by an explicit setReady(true).

Also removes the "Support OmniVoice" heart button from NavRail — the
donate page stays reachable from Settings' footer and the Contact page.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:09:54 +05:30
33379890ad fix(voice): free-text instruct now filtered before every generate/save call (#1010) (#1018)
buildDesignInstruct() already keeps Studio's design/clone generate
calls (useTTS.js) from round-tripping a 400 "Unsupported instruct
items" — it filters free-text against the active engine's supported
vocabulary client-side, with a toast instead of a failed request. Three
other call sites built their own instruct string directly and skipped
it entirely:

- handleSegmentPreview (Dub tab's per-segment preview) — instruct comes
  straight from segment/preset data; a preset's raw attrs merged with a
  free-text style field can carry phrases outside the vocabulary.
- handleSaveProfile / handleSaveHistoryAsProfile — both always create a
  kind='clone' profile; the backend only sanitizes instruct on save for
  kind='design' (see profiles.py's heal_design_instruct branch), so a
  clone profile could silently persist an unusable instruct and then
  400 every single time it's later used to generate.

All three now filter through the same buildDesignInstruct({}, instruct)
call useTTS.js's own clone path already uses, with the same
unsupported/duplicate-item toasts.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:47:06 +05:30
7f8a42ce51 fix(tts): mlx-audio CSM cloning drops ref_text, breaking every clone attempt (#1012, #1013) (#1017)
MLXAudioBackend.generate() reads voice/ref_audio/language/speed from
its kwargs but never extracted ref_text — it was built, then silently
never passed through to self._model.generate(). CSM (sesame.py) only
builds its cloning context when BOTH ref_audio AND ref_text are
present; with ref_text missing, the context list stays empty and
indexing into it raises "IndexError: list index out of range" deep
inside mlx-audio, instead of the clone ever being attempted. Voice
cloning on the CSM engine could never have worked as shipped.

generation.py already threads ref_text all the way through — even
auto-transcribing it via the GPU pool when the caller supplies
ref_audio without one (~line 780) — so the value was always available
in kwargs; it just never survived the crossing into this specific
backend.

Reported with the precise root cause and a working fix (community
member independently diagnosed and patched it locally, confirmed
working on MPS/0.3.12). Two-line fix: extract ref_text and pass it
through when both ref_audio and ref_text are present (guards against
passing an orphaned ref_text with no accompanying audio to engines
that don't expect it).

Tests: tests/test_engines.py — ref_text is passed through when paired
with ref_audio, omitted when ref_audio is absent.

Also documents the second bug from the same report (#1013): macOS
microphone permission never prompts, so OmniVoice never appears in
System Settings to grant access. Root-caused to an unresolved upstream
Tauri/WebKit limitation (WKWebView's requestMediaCapturePermissionFor
delegate — wry#1195, tauri#11951, fix wry#1196 still open/unmerged, no
released version to bump to) — not something fixable here without an
unverified native Rust/WKWebView hack this session has no way to test.
Documented in docs/install/troubleshooting.md with the confirmed
workaround (record elsewhere, upload the file).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:34:18 +05:30
4de3c824d0 fix(gallery): surface the real error instead of a generic guess (#1009)
Found while triaging Discord: a community member (lehoangan227) hit
"Could not create that voice — the engine may be loading" trying to
use an archetype from the Gallery. That message is hardcoded and
shown for ANY failure — the actual cause (a 500, a validation error,
anything) is caught and discarded.

api/client.js's ApiError already builds a clean, user-facing message
for every failure mode (HTTP status + backend detail, a network
failure, or a detected backend crash) — this codebase's own
established convention elsewhere is to interpolate that message via
`{{message}}` (see BatchQueue.jsx, Settings.jsx, ToolsPage.jsx). The
Gallery's own catch blocks just weren't following it.

Fixed the whole class across VoiceGallery.jsx (use/preview),
CommunityZone.jsx (add-to-voices, whose catch clause didn't even bind
the error), and ImportsZone.jsx (search/upload/save/delete/trim —
handleDelete previously failed completely silently, no message at
all). All now interpolate the real error message, matching the
gallery.download_failed key that already did this correctly a few
lines away.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:45:34 +05:30
11fbbd3f6d fix(dub): cross-language dub no longer speaks source-language reference text verbatim (#1004) (#1008)
extract_speaker_clones/extract_segment_refs pair each audio slice (cut
at ASR segment timestamps) with that segment's own `text` field, on
the assumption the two agree. They routinely don't — Whisper (and
friends) frequently drift on segment boundaries: a trailing word
audible in [start, end] but missing from text, or vice versa. When the
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming
breaks down and the clone can emit the mismatched reference text
itself instead of the target-language line it was asked to speak —
reported with an exceptionally clear root-cause diagnosis and a
working A/B repro (matched pair: clean on the first try; mismatched
pair: wrong language 6/6 times).

Fix (as proposed in the report): re-transcribe each written reference
clip via the already-loaded, already-warm active ASR backend and use
that transcript as ref_text — this guarantees the pair matches by
construction, independent of whether the original segment text was
ever right. Falls back to the original text on any re-transcribe
failure or empty result — never a regression from current behavior,
only ever a fix.

New services.speaker_clone.refine_ref_text (single clip, unit-testable
against a duck-typed fake ASR backend) and refine_ref_texts (batch —
one executor round-trip per whole clones/seg_clones dict rather than
one per reference). Wired into dub_core.py's two clone-extraction call
sites, routed through _gpu_pool to match the established convention
for ASR-backend calls (the model is mid-lifecycle: TTS is offloaded,
ASR is loaded and exclusive, right where the existing per-chunk
transcribe calls already run on this same pool).

Tests: tests/test_speaker_clone_purity.py — 6 new cases covering the
mismatch-correction path, ASR-failure fallback, empty-transcript
fallback, no-backend no-op, and batch behavior (one failing entry
doesn't affect the others). Full backend suite: 2412 passed, 0 failed.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:18:15 +05:30
f546f04c8e fix(ci): bump Linux release runner to ubuntu-24.04, fixing stale bundled WebKitGTK (#961) (#1007)
The AppImage bundles whatever libwebkit2gtk-4.1-dev the build runner's
apt repos resolve at build time (see the "Linux system deps" step) —
AppRun's LD_LIBRARY_PATH then makes that bundled copy take priority
over the host's system WebKitGTK at runtime. ubuntu-22.04's version
was stale relative to what current distros (Ubuntu 24.04+, Fedora 44)
ship, which is why a from-source build (linking straight against the
host's healthy system library) worked fine on the exact machine where
the shipped AppImage white-screened — the released binary was running
an older, buggier WebKitGTK under the hood regardless of the host.

Bumped the Linux release matrix entry to ubuntu-24.04, and ci.yml's
Tauri shell-check job to match (its own comment already says "Mirror
release.yml" — now it actually does, so a green PR check accurately
predicts the release build will also succeed).

Raises the AppImage's glibc floor from 2.35 to 2.39 (Ubuntu 24.04+) —
README's system-requirements table corrected from the now-false
"Ubuntu 20.04+" claim. No reports of anyone on a pre-2022 distro.

This does not fix the AppRun launcher's separate, related bug (its
WebKitGTK-version auto-detection reads the *system's* pkg-config
version, not the version actually bundled and running) — that would
need a reliable way to read the bundled .so's version from within the
AppImage, which isn't straightforward (WebKitGTK's soname doesn't map
1:1 to its release version) and isn't verifiable without a real Linux
build environment to test against. Left as a known, separate gap.

Cannot be verified from here on a real Ubuntu 26.04 machine — shipped
on the strength of the root-cause diagnosis, pending the reporter's
confirmation.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:21:50 +05:30
270b3b1c4c fix(backend): quitting mid-preload no longer reports a clean shutdown while a GPU-pool thread is still importing (#1000) (#1002)
A user-pasted backend log revealed the real cause behind a class of
'can't reach backend' reports: three rapid restart cycles, each ending
with 'Shutdown: done.' immediately followed by a 'Model loading failed:
Could not import module AutoFeatureExtractor' error. That error text is
transformers' own generic lazy-import wrapper (import_utils.py's
_LazyModule.__getattr__), not a real dependency problem — pyproject.toml
already pins transformers/torch/torchaudio/soundfile/librosa as core,
non-optional deps, and the same venv loaded the model successfully 90
seconds later in the same log.

Root cause: preload_task and capture_preload_task were created at
startup but never referenced in the lifespan shutdown block — idle_task
and worker_task got cancelled-and-awaited, the preload tasks were simply
abandoned. Cancelling an asyncio task awaiting run_in_executor() can't
stop the underlying OS thread once it's inside blocking import/load
work, so 'Shutdown: done.' logged while a GPU-pool thread was still
mid-Version: ImageMagick 7.1.2-25 Q16-HDRI aarch64 037e46295:20260604 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/license/
Features: Cipher DPC HDRI Modules
Delegates (built-in): bzlib freetype heic jng jpeg lcms ltdl lzma png tiff webp xml zlib zstd
Compiler: clang (21.0.0)
Usage: import [options ...] [ file ]

Image Settings:
  -adjoin              join images into a single multi-image file
  -border              include window border in the output image
  -channel type        apply option to select image channels
  -colorspace type     alternate image colorspace
  -comment string      annotate image with comment
  -compress type       type of pixel compression when writing the image
  -define format:option
                       define one or more image format options
  -density geometry    horizontal and vertical density of the image
  -depth value         image depth
  -descend             obtain image by descending window hierarchy
  -display server      X server to contact
  -dispose method      layer disposal method
  -dither method       apply error diffusion to image
  -delay value         display the next image after pausing
  -encipher filename   convert plain pixels to cipher pixels
  -endian type         endianness (MSB or LSB) of the image
  -encoding type       text encoding type
  -filter type         use this filter when resizing an image
  -format "string"     output formatted image characteristics
  -frame               include window manager frame
  -gravity direction   which direction to gravitate towards
  -identify            identify the format and characteristics of the image
  -interlace type      None, Line, Plane, or Partition
  -interpolate method  pixel color interpolation method
  -label string        assign a label to an image
  -limit type value    Area, Disk, Map, or Memory resource limit
  -monitor             monitor progress
  -page geometry       size and location of an image canvas
  -pause seconds       seconds delay between snapshots
  -pointsize value     font point size
  -quality value       JPEG/MIFF/PNG compression level
  -quiet               suppress all warning messages
  -regard-warnings     pay attention to warning messages
  -repage geometry     size and location of an image canvas
  -respect-parentheses settings remain in effect until parenthesis boundary
  -sampling-factor geometry
                       horizontal and vertical sampling factor
  -scene value         image scene number
  -screen              select image from root window
  -seed value          seed a new sequence of pseudo-random numbers
  -set property value  set an image property
  -silent              operate silently, i.e. don't ring any bells
  -snaps value         number of screen snapshots
  -support factor      resize support: > 1.0 is blurry, < 1.0 is sharp
  -synchronize         synchronize image to storage device
  -taint               declare the image as modified
  -transparent-color color
                       transparent color
  -treedepth value     color tree depth
  -verbose             print detailed information about the image
  -virtual-pixel method
                       Constant, Edge, Mirror, or Tile
  -window id           select window with this id or name
                       root selects whole screen

Image Operators:
  -annotate geometry text
                       annotate the image with text
  -colors value        preferred number of colors in the image
  -crop geometry       preferred size and location of the cropped image
  -encipher filename   convert plain pixels to cipher pixels
  -extent geometry     set the image size
  -geometry geometry   preferred size or location of the image
  -help                print program options
  -monochrome          transform image to black and white
  -negate              replace every pixel with its complementary color
  -quantize colorspace reduce colors in this colorspace
  -resize geometry     resize the image
  -rotate degrees      apply Paeth rotation to the image
  -strip               strip image of all profiles and comments
  -thumbnail geometry  create a thumbnail of the image
  -transparent color   make this color transparent within the image
  -trim                trim image edges
  -type type           image type

Miscellaneous Options:
  -debug events        display copious debugging information
  -help                print program options
  -list type           print a list of supported option arguments
  -log format          format of debugging information
  -version             print version information

By default, 'file' is written in the MIFF image format.  To
specify a particular image format, precede the filename with an image
format name and a colon (i.e. ps:image) or specify the image type as
the filename suffix (i.e. image.ps).  Specify 'file' as '-' for
standard input or output., and interpreter finalization tore down module
state under it — producing exactly this misleading error.

Fix: extract the existing cancel+bounded-await pattern into
_cancel_and_await_tasks() and apply it to all four background tasks, not
just two. An early-stage load (still importing, not yet mid weight-
download) now gets a real chance to finish before shutdown proceeds; a
load genuinely deep in blocking work still times out at the same 3s
bound, and _reset_gpu_pool() abandons it same as before. Also: both
error handlers around this path logged only str(exc), discarding
__cause__ — added exc_info so a future incident (even one this fix
doesn't fully prevent) surfaces the real underlying error instead of the
misleading generic wrapper text.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:09:18 +05:30
5a7d9cc05c feat(asr): generic OpenAI-compatible transcription backend (#877) (#1003)
First slice of the community's two-track proposal for #877: a generic
OpenAI-compatible ASR backend that works TODAY, without waiting on
transformers to ship a direct Qwen3-ASR integration (tracked separately,
still blocked upstream). Points OmniVoice's transcription at any server
exposing POST /v1/audio/transcriptions — a self-hosted Qwen3-ASR/
FunASR/SenseVoice server, or OpenAI's own API.

- New OpenAICompatASRBackend (backend/services/asr_backend.py): a pure
  network client, no local model, no install. Prefers
  response_format=verbose_json for real per-segment timestamps,
  degrades to plain text (matching MoonshineASRBackend's shape) when a
  minimal server rejects that format. Never leaks a raw SDK/httpx
  exception to the caller (#977 convention) — wraps network/auth
  failures in a clean, actionable RuntimeError naming the server.
- Settings persist via the same encrypted-secret convention as
  services/llm_providers.py (settings_store.set_secret for the API key
  — Fernet-encrypted, never a .env row, never echoed back; get_text/
  set_text for base_url/model). New GET/PUT /api/settings/
  asr-openai-compat, loopback-gated like every other settings route.
- Frontend: a small settings panel (Settings → Models) mirroring
  HFMirrorPanel's exact structure. No ASR engine picker exists yet for
  ANY ASR backend (only TTS has one) — activating this engine still
  needs OMNIVOICE_ASR_BACKEND=openai-compat-asr; documented plainly
  rather than pretending otherwise.
- README's ASR Engines table (9 → 10 engines) and docs/features.yaml's
  drift-checker inventory updated; the '9 engines, all fully local'
  claim corrected since this one genuinely isn't.
- docs/engines/openai-compatible-asr.md: setup steps + an explicit
  privacy note (unlike every other ASR engine, audio leaves the
  machine to whatever server is configured).

Regression tests: tests/test_asr_openai_compat_877.py (12 tests) —
is_available() gating, verbose_json + plain-text response adaptation,
network-failure error hygiene, SDK retry disabling, and the settings
endpoints' persist/mask/clear-vs-unchanged semantics.

Fixed two real full-suite-only failures found during verification (not
brushed aside): the API route inventory snapshot needed regenerating
for the two new routes, and this file's own tests had a module-
staleness bug — a collection-time settings_store import went stale
relative to a test-time-fresh fixture when another test elsewhere in
the ~2400-test suite reimports the module — fixed by making
settings_store itself a fixture resolved at test-run time, same
lesson already applied to tests/test_mm2_lifecycle.py earlier this
session.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 08:33:03 +05:30
5871ec9d28 release: freeze v0.3.12 — version bump, lockfiles, changelog (#1001)
19-issue GitHub sweep: 16 PRs merged since v0.3.11, most fixing reports
filed same-day or in the days prior. Highlights: MLX-Audio's 6 other
curated models are finally selectable (was silently stuck on Kokoro
regardless of what was downloaded), first-run no longer dead-ends behind
restricted networks or corporate TLS-inspecting proxies, dubbing/batch
TTS honor your active engine selection, and a run of sharp community
diagnoses (ROCm wheel index, Windows dictation focus-steal, a genuine
frontend crash regression) got fixed largely because reporters did the
hard diagnostic work themselves.

Full backend suite: 2390 passed, 0 failed. Full frontend suite: 918
passed, 0 failed. Version lockstep (tests/test_app_version.py): 6/6
passed. Docker frozen-lockfile parity (bun install --frozen-lockfile):
clean, no drift.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 07:28:41 +05:30
58b023ff16 fix(diagnostics): TTS generate timeout message tells you to Flush/Unload (#939) (#999)
The guard itself (#851/#896) is working as designed — this is a message
gap, not a logic bug. The GPU-branch timeout guidance explained VRAM
contention but never mentioned the Flush/Unload action that actually
resolves it, even though: (a) that action already exists (POST
/system/flush-memory, wired to the header's Flush button), and (b) the
sibling ASR-timeout guard's message already recommends it verbatim
(asr_backend.py's _CUDA_VRAM_BUDGET_GB guidance). The maintainer ended up
manually explaining 'Settings → Models → Flush caches / Unload' in an
issue thread reply — information the error message should have carried
itself.

String-only change, no control-flow touched, mirrors the exact precedent
of #896 (a guidance-only change to this same function).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 05:18:00 +05:30
b61d17dd61 fix(diagnostics): filter Tauri's benign IPC-fallback warning from frontend log capture (#975) (#998)
On some Windows configurations, Tauri's custom-protocol IPC probe fails
once at startup and Tauri logs a console.warn before silently — and
successfully — falling back to postMessage + WebSocket. Fully functional,
happens at most once per launch, and not a bug in our code (confirmed:
this is Tauri's own internal fallback mechanism, structurally intentional
across its recent 2.11.x releases, not something being actively patched
upstream — so not bumping the framework speculatively for this).

It IS real noise though: as a captured console.warn it spuriously flips
the Settings > Logs footer's Frontend pill to "1 warning" on every
affected Windows launch. Filtered at the capture source (consoleBuffer.js)
rather than the display layer, so it never enters the ring buffer or a
copied diagnostic dump either — narrowly scoped to this one known message
prefix, not a general warning-suppression mechanism.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 05:17:12 +05:30
93aab6dadc docs(linux): mention yt-dlp as an optional prerequisite (#973) (#997)
The preflight system check already warns in-app when yt-dlp is missing
(Voice Gallery/Dub YouTube downloads fail without it), but the install
docs never mentioned it — a user has to hit the in-app warning first
instead of seeing it up front alongside the other optional prereqs.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:42:00 +05:30
7098be1c8b fix(dictation): Windows pill no longer steals foreground focus (#982) (#996)
Root-caused and verified against the actual code (community diagnosis in
#982 was accurate): showing the dictation pill — an always-on-top top-level
WebviewWindow — grants it Win32 foreground activation by default on
Windows, which macOS doesn't do for a shown window. Nothing marked the pill
non-activating, so on Windows the synthesized Ctrl+V from `simulate_paste`
landed back in the pill instead of the app being dictated into. Code review
also found a second, more direct culprit at the same call sites: the
global-shortcut Pressed handler's `win.set_focus()` was only skipped on
macOS (`#[cfg(not(target_os = "macos"))]`), so Windows explicitly focused
the pill on top of the implicit Win32 activation; the tray "dictate" menu
handler called `win.show(); win.set_focus();` unconditionally on every
platform, including Windows.

This is the Windows counterpart of #287 (macOS auto-paste — don't steal
focus): same class of bug, same "pill must stay unfocused so the paste
lands in the target app" intent (already stated in the `grant_webview_
media_permissions` module comment), different OS mechanism.

Fix, mirroring #287's platform-cfg pattern:
  1. WS_EX_NOACTIVATE applied to the pill's HWND once, right after creation
     (`mark_pill_noactivate`), via the `windows` crate pinned to the exact
     0.61.x tauri itself already resolves to — Cargo.lock confirms this
     unifies to the same `windows 0.61.3` already in the graph, so
     `WebviewWindow::hwnd()`'s HWND and our Win32 calls share one type and
     no new crate version was added.
  2. `ShowWindow(SW_SHOWNOACTIVATE)` (`show_pill_noactivate`) in place of
     `.show()` at the two dictation-trigger call sites (global shortcut +
     tray "Start Dictation"), since `.show()` alone still raced the style
     bit on some paths.
  3. The explicit `set_focus()` calls at those same two sites are now
     skipped on Windows too, the same way they already were on macOS.

macOS and Linux are untouched: the macOS cfg branch is unchanged, and the
Linux branch of the `set_focus()` guard still runs exactly as before.

The pill's auto-dismiss (`scheduleDismiss`/`dismiss` in CaptureWidget.jsx)
was checked and is a plain unconditional setTimeout chain — it is not
gated on any native focus-loss/paste-completion signal, so there's no
independent bug to fix there. The "never dismisses" symptom is a
consequence of the focus-steal, not a separate stall: once the pill wrongly
held foreground for the whole session, hiding it later left Windows'
foreground state inconsistent. With the pill never taking focus, the
target app stays foreground throughout and there's nothing to reconcile.

Win32 window-activation syscalls (`#[cfg(target_os = "windows")]`) can't
run under `cargo test`/`cargo build` on this non-Windows CI runner, so the
new `pill_noactivate_tests` module tests the pure flag math
(`with_noactivate_style`) instead — platform-agnostic, runs everywhere,
verified passing here. The actual HWND-touching code is logic-reviewed but
UNVERIFIED on real Windows; the reporter offered to test a patched build,
which is the recommended next step before this ships in a release.

`cargo build` and `cargo test --lib` both pass (60/60 tests, including the
3 new ones); the pre-existing `setup.rs` unreachable_code warning (#286) is
unrelated.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:41:25 +05:30
8520b84b68 fix(clone): voice-design panel no longer crashes on a partial vd_states shape (#983) (#995)
Crash: DesignMethodPanel's optLabel() called val.replace(...) on an
undefined category value — a regression from f678e33, which swapped a
safe plain template literal for an i18n lookup that assumes vdStates[key]
is always a string. Both occurrences (the label kicker and the chip
list) are now guarded, falling back to 'Auto' the same way the rest of
the component treats an unset category.

Root cause: vdStates could actually go partial in real usage. Selecting
a design profile (useProfiles.js) or restoring legacy localStorage state
(useAppData.js) applied the backend/stored vd_states object as-is, with
no check that all 6 CATEGORIES keys were present — so an older client,
hand-edited payload, or partial API write reproduced the crash on
selection. Both call sites now run the restored object through
mergeDescribedAttrs() (voiceInstruct.js), the existing completion helper
already used for the "describe your voice" path, which fills any
missing/unknown category with 'Auto'. useAppData.js also gained the
typeof === 'object' guard useProfiles.js already had.

Closes the class at the source: POST /profiles now completes vd_states
against CATEGORY_ORDER (core/describe_voice.py, the same list the
frontend's CATEGORIES mirrors) before persisting, so a design profile
can never be *saved* with an incomplete shape regardless of which
client wrote it — updated two existing tests whose fixtures asserted
the old (partial) persisted shape.

Regression tests: DesignMethodPanel render test with a partial vdStates
input, a mergeDescribedAttrs unit test for the exact partial shape from
the issue, and a backend test asserting POST /profiles fills all 6 keys.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:34:14 +05:30
549fa4009f feat(engines): expose MLX-Audio's curated model picker (#981) (#994)
mlx-audio multiplexes 7+ curated models (Kokoro, CSM, Qwen3-TTS, Dia,
Chatterbox, MeloTTS, OuteTTS) behind a single "mlx-audio" backend id, but
MLXAudioBackend resolved its active model ONLY from the
OMNIVOICE_MLX_AUDIO_MODEL env var — invisible to Settings and unreachable
without restarting the packaged app with that var set. A user who
downloaded e.g. Llama-OuteTTS via Settings → Models had no way anywhere
in the UI or API to actually load it; the backend silently kept using
Kokoro.

Fix:
- MLXAudioBackend.__init__ now resolves its model via
  prefs.resolve("mlx_audio_model_id", env=..., default=...), mirroring
  active_backend_id()'s env > prefs > default order exactly.
- get_active_tts_backend()'s switch-detection now also tracks the
  resolved mlx-audio model key, so a model-only change (same backend id)
  invalidates the cached instance and reconstructs it — no app restart
  needed to pick up a different curated model.
- POST /engines/select gained an optional model_id field; for
  family=tts/backend_id=mlx-audio it validates against
  MLXAudioBackend.CURATED_MODELS (or a raw HF repo id, matching the
  class's existing tolerance) and persists it via prefs.
- GET /engines now includes a curated_models roster + active_model_id on
  the mlx-audio entry only.
- Settings → Engines renders a small model dropdown on the mlx-audio row,
  pre-selected to the active model, wired through selectEngine's new
  optional modelId argument.

Regression coverage: prefs resolution + env override, cache invalidation
on model-only switch, /engines/select 400s on an unknown model id and
persists a valid one, curated_models present only on mlx-audio, and a
new EngineCompatibilityMatrix vitest suite for the dropdown.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:32:24 +05:30
dfe2bd1bc3 fix(install): classify SSL handshake failures + trust the OS cert store (#976) (#992)
Windows users behind a corporate/antivirus TLS-inspecting proxy got a raw
`[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE]` on every model install — the TCP
connection reaches the server fine, but the handshake fails because the
OS trusts the proxy's re-signed root CA and Python's bundled certifi CA
list doesn't. A genuinely different failure mode from #984 (that was
TCP-level unreachability to a blocked host, before any TLS negotiation).

- backend/core/failure.py: new SSL_HANDSHAKE_FAILURE classification
  (handshake/cert-verify-failed/sslv3_alert/sslcertverificationerror
  substring markers) with an actionable hint, added to
  _CONTEXT_FREE_HINT_CLASSES so append_hint() (already called by
  setup/download.py's install worker) surfaces it without further wiring.
- backend/main.py: truststore.inject_into_ssl() at module level, before
  any huggingface_hub/requests/httpx network I/O — patches ssl.SSLContext
  to verify against the OS trust store instead of only certifi's bundled
  CA list. Not platform-gated (correctness improvement everywhere);
  wrapped in try/except so it never blocks startup.
- pyproject.toml/uv.lock: truststore>=0.9 — pure Python, MIT, PyPA-
  maintained, zero transitive deps, same class of fix as socksio.

Verified: uv lock --check + uv sync --frozen clean (lockfile diff is
just the one new package); main.py imports cleanly; full backend suite
passes; no hiddenimports entry needed (main.py is PyInstaller's direct
entry script per backend.spec, so a top-level import traces normally —
unlike socksio's case, which was httpx's internal lazy import).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:29:21 +05:30
324d417d27 fix(engines): mlx-audio no longer crashes on unsupported languages, error messages never leak raw exception internals (#977) (#993)
Root cause: MLXAudioBackend.generate() blindly truncated the full
language display name to two characters (language[:2].lower()),
assuming an ISO code — 'Dutch' -> 'du', which crashed Kokoro's vendored
pipeline's internal assertion (assert lang_code in LANG_CODES, (lang_code,
LANG_CODES)) for any language whose first two letters didn't coincidentally
match one of Kokoro's single-letter codes. The raw AssertionError's
tuple-containing-a-dict args then leaked straight into the user-facing
500 message via two stacked f"...{e}" formatters in generation.py.

- resolve_kokoro_lang_code() resolves against the AUTHORITATIVE
  ALIASES/LANG_CODES table read from the installed mlx_audio package
  (never a hardcoded guess), and only applies when Kokoro is the actual
  active curated model — other curated models (CSM, Dia, Qwen3-TTS,
  OuteTTS, ...) either ignore the kwarg or expect a different format, so
  Kokoro's strict validation doesn't wrongly reject them. Unsupported
  languages now raise a clear ValueError naming what Kokoro supports,
  which generation.py already converts to a clean 400.
- _safe_exc_text() hardens both generic exception formatters in
  generation.py: if any element of an exception's .args is a container
  (dict/list/tuple/set), never interpolate str(e) raw — name the
  exception type and point at the log instead. Protects every current
  and future engine's generate() from leaking a raw container repr, not
  just this one Kokoro assertion.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:29:01 +05:30
637020b82b fix(engines): nemo-parakeet install hint stops recommending a shared-venv-breaking pip install (#974) (#991)
The Engines page told users to run `pip install nemo_toolkit[asr]` for
the NeMo Parakeet ASR engine. nemo_toolkit[asr]==2.7.3 hard-pins
transformers>=4.57,<4.58, which is unsatisfiable alongside OmniVoice's
own transformers>=5.3 requirement (needed by
omnivoice/models/omnivoice.py for HiggsAudioV2TokenizerModel). A user
who followed the hint ended up with a backend that wouldn't start
(ImportError: cannot import name 'HiggsAudioV2TokenizerModel').

_INSTALL_HINTS["nemo-parakeet"] in backend/services/asr_backend.py now
states plainly that installing into the shared venv will break the
backend, names the transformers conflict, and tells users to use a
separate/dedicated Python environment instead — without implying a
safe one-line fix or an isolated-venv env var exists (unlike
dots-tts/moss-tts-v15/confucius4-tts, nemo-parakeet has no isolated
venv option yet; that's a separate, larger follow-up).

Also adds one sentence to docs/install/troubleshooting.md's existing
"engine venv clash" section (#11) pointing at the same class of issue
on the ASR side, and a regression test asserting the hint never again
contains the literal bare `pip install nemo_toolkit[asr]` string.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 04:26:12 +05:30
dab6456581 docs(linux): stop advertising a .deb package that isn't published (#990)
README's Quickstart badges linked a 'Download Debian .deb' button
straight to the releases page — but .deb bundling was deliberately
dropped from release.yml (tauri-cli bug, 'Failed to create control
scripts') and no release has ever shipped one. A community member
investigating #961 confirmed this by checking the actual release
assets. Users clicking that badge got a broken promise, not a package.

Removed the badge; docs/install/linux.md's '## Install (.deb)' section
now honestly states it's unavailable pending a tauri-cli fix, points to
the AppImage as the supported path, and keeps the historical pre-v0.3
.deb upgrade note (ffprobe conflict) since that's still relevant to
existing installs.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:40 +05:30
015d29cafb test(voice-design): lock in RTL/non-Latin script handling for the instruct field (#980) (#989)
Issue #980 reported a raw 400 for Hebrew text ('שמואל') typed into the
Clone tab's Style field. Investigated: this is the same failure class
as #612 (Vietnamese free-text) and was already fixed when #612 landed
in commit 10b9d69 (first released v0.3.8) — buildDesignInstruct() drops
any unsupported free-text client-side before it ever reaches the
backend's validator, regardless of script. The reporter was on v0.3.7,
which predates that fix.

No behavior change needed — only a regression test, since the existing
Vietnamese test case covered Latin-script-with-diacritics but nothing
exercised a right-to-left / non-Latin script specifically.

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:24 +05:30
1a03c59f82 fix(install): AMD ROCm torch reinstall targets rocm6.4, not rocm6.2 (#988)
Community-diagnosed (issue #972, Kaihui-AMD): pyproject.toml pins
torch==2.8.0, but the rocm6.2 wheel index only ever published up to
2.5.1 — the reinstall silently failed to resolve and fell back to the
default CUDA build, which runs on CPU on an AMD GPU. The failure was
correctly logged (bootstrap.rs's emit_log warning), just never actioned
because the index itself couldn't succeed. rocm6.4 carries a matching
torch==2.8.0 build.

Docs updated with the corrected index plus a repo.radeon.com find-links
path for users who want a driver-matched ROCm 7.2.x build the PyTorch
index doesn't carry (OMNIVOICE_TORCH_INDEX only accepts a PEP 503 index,
not find-links, so that's documented as a manual step).

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 04:25:00 +05:30
8281b7c798 fix(engines): dub and batch TTS honor the active-engine selection (#987)
* fix(engines): dub and batch TTS honor the active-engine selection — with a real capability gate, not a silent OmniVoice fallback

Dub generation and batch TTS hardcoded services.model_manager.get_model()
(OmniVoice) regardless of the engine picked in Settings → Engines. A user
selecting VoxCPM2 (or any other engine) still got OmniVoice output with no
error — the silent fallback IS the bug class, not just the one report.

Root-caused and fixed for the whole class:
  - New `TTSBackend.supports_cloning` capability flag (default True) marks
    engines that can only offer fixed preset voices — kittentts,
    supertonic3, sherpa-onnx set it False. MLXAudioBackend exposes it as an
    instance property (Kokoro doesn't clone, CSM does) since the adapter
    multiplexes multiple models with different capabilities.
  - `cloning_capable_engine_ids()` and a shared `resolve_generation_backend()`
    helper in services/tts_backend.py centralize engine resolution
    (id → is_available() → routing gate → optional cloning gate), mirroring
    generation.py's /generate resolution instead of inventing a third
    parallel mechanism. Both routers now standardize on the existing
    get_active_tts_backend() cache (unload-on-switch already handled).
  - dub_generate.py's two TTS-generate call sites (main run + OOM retry) and
    the /dub/preview-segment route resolve once, up front, with
    require_cloning=True — dub's ref_audio is populated for essentially
    every real job, so an engine that can't clone fails the whole job with
    one actionable message instead of mis-cloning per segment.
  - batch.py resolves once per job, require_cloning only when voice_id is
    pinned — an unpinned batch job runs fine on any engine.
  - Applied the three pre-existing TODO(#312) comments: mastering now skips
    via `applies_own_mastering` for both pipelines, matching generation.py.

Regression tests cover the capability-id list, the fail-fast gate (proving
no OmniVoice fallback), the success path on a selected non-OmniVoice
engine, batch's pinned-vs-unpinned voice_id behavior, and the mastering
skip for both pipelines. Three existing dub tests that mocked get_model()
directly were updated to mock the new resolver instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(engines): exclude model-dependent adapters from cloning_capable_engine_ids()

getattr(cls, "supports_cloning", True) at the CLASS level returns a
property descriptor object (always truthy) when the flag is an instance
@property, not a plain attribute — MLXAudioBackend uses exactly this
pattern because its cloning capability depends on which of its 7+ curated
models is loaded (only CSM clones; Kokoro etc. don't). Without this fix,
the dub/batch capability-gate error message would always recommend
'switch to mlx-audio' even when the user's configured MLX model can't
clone, sending them in a circle back to the same error.

isinstance(value, bool) distinguishes a resolved boolean from a
descriptor object, so mlx-audio is excluded from the suggestion list
until its actual per-instance capability can be checked (already handled
correctly by resolve_generation_backend()'s per-call instance check).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): engine-aware dub/batch entry (#987)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 03:42:14 +05:30
efe57ee49d fix(ui): loaded-models panel says when a resident model is not the active engine (#985)
* fix(ui): loaded-models panel says when a resident model is not the active engine (#TBD)

A switched-away TTS model stays resident in VRAM until Unload/Flush or the
idle evictor, so users who picked another engine (e.g. voxcpm2) saw
"OmniVoice TTS - cuda:0 - 1937 MB" in the LOADED MODELS flyout and concluded
synthesis was still routing to OmniVoice. It wasn't - the panel just gave no
hint that resident != active.

/model/loaded entries for TTS-family models (in-process OmniVoice +
subprocess sidecars) now carry engine_id + is_active_engine, computed against
active_backend_id(); attribution failure degrades to the old shape
(is_active_engine: null) and non-TTS entries (ASR, diarization) are left
unannotated. The flyout renders a muted "not active - safe to unload" tag
(i18n: header.model_not_active, en + zh-CN) on inactive entries; Unload/Flush
behavior is unchanged. Regression tests cover both attribution states, the
ASR non-label, and the degradation path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): loaded-models active-engine hint entry (#985)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(mm2): string-target monkeypatch for active_backend_id — immune to sys.modules reimports

The two attribution tests patched the collection-time module alias; other
suites pop+reimport services.* modules mid-run, so in full-suite order the
patch landed on a stale module object while _active_tts_id late-imported the
fresh one (CI-only failure). String targets resolve at patch time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:41:47 +05:30
7f77f4d7bf fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#986)
* fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#TBD)

Field report (Discord): baked-in echo/reverb on some voices. apply_mastering()
hardcoded a Reverb that ran on every non-raw synthesis before the user's
preset chain — broadcast shipped reverb it never declared, podcast broke its
"no reverb" promise, cinematic/warm got doubled reverb.

The mastering pre-stage is now data-driven (MASTERING_CHAIN: highpass +
compressor, same params as before) and reverb-free; cinematic/warm keep their
user-chosen reverb. Regression tests pin the contract, incl. a burst-then-
silence echo-tail check and pedalboard-missing passthrough.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): hidden mastering reverb entry (#986)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:26:19 +05:30
087309259b fix(setup): first-run network check is mirror-aware and never hard-blocks (#984)
* fix(setup): first-run network check is mirror-aware and never hard-blocks

Field report (Discord, China): the Launchpad preflight probed hardcoded
huggingface.co:443 and any failure disabled Continue outright — users behind
the GFW were stuck on the very first screen, before Settings (and its
HF mirror quick-pick) was even reachable.

- The probe now targets the HF endpoint actually in effect (HF_ENDPOINT /
  hf_endpoint pref via configured_hf_mirror), with the real port.
- An unreachable endpoint is a WARNING, not a blocker: local-first — cached
  models work offline, and downloads surface their own actionable errors.
- When huggingface.co is blocked but hf-mirror.com answers, the fix text says
  exactly that, and the wizard shows an inline mirror quick-pick (presets +
  custom URL) that applies via PUT /hf-mirror — effective immediately for
  downloads — then re-checks.
- Docs updated (downloading-models, install troubleshooting); regression
  tests cover warn-not-fail, mirror-host probing, and the mirror suggestion.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(changelog): open [Unreleased] with the preflight mirror fix (#984)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:11:18 +05:30
94 changed files with 4374 additions and 435 deletions
+1 -1
View File
@@ -149,7 +149,7 @@ jobs:
- os: windows-2022
label: Windows
rust_target: x86_64-pc-windows-msvc
- os: ubuntu-22.04
- os: ubuntu-24.04
label: Linux
rust_target: x86_64-unknown-linux-gnu
runs-on: ${{ matrix.os }}
+13 -2
View File
@@ -213,13 +213,24 @@ jobs:
bundles: "msi,updater"
# Linux: ship .AppImage only. AppImage is universal (no distro
# package-manager dep), runs on any glibc-2.31+ host, and is the
# package-manager dep), runs on any glibc-2.39+ host, and is the
# Linux auto-update target. The .deb target was dropped: tauri-bundler
# fails it with "Failed to create control scripts: No such file or
# directory" (no custom deb config of ours is at fault) — revisit on a
# tauri-cli bump. FUSE unavailability on GH runners is handled via
# APPIMAGE_EXTRACT_AND_RUN=1.
- os: ubuntu-22.04
#
# Bumped from ubuntu-22.04 → ubuntu-24.04 (#961): the AppImage
# bundles whatever `libwebkit2gtk-4.1-dev` the build runner's apt
# repos resolve (see the "Linux system deps" step below) — 22.04's
# was meaningfully stale relative to what current Ubuntu/Fedora
# ship, and AppRun's LD_LIBRARY_PATH makes that bundled, stale copy
# take priority over a healthy system WebKitGTK at runtime. Raises
# the AppImage's glibc floor from 2.35 to 2.39 — pre-2022 distros
# (Ubuntu <22.04, Debian <12) lose support; no report of anyone on
# something that old has come in, and the project's own install
# docs already assume Debian 12 / Ubuntu 22.04+.
- os: ubuntu-24.04
arch: x86_64-unknown-linux-gnu
label: "Linux x64"
rust_target: x86_64-unknown-linux-gnu
+74
View File
@@ -6,6 +6,80 @@ The format is loosely based on [Keep a Changelog](https://keepachangelog.com/).
Versions track the desktop app (`tauri.conf.json` + `frontend/src-tauri/Cargo.toml`).
The bundled TTS model package (`pyproject.toml`) is versioned independently.
## [Unreleased]
## [0.3.14] — 2026-07-09
A fast follow to v0.3.13: **every engine family now has a visible picker.** Settings → Engines showed only a TTS table, with the ASR and LLM pickers hidden behind a low-discoverability tab — so the 10 transcription engines (including the new OpenAI-compatible backend) looked unswitchable without env vars. Now all three families get their own table. Also in: the Linux AppImage's white-screen auto-workaround now checks the WebKitGTK it actually ships (not whatever your system reports), and installing to a different drive on Windows is properly documented.
### Added
- **ASR engines get the same Settings picker TTS has.** Settings → Engines now shows a visible picker table per family — TTS, ASR, and LLM — instead of a single TTS-titled table with the other families tucked behind a tab (README even promised a Settings ASR picker that didn't exist). The OpenAI-compatible backend and the 9 local ASR engines become selectable with one click, no env vars needed; an explicit `OMNIVOICE_ASR_BACKEND` still wins over the Settings pick, so pinned setups behave exactly as before. (no issue — UX gap found during #877)
### Fixed
- **The Linux AppImage's white-screen auto-workaround now checks the right WebKitGTK.** The launcher decided whether to apply the compositing workaround by asking the *system's* `pkg-config` — but the version that actually runs is the *bundled* one, which the AppImage prioritizes. On any machine where the two diverge (e.g. building from source with newer dev packages installed), the detection read the wrong number and could skip a workaround the running library needed. The build now stamps the bundled version into the AppImage at package time, and the launcher reads that stamp — correct by construction. The launcher's shell tests also now run in CI, which they previously never did. (#961 follow-up)
### Docs
- **Windows: installing to a different drive is documented** — the wizard's directory picker works for any local drive; mapped network drives are a Windows Installer limitation (not installable-to by design); and the big data (models/voices) moves independently via Settings → Storage or Portable mode. (#938)
## [0.3.13] — 2026-07-09
The community-fixes release. Two contributors didn't just report bugs — they diagnosed them to the exact line and submitted the fixes that shipped: **voice cloning on mlx-audio's CSM model works for the first time**, and **macOS live recording finally gets its microphone permission prompt** (both @MahdiHedhli). A third reporter's A/B analysis fixed **cross-language dubs speaking the wrong language**. On top of that: a backend shutdown race that produced confusing crash-on-quit reports is fixed, the Linux AppImage stops shipping a stale WebKitGTK that white-screened current distros, and a new OpenAI-compatible transcription backend opens a path to Qwen3-ASR today. Thank you to everyone who filed, diagnosed, and contributed — this release is mostly yours.
### Added
- **A path to Qwen3-ASR today: generic OpenAI-compatible transcription.** The direct integration is still blocked on `transformers>=5.13` stabilizing upstream, but a community member proposed splitting the work — add a backend that talks to any OpenAI-compatible transcription server right now. Point OmniVoice at a self-hosted Qwen3-ASR/FunASR/SenseVoice server, or OpenAI's own API, configured in Settings → Models. No install; audio does leave your machine to whichever server you configure, unlike every other ASR engine. (#877)
### Fixed
- **The Linux AppImage no longer white-screens on current distros with a healthy system WebKitGTK.** The release build ran on an older CI base image, and the resulting AppImage bundles whatever `libwebkit2gtk` that image's apt repos resolve — which the AppImage's own `LD_LIBRARY_PATH` then prioritizes over your system's newer, healthy copy at runtime. A from-source build (which links straight against your system library) worked fine on the exact same machine where the shipped AppImage didn't — that split was the tell. Bumped the release build to a current Ubuntu LTS. Raises the AppImage's minimum host to glibc 2.39 (Ubuntu 24.04+); no reports from anyone on an older distro. (#961)
- **Backend shutdown no longer races a still-loading model, surfacing a confusing crash on restart.** Quitting the app while a model was still loading in the background let shutdown report itself "done" while a background thread was still mid-import; tearing the process down under that thread produced a misleading error (a generic transformers import-failure message, unrelated to the real cause) that looked like a real crash rather than a timing issue. All background tasks are now properly cancelled and awaited before shutdown proceeds. (#1000, likely the same class behind #941 and #979)
- **Cross-language dub no longer speaks the source-language reference line verbatim.** Auto-generated speaker clones pair an audio slice with the ASR segment's own text field, assuming the two agree — but ASR segment text and its timestamps routinely drift (a trailing word audible in the clip but missing from the text, or vice versa). A mismatched (reference audio, reference text) pair breaks zero-shot TTS prompt priming badly enough that the clone can emit the reference text itself instead of the target-language line it was asked to speak. Each reference clip is now re-transcribed after it's written, so the pair matches by construction — reported with an exceptionally clear root-cause diagnosis and a working A/B repro. (#1004)
- **Voice Gallery errors now say what actually went wrong.** "Use voice", "Preview", search, upload, save, delete, and trim in the Gallery all showed the same hardcoded guess ("the engine may be loading") on ANY failure — a 500, a validation error, a genuinely unrelated bug — discarding the real, already-clean backend error message in the process. Every one of those now shows the actual error.
- **Voice cloning on mlx-audio's CSM model no longer crashes with an opaque "list index out of range".** `MLXAudioBackend.generate()` read `voice`/`ref_audio`/`language`/`speed` from its kwargs but silently dropped `ref_text` — CSM only builds its cloning context when both `ref_audio` and `ref_text` are present, so cloning on this engine could never have worked as shipped. Reported with the exact root cause and a working fix. (#1012, #1013)
- **A dub segment's free-text style tags no longer 400 the segment preview.** A validator-safe instruct builder already keeps Studio and Clone generation from round-tripping a 400 on unsupported free-text (a preset's raw attrs, an old profile's stray descriptive phrase) — but the Dub tab's segment preview, and saving a profile from a clone or from history, built their instruct strings directly and skipped it. Same guard now applies everywhere an instruct string is sent. (#1010)
- **The dub editor's play button no longer sticks permanently disabled after an audio-decode hiccup.** When the initial WaveSurfer decode fails, the timeline falls back to loading pre-computed peaks — the waveform draws fine, but the button's enabled state only relied on the `ready` event firing again for that recovery load, which it didn't reliably do. Each fallback path now confirms readiness explicitly once it settles.
- **macOS: live recording finally works — the microphone permission prompt now actually appears.** The app never showed up in System Settings → Privacy & Security → Microphone because macOS never saw a legitimate request: Tauri enables Hardened Runtime by default, which blocks microphone hardware access unless the matching entitlement is in the signed bundle — and it wasn't. Diagnosed to the exact mechanism and fixed by a community contributor (@MahdiHedhli), who also corrected our initial mis-read of this as an upstream WebKit limitation. (#1013, #1016)
- **Quitting during a slow model load waits longer before giving up.** A post-merge code review of the shutdown-race fix flagged that its 3-second wait could still be outrun by a cold model import on a slow disk, reproducing the original confusing-crash-on-quit in rare cases. The wait is now 20 seconds — imperceptible on a normal quit (tasks finish or cancel in milliseconds), only felt in the exact case it protects. (#1020)
### Changed
- **Removed the donate heart from the nav rail.** Support OmniVoice is still one click away from Settings and the Contact page.
### CI
- **The "flaky trio" is root-caused and neutralized.** Three tests failed intermittently on CI — never locally — across unrelated PRs, costing a re-run each time. Cause: a leaked half-precision torch default from some earlier test in CI's ordering (the giveaway: a failing assertion's observed value was exactly float16(0.1)). An autouse test-suite guard now resets the leak between tests and names the offending test in CI output when it fires. (#1021)
## [0.3.12] — 2026-07-08
A community-issue sweep — nineteen open reports triaged in one pass, most fixed same-day. The through-line: **your active engine selection is now honored everywhere** (dubbing, batch, and — new in this release — MLX-Audio's own curated models are finally selectable instead of always silently defaulting to Kokoro), **first-run stops dead-ending users on restricted networks or behind corporate TLS proxies**, and a run of sharp community diagnoses (a one-line ROCm index fix, a Windows-only focus-stealing bug, a genuine crash regression) got fixed largely because reporters did the hard diagnostic work themselves. Thank you.
### Added
- **MLX-Audio's other 6 curated models are finally selectable.** The engine multiplexes Kokoro, CSM, Qwen3-TTS, Dia, Chatterbox, MeloTTS, and OuteTTS, but there was no way anywhere in the UI or API to pick which one loads — downloading a model via Settings → Models did nothing, since the backend always defaulted to Kokoro regardless. Settings → Engines now shows a model picker on the mlx-audio row; switching takes effect immediately, no restart needed. (#981)
### Fixed
- **First-run no longer dead-ends behind restricted networks (e.g. China).** The system check probed hardcoded huggingface.co, and any failure locked the Continue button — users behind the Great Firewall were stuck on the very first screen, even when they had already configured a working mirror. The check now probes the Hugging Face endpoint actually in effect, an unreachable endpoint is a warning instead of a blocker (models already on disk keep working offline), and when huggingface.co is blocked but the hf-mirror.com community mirror answers, the wizard says so and offers a one-click mirror switch right on the check screen — no restart needed. (#984)
- **Installs behind a corporate or antivirus TLS-inspecting proxy no longer fail with a raw SSL error.** `SSLV3_ALERT_HANDSHAKE_FAILURE` happens when a proxy re-signs HTTPS traffic with a root CA your OS trusts but Python's bundled certificate list doesn't — a different failure mode from the network-blocking case above. OmniVoice now trusts your OS's certificate store directly, which should resolve the handshake outright rather than just explain it better. (#976)
- **The loaded-models panel now says when a resident model is not your active engine.** Switching TTS engines keeps the previous model in VRAM (so switching back is instant) — but the panel showed it with no context, so "OmniVoice TTS — 1.9 GB" after selecting VoxCPM2 looked like the selection was ignored. A field report confirmed the confusion. Resident-but-inactive models are now tagged "not active — safe to unload", and the API self-describes each entry's engine. (#985)
- **Voices no longer ship with a hidden echo.** Every non-raw synthesis was getting a small room reverb baked in by the mastering pre-stage — on top of whatever effect preset you chose, so even "Podcast" (which promises *no reverb*) had some, and Cinematic/Warm got it twice. A field report ("a lot of echo/reverb on some of the voices") led straight to it. The mastering stage is now highpass + compressor only; reverb happens only when a preset explicitly declares it. Also documented: cloned voices reproduce the reference clip's room acoustics — dry, close-mic references clone cleanest. (#986)
- **Your engine selection now actually applies to Dubbing and Batch TTS.** Both hardcoded OmniVoice regardless of what was picked in Settings → Engines — pick VoxCPM2, dub anyway with OmniVoice, no error. Both now resolve the active engine up front; an engine that can't clone from reference audio (KittenTTS, Sherpa-ONNX, Supertonic 3 — fixed preset voices only) fails the job immediately with a clear message naming which engines do support it, instead of silently substituting OmniVoice or mis-cloning every speaker into one voice. Batch only requires cloning when a specific voice is pinned — an unpinned batch job runs on any engine. (#987)
- **AMD ROCm torch install no longer silently falls back to CPU.** A community member (Kaihui-AMD) diagnosed it precisely: the ROCm wheel index we pointed at tops out at PyTorch 2.5.1, but the app pins `torch==2.8.0` — the reinstall was unsatisfiable and silently kept the default CUDA build, which runs on CPU on an AMD GPU. Bumped the default index to one that actually carries the pinned version. (#972)
- **mlx-audio no longer crashes on unsupported languages.** Selecting a language like Dutch, Spanish, or Portuguese with mlx-audio's Kokoro model crashed with a raw, unreadable internal-details dump instead of a real error — the code was guessing an ISO language code by truncating the language name, which only worked by coincidence for a few languages. Unsupported languages now fail cleanly with a message naming what's actually supported, and no engine can leak a raw crash-internals dump into an error message again. (#977)
- **The voice-design panel no longer crashes on certain saved voice profiles.** A genuine regression: an earlier translation fix accidentally introduced a crash when a saved design profile's data was incomplete (possible from an older app version or a partial save). Fixed at every layer — the render no longer crashes, both places that restore saved data complete it first, and profiles can no longer be *saved* with incomplete data in the first place. (#983)
- **Windows: the dictation pill no longer steals focus.** Pressing the dictation shortcut activated the pill window, which meant the auto-paste landed back in OmniVoice instead of whatever app you were dictating into, and the pill would get stuck on screen. Precisely diagnosed by a community reporter; fixed to match how this already worked on macOS. (#982)
- **The nemo-parakeet ASR engine's install hint no longer breaks your backend.** Following the in-app "pip install nemo_toolkit[asr]" instruction silently downgraded core packages your backend needs to start — the install reported success, and the breakage only showed up on the next restart. The hint now says plainly that this isn't safe to install into the shared environment. (#974)
- **A stuck generate now tells you the actual fix.** When a job times out from GPU/VRAM contention, the error explained why but never mentioned Flush/Unload — the one action that actually resolves it, and one the sibling ASR-timeout error already recommended. (#939)
### Changed
- **README and Linux docs no longer advertise a `.deb` package that isn't published.** `.deb` bundling is disabled in the release pipeline pending a tauri-cli fix; the docs now say so honestly instead of pointing at a file that was never in any release. (#961 investigation, #990)
- **Linux install docs mention `yt-dlp` as an optional prerequisite** — previously only surfaced via an in-app warning after the fact. (#973)
- **A benign Tauri startup warning no longer looks like an app problem.** On some Windows configurations, Tauri's own internal IPC fallback logs a warning that's fully harmless (it silently and successfully falls back to another transport) — it was spuriously flipping the Settings → Logs footer to show "1 warning" on every launch. Filtered out of the diagnostic capture. (#975)
## [0.3.11] — 2026-07-05
The multi-language release — dubbing into several languages at once is finally a mature, honest workflow: **"Generate N dubs" now translates each language before rendering it** (with visible per-language progress), **switching languages never destroys your work** (every track keeps its own text, subtitles, and audio cache), completed tracks always show their tabs, and dialogue stops starting seconds early because of footsteps — a community reporter's theory, confirmed exactly. Around it, a reliability sweep driven by same-day field reports: your LLM provider finally survives a restart, SOCKS-proxy users can synthesize again (installed models now load without touching the network at all), timeline boxes are visible on every WebView2 runtime, running from source works again — and when the backend crashes, **it now tells you the exit code and attaches the evidence to your bug report automatically**.
+13 -174
View File
@@ -3,7 +3,7 @@
**OmniVoice Studio**
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The latest stable release is **v0.3.5**; `main` rolls ahead at **v0.3.6** (latest release + 1 patch — see the Versioning rule below).
OmniVoice Studio is an open-source, fully-local ElevenLabs alternative — a desktop app for voice cloning, voice design, video dubbing, and real-time dictation across 646 languages. It runs entirely on the user's machine (CUDA/MPS/ROCm/CPU auto-detect), with no API keys, no accounts, and no cloud dependencies. It's an active beta with a growing user base who hit it with real workloads (50-video batches, multi-engine setups, edge-OS platforms) and report friction in GitHub Issues and Discord. The current version lives in `frontend/package.json` (the single source of truth — see Versioning); the latest stable tag is on the [Releases page](https://github.com/debpalash/OmniVoice-Studio/releases/latest). With `AUTO_VERSION_BUMP` off (the current owner setting), `main` holds at the released version between releases.
**Core Value:** **A first-run that actually works.** A user who downloads the installer (or clones the repo) should reach a working voice-cloning or dubbing output without hitting a wall — and when something does go wrong, the error or docs should tell them exactly what to do.
@@ -16,175 +16,21 @@ Everything else (new engines, fancy features) is downstream of "the thing instal
- **Default features must work on every platform (strict rule, 2026-05-20):** A feature that ships in default mode — out-of-the-box, no user customization, no opt-in toggle — must behave identically on macOS, Windows, and Linux. Platform-specific *implementation code* is allowed for OS APIs / shells / packaging, but the user-visible *default behavior* cannot diverge. Platform-only features (e.g., a macOS-only global shortcut, a Windows-only path picker) must go behind explicit user opt-in: Settings toggle, env var, or CLI flag. When a default doesn't work on a platform, that's a P0 bug — either fix it on the missing platform or move it behind opt-in. No third option.
- **Backward-compatible project data**: Existing `omnivoice_data/` (user voices, projects, settings) must keep working without manual migration. Any DB schema change goes through alembic with a tested upgrade path.
- **Local-first guarantee preserved**: Auto bug reporting (new addition) must be **opt-in**, must submit only to GitHub Issues (no third-party telemetry endpoint), and the app must remain fully functional with reporting disabled. No required cloud calls, accounts, or API keys.
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release (currently **v0.3.5**). ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
- **Beta release cadence (no RC, no ceremony — strict rule, 2026-05-20):** the v0.3.x line has **no release candidates, no 48h soak, no formal release ceremony**. Every fix goes continuous-to-main; the owner tags a patch (`v0.3.Z`) from main whenever the current state is worth cutting. No `-rc` tags. No phased release. No `v0.4` deferrals while the v0.3.x line is open — every open issue and every open community PR gets absorbed into the v0.3.x line or explicitly declined. Users follow `main` for previews; users wanting stable stay on the latest tagged release. ROADMAP.md's Phase 6 "Release/Verify/Retro" entries are obsolete unless the user revives them.
<!-- GSD:project-end -->
<!-- GSD:stack-start source:research/STACK.md -->
## Technology Stack
## Recommended Stack — Per Capability
### Capability 1 — HuggingFace Token Persistence (issue #35)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `huggingface_hub` (already pinned transitively by `transformers>=5.3.0`) | `≥1.12.x` (latest 2026) | Auth + cache + token storage | Canonical, used by every HF library already in the stack. `HfFolder` is **superseded** in v1.x by the higher-level `login()` / `auth_list()` / `auth_switch()` API. |
| `keyring` (Python) | `≥25.x` | Optional OS-keychain backing | Only adopt if a future hardening pass wants Keychain/Credential-Manager/SecretService. **Not recommended for this milestone** — adds a native dep (`dbus`, `pywin32`) per platform with no real security win over `0600` file storage in `HF_HOME`. |
| Shell | One-liner to persist `HF_TOKEN` |
|-------|---------------------------------|
| macOS zsh (default since 10.15) | `echo 'export HF_TOKEN=hf_xxx' >> ~/.zshrc && source ~/.zshrc` |
| Linux bash | `echo 'export HF_TOKEN=hf_xxx' >> ~/.bashrc && source ~/.bashrc` |
| Windows PowerShell (user scope) | `[Environment]::SetEnvironmentVariable("HF_TOKEN","hf_xxx","User")` (new shells only) |
| Windows cmd | `setx HF_TOKEN "hf_xxx"` (user scope, new shells only) |
- [HF environment variables docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH confidence (official, current)
- [HF authentication API docs](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH confidence
- [Microsoft `setx` docs](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH confidence
### Capability 2 — In-App Structured Bug Reporting (opt-in, GitHub Issues)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| GitHub REST API `POST /repos/{owner}/{repo}/issues` | `2026-03-10` API version | Server-side issue creation | Official, stable. Requires auth. |
| **Prefilled-URL pattern** (`github.com/{owner}/{repo}/issues/new?title=…&body=…&labels=…`) | n/a | Zero-auth fallback | **This is the recommended primary path for v0.3.x.** No token needed, no GitHub App registration needed, user's browser opens with a prefilled form, they review and click Submit. They own the issue, the OSS project gets the report, and OmniVoice never holds a credential. |
| `gh-app-jwt` + GitHub App (Rust crate `octocrab` or Python `pygithub`) | only if we later want fully-automated submission | Programmatic posting under an app identity | **Defer to a later milestone.** Requires registering a public GitHub App, hosting a token-exchange endpoint, and managing rate-limit quotas — disproportionate for stabilization scope. |
| `platform`, `psutil`, `torch.cuda` (already in deps) | already pinned | Capture OS, CPU/GPU/VRAM info | No new deps. |
| `httpx` (already in `dev-dependencies`, promote to runtime if needed) | `≥0.28.1` | HTTP for the API call path (if/when we add auth) | Modern async-first, already used in test suite. |
- ✓ No token storage in OmniVoice → no security surface
- ✓ Opt-in by definition (user has to click Submit on github.com)
- ✓ User owns the issue → can be replied to, edited, closed by them
- ✓ Zero infra cost — no proxy, no app, no rate-limit management
- ✓ Works identically on macOS / Windows / Linux via Tauri's `shell.open`
- ✓ Survives our project being forked (just change the URL)
- OS name + version (`platform.platform()`)
- Python version (`sys.version`)
- OmniVoice version (`pyproject.toml`)
- Backend git SHA (if installed from source) or installer build ID
- CPU model, RAM (`psutil.cpu_count()`, `psutil.virtual_memory()`)
- GPU vendor/model/VRAM (`torch.cuda.get_device_name()`, `torch.cuda.mem_get_info()`, MPS detect)
- Active TTS engine + list of installed engines
- Frontend: bun version, OS shell
- Last error message + stack trace if launched from an error toast
- Audio file contents (privacy — reference samples may contain user's voice)
- File paths containing `/Users/<name>/` (strip home dir → `~/`)
- HF token, OpenAI keys, any env var matching `*TOKEN*|*KEY*|*SECRET*`
- [GitHub URL query parameters for issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/creating-an-issue#creating-an-issue-from-a-url-query) — HIGH confidence
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (widely used reference impl)
- [GitHub REST API: Create an issue](https://docs.github.com/en/rest/issues/issues#create-an-issue) — HIGH confidence (for the future auto-submit path)
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — reviewed, **rejected for milestone** due to local-first constraint
### Capability 3 — `uv venv` Mirror Fallback for Restricted Networks (issues #57, #60)
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `uv` (already used) | `≥0.5.x` | Python+venv bootstrap | Existing dep. |
| `UV_PYTHON_INSTALL_MIRROR` env var | uv `0.4.x`+ | Override python-build-standalone download URL | **Official, current.** Replaces `https://github.com/astral-sh/python-build-standalone/releases/download/...` in download URL construction. No built-in fallback if mirror fails. |
| `UV_PYTHON_PREFERENCE=only-system` (or CLI flag `--python-preference only-system`) | uv `0.4.x`+ | Skip the python-build-standalone download entirely; use the user's system Python | **The reliable escape hatch** when no mirror works. Requires a compatible Python `>=3.11` to already be on PATH. |
| `UV_HTTP_TIMEOUT`, `UV_HTTP_CONNECT_TIMEOUT`, `UV_HTTP_RETRIES` | uv `0.4.x`+ | Tune retry behavior for flaky links | Defaults are 30s / 10s / 3 — bump to 120s / 30s / 5 for restricted networks. |
# Pseudocode for the bootstrap
# Final fallback: don't download Python at all
- `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (Tsinghua — fastest in China)
- `UV_DEFAULT_INDEX=https://mirrors.aliyun.com/pypi/simple` (Aliyun fallback)
- Russia: no major government-blessed PyPI mirror; users typically tunnel via VPN. Document this honestly rather than ship a broken default.
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (official)
- [uv issue #5224 — python-build-standalone mirror support](https://github.com/astral-sh/uv/issues/5224) — HIGH (the feature was added)
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms real user pain, no built-in fallback)
- [uv python-versions concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH (documents `python-preference` semantics)
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained mirror list; verify each URL still works before shipping)
### Capability 4 — Supertonic-3 TTS Engine
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| `supertonic` (PyPI) | `1.3.1` (latest, May 18 2026 — Phase 3 Wave 1 to verify constructor signature before bump) | Official Supertonic-3 inference SDK | Authoritative wrapper from Supertone Inc. Wraps the ONNX session orchestration so we don't have to. |
| `onnxruntime` | `≥1.17.x` (any recent) | ONNX inference runtime | Already a transitive dep of WhisperX (via CTranslate2 path is separate, but `onnxruntime` itself ships for kittentts and audioseal). Verify with `uv tree` after adding — should resolve cleanly. |
| `huggingface_hub` (already pinned) | `≥1.12.x` | Model weight download (~400 MB on first use) | Reuses existing HF token + cache infrastructure. The user's existing `HF_TOKEN` (Capability 1) works for the Supertonic model download too. |
| `numpy`, `soundfile` (already pinned) | already pinned | Audio I/O + array math | No new deps. |
- `text_encoder.onnx`
- `latent_denoiser.onnx`
- `voice_decoder.onnx`
- 44.1 kHz sample rate, 24-dim latent, 128-dim style
- ~99M parameters total
- Tokenizer: `AutoTokenizer.from_pretrained(model_path)` — loads from `tokenizer.json` shipped with model
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official)
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official)
- [supertonic PyPI page](https://pypi.org/project/supertonic/) — HIGH (`1.3.1` confirmed 2026-05-18; same publisher, MIT, same 4 deps)
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure details)
### Capability 5 — Cross-Platform Documentation Tooling
| Technology | Version | Purpose | Why Recommended |
|------------|---------|---------|-----------------|
| Plain Markdown in `docs/` + GitHub-rendered (current state) | n/a | Install tutorial, troubleshooting | Zero new infra. Renders inline on GitHub for issue-replies. No build step to break. |
| Existing `scripts/smoke-test.sh` + Playwright `tests/` (already in `package.json`) | already pinned | Verify install paths actually work | **This is the real solution to "docs drift."** If smoke-test exercises the install path described in docs, docs that drift will break CI. |
| **Future** (defer): Astro Starlight | `≥0.30` | Standalone docs site at `docs.omnivoice.studio` | Adopt only when docs exceed ~20 markdown files and need search/versioning. Tauri, the framework OmniVoice already depends on, uses Starlight — well-traveled choice. Material for MkDocs entered maintenance mode in November 2025 per Docsio's 2026 review — **avoid** for new docs. |
| Project | What they do |
|---------|--------------|
| **OBS Studio** | Docs at `obsproject.com/docs` (Sphinx, separate repo). Install paths in README, wiki for community-contributed. CI doesn't gate on docs drift. |
| **Audacity** | Manual at `manual.audacityteam.org` (MediaWiki). README is minimal. Install path = "use the installer." No automated sync. |
| **Tauri** | Docs at `v2.tauri.app` (Astro Starlight, separate repo `tauri-apps/tauri-docs`). README is minimal. Heavy reliance on community contributions and PR review. |
| **VS Code** | Docs at `code.visualstudio.com/docs` (separate repo, Markdown). README is minimal. Manual sync; docs team is staffed. |
- [Tauri docs (Astro Starlight)](https://github.com/tauri-apps/tauri-docs) — HIGH (reference for "if we ever move off README")
- [OBS Studio docs](https://docs.obsproject.com/) — HIGH (Sphinx, separate site reference)
- [Audacity Manual](https://manual.audacityteam.org/) — HIGH (MediaWiki reference)
- [Docsio: Material for MkDocs 2026 review (maintenance mode)](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with project's own GitHub activity)
- [Docsio: Starlight 2026 review](https://docsio.co/blog/starlight-docs) — MEDIUM
## Installation
# No new Python dependencies needed for Capabilities 1, 2, 3, 5.
# Only Capability 4 adds a runtime dep:
# Verify no regressions:
# Should show single versions of each; no duplicates.
## Alternatives Considered
| Recommended | Alternative | When to Use Alternative |
|-------------|-------------|-------------------------|
| HF token via in-app Settings → `huggingface_hub.login()` | OS keyring via `keyring` package | Only if a security hardening milestone later demands OS-native credential storage. Not worth the cross-platform native-dep cost for v0.3.x. |
| Prefilled-URL GitHub Issues | GitHub App + device flow + authenticated POST | When milestone budget can afford registering a public GitHub App and hosting a token-exchange function. Defer. |
| Prefilled-URL GitHub Issues | Sentry / `sentry-tauri` | Never — violates the "no third-party telemetry endpoint" constraint in PROJECT.md. |
| `UV_PYTHON_INSTALL_MIRROR` chain + `only-system` fallback | Bundle Python in the Tauri installer | Adds ~30 MB to every installer for ~5% of users. Revisit if the bootstrap is still a top complaint in v0.4. |
| In-repo Markdown docs | Astro Starlight standalone site | When docs grow past ~20 pages and need full-text search. Tauri provides a precedent if/when we get there. |
| In-repo Markdown docs | MkDocs / Material for MkDocs | **Avoid** for new sites — Material for MkDocs is in maintenance mode as of Nov 2025. |
## What NOT to Use
| Avoid | Why | Use Instead |
|-------|-----|-------------|
| `HfFolder.save_token()` directly | Older API; v1.x `login()` does the same plus git-credential integration and is the documented path | `huggingface_hub.login(token=val, add_to_git_credential=False)` |
| Setting `HF_TOKEN` via shell rc files as the *only* persistence mechanism | Different per OS, fragile, opaque to the user, breaks in installer-launched processes that don't source shell rc | Write to `$HF_HOME/token` via `login()`. Document env var as override only. |
| `setx` for HF token persistence | Doesn't propagate to current shell; common source of "I set it but it's empty" bug reports | `[Environment]::SetEnvironmentVariable(...,"User")` in PowerShell, or the in-app Settings field |
| PAT-based GitHub Issues posting from OmniVoice | Would require shipping or asking for a token; breaks local-first promise | Prefilled-URL pattern (user submits from their browser) |
| `sentry-tauri` for OmniVoice | Third-party telemetry endpoint — violates PROJECT.md constraint | Local-only `backend.log` rotation + opt-in prefilled-URL reporter |
| `hf_transfer` for downloads | Deprecated in favor of `hf-xet` per HF docs | Default `huggingface_hub` (uses `hf-xet` automatically when available) |
| `--python-preference managed` (default) without mirror config in restricted-network installers | Hits GitHub CDN, times out, user sees raw `uv` error | Configure `UV_PYTHON_INSTALL_MIRROR` + retry chain + `only-system` final fallback |
| Material for MkDocs as a *new* docs choice | Entered maintenance mode November 2025 | If docs site is eventually needed, use Astro Starlight (Tauri precedent) |
## Stack Patterns by Variant
- Set `UV_PYTHON_INSTALL_MIRROR` to one of the gh-proxy URLs at install time
- Set `UV_DEFAULT_INDEX=https://pypi.tuna.tsinghua.edu.cn/simple` (China) or document VPN requirement (Russia)
- Fall back to `UV_PYTHON_PREFERENCE=only-system` if all mirrors fail
- Increase `UV_HTTP_TIMEOUT=120`, `UV_HTTP_RETRIES=5`
- Default path: in-app Settings field → `login()` → file at `$HF_HOME/token`
- Power-user path: `export HF_TOKEN=...` in shell rc (documented but not promoted)
- Both paths are read at HF library import time; env var wins on conflict
- Default path: in-app "Report a bug" → prefilled GitHub Issues URL → user reviews + submits in browser
- All optional capture toggles default ON except "include reproduction file" (privacy)
- No path posts to any URL except `github.com/{owner}/{repo}/issues/new` (rendered locally as a URL, opened via `shell.open`)
- `uv add supertonic` → new TTSBackend subclass in `backend/services/tts_backend.py`
- Auto-detected and added to the engine picker in Settings
- ~400 MB model download on first synthesize call, cached in `$HF_HUB_CACHE`
- Existing IndexTTS/CosyVoice/etc. installs are untouched (no shared model weights)
## Version Compatibility
| Package A | Compatible With | Notes |
|-----------|-----------------|-------|
| `supertonic@1.3.1` | `onnxruntime>=1.17`, `numpy>=1.24`, `huggingface_hub>=0.20` | All deps already satisfied transitively by current `pyproject.toml`. |
| `huggingface_hub>=1.12` | `transformers>=5.3.0` (current pin) | `HfFolder` retained as deprecated alias; `login()`/`get_token()` are the canonical APIs. |
| `uv>=0.5` | `UV_PYTHON_INSTALL_MIRROR`, `UV_PYTHON_PREFERENCE` | Both env vars stable since uv 0.4.x. |
| Tauri v2 + `@tauri-apps/api/shell` | `shell.open()` for the prefilled-URL pattern | Already in the desktop app; no new permission needed beyond what the existing "open external link" plugin grants. |
## Sources
- [Hugging Face Hub environment variables](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables) — HIGH (verified against v1.12.1 docs, current 2026)
- [Hugging Face Hub authentication API](https://huggingface.co/docs/huggingface_hub/en/package_reference/authentication) — HIGH (verified `login()` is the canonical 1.x API)
- [Microsoft `setx` reference](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setx) — HIGH (confirms "current shell" gotcha)
- [PowerShell `about_Environment_Variables`](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables) — HIGH
- [uv environment variables reference](https://docs.astral.sh/uv/reference/environment/) — HIGH (verified all mirror + retry env vars)
- [uv issue #5224 — python-build-standalone mirror](https://github.com/astral-sh/uv/issues/5224) — HIGH (feature shipped)
- [uv issue #14187 — venv on Chinese network](https://github.com/astral-sh/uv/issues/14187) — HIGH (confirms user pain, justifies fallback chain)
- [uv `python-preference` semantics](https://github.com/astral-sh/uv/blob/main/docs/concepts/python-versions.md) — HIGH
- [Supertone/supertonic-3 model card](https://huggingface.co/Supertone/supertonic-3) — HIGH (official, 99M params, 31 languages, OpenRAIL-M)
- [supertone-inc/supertonic GitHub](https://github.com/supertone-inc/supertonic) — HIGH (official inference API)
- [supertonic 1.3.1 on PyPI](https://pypi.org/project/supertonic/) — HIGH (released 2026-05-18, MIT code license; bumped from 1.2.3 after Phase 3 research)
- [onnx-community/Supertonic-TTS-ONNX](https://huggingface.co/onnx-community/Supertonic-TTS-ONNX) — HIGH (ONNX file structure)
- [GitHub Docs: Authenticating to the REST API](https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api) — HIGH
- [GitHub Docs: Generating a user access token for a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) — HIGH (device flow reference)
- [sindresorhus/new-github-issue-url](https://github.com/sindresorhus/new-github-issue-url) — HIGH (canonical prefilled-URL reference impl)
- [sentry-tauri](https://github.com/timfish/sentry-tauri) — MEDIUM (reviewed, rejected on PROJECT.md constraint, not on quality)
- [dautovri/mirrors-china](https://github.com/dautovri/mirrors-china) — MEDIUM (community-maintained, verify URLs are still live before pinning in production)
- [Tauri 2 docs (Astro Starlight reference)](https://v2.tauri.app/) — HIGH (precedent for docs framework if we ever migrate)
- [Docsio: Material for MkDocs entered maintenance mode Nov 2025](https://docsio.co/blog/mkdocs-material) — MEDIUM (third-party review, but signal aligns with the project's own GitHub commit activity)
The May-2026 stack research that used to live here served five capabilities that have all since shipped (HF-token Settings panel, prefilled-URL bug reporting, uv mirror fallback for restricted networks, the Supertonic-3 engine, in-repo Markdown docs). Follow the patterns in the code itself; the durable *don'ts* that research established:
- **No third-party telemetry endpoints, ever** (`sentry-tauri` was evaluated and rejected) — bug reporting stays opt-in via prefilled GitHub-issue URLs.
- **No PAT/token-based GitHub posting from the app** — the user submits from their own browser.
- **Don't recommend `setx` for env vars on Windows** (silent truncation, no current-shell propagation) — use the in-app Settings panel or PowerShell `[Environment]::SetEnvironmentVariable`.
- **Don't adopt Material for MkDocs** for any future docs site (maintenance mode since Nov 2025) — Astro Starlight is the precedent if docs ever outgrow the repo.
- **`hf_transfer` is deprecated** — default `huggingface_hub` (hf-xet) handles downloads.
For anything new: prefer what's already pinned in `pyproject.toml` / `frontend/package.json`, and check `uv tree` for conflicts before adding a dependency.
<!-- GSD:stack-end -->
<!-- GSD:conventions-start source:CONVENTIONS.md -->
@@ -222,16 +68,9 @@ No project skills found. Add skills to any of: `.claude/skills/`, `.agents/skill
<!-- GSD:skills-end -->
<!-- GSD:workflow-start source:GSD defaults -->
## GSD Workflow Enforcement
## Workflow
Before using Edit, Write, or other file-changing tools, start work through a GSD command so planning artifacts and execution context stay in sync.
Use these entry points:
- `/gsd-quick` for small fixes, doc updates, and ad-hoc tasks
- `/gsd-debug` for investigation and bug fixing
- `/gsd-execute-phase` for planned phase work
Do not make direct repo edits outside a GSD workflow unless the user explicitly asks to bypass it.
Direct repo edits are authorized (owner decision, 2026-07-08). The GSD command gate that used to live here referenced `/gsd-quick` / `/gsd-debug` / `/gsd-execute-phase` skills that are not installed in this environment; the owner chose to keep working directly rather than restore them. The working conventions that matter are in **Conventions** above — versioning, docs-sync, changelog, localization, fix quality, keep-main-green — plus: gate every merge on the "Tests (backend + frontend)" check passing and the PR being MERGEABLE, and check the open-PR queue before implementing any community-reported fix (contributors may have already submitted one).
<!-- GSD:workflow-end -->
+7 -5
View File
@@ -190,7 +190,6 @@ The eight headliners — and twelve more waiting under the fold.
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/macOS-DMG_(Apple_Silicon)-000?style=for-the-badge&logo=apple&logoColor=white" alt="Download macOS DMG" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Windows-MSI_(x64)-0078D4?style=for-the-badge&logo=windows&logoColor=white" alt="Download Windows MSI" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Linux-AppImage_(x64)-FCC624?style=for-the-badge&logo=linux&logoColor=black" alt="Download Linux AppImage" /></a>
<a href="https://github.com/debpalash/OmniVoice-Studio/releases/latest"><img src="https://img.shields.io/badge/Debian-.deb-A81D33?style=for-the-badge&logo=debian&logoColor=white" alt="Download Debian .deb" /></a>
<br/>
<sub><b>macOS:</b> first launch needs a one-time approval — right-click → <b>Open</b> (or System Settings → Privacy &amp; Security → <b>"Open Anyway"</b> on macOS 15). No Terminal needed. <a href="docs/install/macos.md#gatekeeper-quarantine">Why?</a></sub>
<br/>
@@ -267,7 +266,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
| | **Minimum** | **Recommended** |
|---|---|---|
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 20.04+ | Any modern 64-bit OS |
| **OS** | Windows 10, macOS 12+ (Apple Silicon), Ubuntu 24.04+ (glibc 2.39+) | Any modern 64-bit OS |
| **RAM** | 8 GB | 16 GB+ |
| **VRAM (GPU)** | 4 GB (auto-offloads TTS to CPU) | 8 GB+ (NVIDIA RTX 3060+) |
| **Disk** | 10 GB free (models + cache) | 20 GB+ SSD |
@@ -287,7 +286,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
### 🗣️ TTS Engines
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var.
**14 engines, one picker.** OmniVoice (default, 600+ languages) is always available; CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS-Nano, KittenTTS, MLX-Audio, and Sherpa-ONNX are opt-in and auto-detected — plus six lazy-installed heavyweights (IndexTTS 2, OmniVoice GGUF, Supertonic 3, MOSS-TTS-v1.5, dots.tts, Confucius4-TTS). Switch in **Settings → TTS Engine** or via the `OMNIVOICE_TTS_BACKEND` env var — the selection applies everywhere synthesis happens: single-clip generation, Voice Cloning, Video Dubbing, and Batch TTS.
<details>
<summary><b>📊 The full matrix</b> — 14 engines × platform × clone/instruct × license</summary>
@@ -313,6 +312,8 @@ Professional-grade voice AI, minus the subscription and the cloud.
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
>
> **Clone** matters beyond single-clip generation: Video Dubbing (and any Batch job with a pinned voice) needs reference-audio cloning to preserve speaker identity, so picking a Clone-less engine (KittenTTS, Sherpa-ONNX, Supertonic 3) as the active engine fails those jobs up front with an actionable message instead of silently falling back to OmniVoice.
>
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
</details>
@@ -321,10 +322,10 @@ Professional-grade voice AI, minus the subscription and the cloud.
### 🎧 ASR Engines
**9 engines, all fully local** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → ASR Engine** or via the `OMNIVOICE_ASR_BACKEND` env var.
**10 engines** — they power dictation, video dubbing, and subtitles. **WhisperX** is the cross-platform default (~100 languages, word-level timing); the rest are opt-in and auto-detected. Switch in **Settings → Engines** (the ASR Engines table — same picker TTS has), or pin one with the `OMNIVOICE_ASR_BACKEND` env var (the env var wins over the Settings pick). Nine run fully on-device; one (OpenAI-compatible) is an optional remote client for pointing at Qwen3-ASR or another compatible server — see below.
<details>
<summary><b>📊 The full lineup</b> — 9 engines, what each is best at, and compute-type notes</summary>
<summary><b>📊 The full lineup</b> — 10 engines, what each is best at, and compute-type notes</summary>
<br/>
@@ -339,6 +340,7 @@ Professional-grade voice AI, minus the subscription and the cloud.
| **Moonshine** | `moonshine` | English | Edge / low-latency, ONNX |
| **FunASR** | `funasr` | 50+ | All-in-one multilingual — built-in VAD + inline speaker diarization (SenseVoice) |
| **sherpa-onnx** (live dictation) | `sherpa-onnx-asr` | 25 EU + 90+ | Live, faster-than-real-time dictation — small streaming/offline ONNX models (Parakeet TDT v3/v2, streaming Zipformer & Paraformer, Whisper Tiny), CPU, identical on macOS / Windows / Linux. Picked per-model in **Settings → Voice**. |
| **OpenAI-compatible** ⚠️ remote | `openai-compat-asr` | Server-dependent | A path to **Qwen3-ASR** today (self-hosted server, no transformers wait), any OpenAI-compatible transcription endpoint, or OpenAI's own API — no install, configure in **Settings → Models**. Audio leaves your machine to whatever server you point it at; see [docs/engines/openai-compatible-asr.md](docs/engines/openai-compatible-asr.md). |
> Whisper-family engines cover ~100 languages; **FunASR / SenseVoice** adds an all-in-one multilingual path with built-in voice-activity detection and inline speaker diarization. **sherpa-onnx** powers the live dictation model picker — you talk and text appears as you speak. Every engine runs on-device — no API keys, no cloud.
+19 -13
View File
@@ -179,6 +179,21 @@ async def _run_batch_pipeline(job_id: str, job: dict):
job["status"] = "failed"
return
# ── Engine resolution (issue #312 class) ────────────────────────────
# Batch used to hardcode OmniVoice via get_model() regardless of the
# engine selected in Settings → Engines. require_cloning only when a
# specific voice is pinned (job["voice_id"]) — an unpinned job is fine on
# any active engine. Resolved ONCE for the whole job (every language
# below shares the same active engine); an uncaught ValueError here
# propagates to _worker()'s existing except-Exception handling, which
# already records a structured job failure via core.failure.build_failure.
from services.tts_backend import resolve_generation_backend
backend = await resolve_generation_backend(
require_cloning=bool(job.get("voice_id")),
cloning_purpose="this batch job's pinned voice",
)
sr = backend.sample_rate
# ── 3. Translate + Generate per language ───────────────────────────
total_langs = len(langs)
outputs = {}
@@ -243,13 +258,10 @@ async def _run_batch_pipeline(job_id: str, job: dict):
total_segments=len(translated_segments),
)
from services.model_manager import get_model
from services.audio_dsp import apply_mastering, normalize_audio
from services.audio_io import atomic_save_wav
import torch
_model = await get_model()
sr = _model.sampling_rate
total_samples = int(duration * sr)
full_audio = torch.zeros(1, total_samples)
total_segs = len(translated_segments)
@@ -295,22 +307,16 @@ async def _run_batch_pipeline(job_id: str, job: dict):
ref_text = row.get("ref_text")
try:
audios = _model.generate(
audio_out = backend.generate(
text=text, language=lang,
ref_audio=ref_audio, ref_text=ref_text,
duration=dur, num_step=16,
guidance_scale=2.0, speed=1.0,
denoise=True, postprocess_output=True,
)
audio_out = audios[0]
# TODO(#312): this route runs the OmniVoice model directly (not the active
# backend), so VoxCPM2 never reaches it. When these routes become
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
mastered = apply_mastering(
audio_out,
sample_rate=sr,
)
return normalize_audio(mastered, target_dBFS=-2.0)
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=sr)
return normalize_audio(audio_out, target_dBFS=-2.0)
except Exception as e:
logger.warning("TTS failed for seg %d (lang=%s): %s", i, lang, e)
return torch.zeros(1, int(dur * sr))
+9
View File
@@ -1006,6 +1006,11 @@ async def dub_transcribe_stream(
clones = done.pop().result()
break
yield _sse_event("ping", {})
if clones:
from services.speaker_clone import refine_ref_texts
clones = await loop.run_in_executor(
_gpu_pool, lambda: refine_ref_texts(clones, _asr_backend),
)
# Wave 3.2: per-segment clone refs. Cut each long-enough segment's
# own reference from the vocals so the dub of each line matches the
# prosody of its source line. Short lines fall back to the
@@ -1025,6 +1030,10 @@ async def dub_transcribe_stream(
),
)
if seg_clones:
from services.speaker_clone import refine_ref_texts
seg_clones = await loop.run_in_executor(
_gpu_pool, lambda: refine_ref_texts(seg_clones, _asr_backend),
)
job["segment_clones"] = seg_clones
except Exception as e:
logger.warning("per-segment clone refs skipped: %s", e)
+55 -46
View File
@@ -11,7 +11,8 @@ from core.db import db_conn
from core.config import DUB_DIR, VOICES_DIR, dub_seg_path
from core.tasks import task_manager
from schemas.requests import DubRequest
from services.model_manager import get_model, _gpu_pool, run_on_gpu_pool_guarded
from services.model_manager import _gpu_pool, run_on_gpu_pool_guarded
from services.tts_backend import resolve_generation_backend
from services.audio_dsp import apply_mastering, normalize_audio, apply_effects_chain, get_effect_chain
from services.audio_io import atomic_save_wav, _safe_torchaudio_save
from services.ffmpeg_utils import (
@@ -156,7 +157,19 @@ async def dub_generate(job_id: str, req: DubRequest):
detail="This dub session has expired or was never created. Re-upload the video to start a new one.",
)
_model = await get_model()
# ── Engine resolution (issue #312 class) ────────────────────────────────
# Dub used to hardcode OmniVoice via get_model() regardless of the engine
# selected in Settings → Engines — a SILENT fallback. Every real dub
# segment's ref_audio resolves to either an auto:<speaker>/auto-seg:<id>
# clone cut from the source video or a saved voice-profile row (see
# `_gen` below), so require_cloning=True: an engine that can't clone
# would either mis-clone per segment or fail deep into the job. Checked
# ONCE here, before the streaming task starts, so a doomed job fails fast
# with one clear message instead of N per-segment ones.
try:
backend = await resolve_generation_backend(require_cloning=True)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
async def _stream(task_id):
total = len(req.segments)
@@ -343,7 +356,7 @@ async def dub_generate(job_id: str, req: DubRequest):
seg_duration = seg.end - seg.start
if seg_duration <= 0.05 or not seg.text.strip():
sr = _model.sampling_rate
sr = backend.sample_rate
# max(0, …): a zero/negative-duration slot must not feed a
# negative length to torch.zeros (raises) — _store_mix_wav
# turns the empty buffer into a harmless in-memory entry.
@@ -377,21 +390,21 @@ async def dub_generate(job_id: str, req: DubRequest):
try:
_t_cache_0 = time.perf_counter()
cached_wav, cached_sr = torchaudio.load(seg_wav_path)
if cached_sr != _model.sampling_rate:
if cached_sr != backend.sample_rate:
import torchaudio.functional as AF
cached_wav = AF.resample(cached_wav, cached_sr, _model.sampling_rate)
cached_wav = AF.resample(cached_wav, cached_sr, backend.sample_rate)
# Pad/trim to slot — except smart_fit, whose mix
# loop needs the natural-rate length to compute the
# audio/video split (the seg_wav_kind guard above
# guarantees these cached WAVs are natural-rate).
if strategy != "smart_fit":
target_samples = int(seg_duration * _model.sampling_rate)
target_samples = int(seg_duration * backend.sample_rate)
current_samples = cached_wav.shape[-1]
if target_samples > current_samples:
cached_wav = torch.nn.functional.pad(cached_wav, (0, target_samples - current_samples))
elif current_samples > target_samples:
cached_wav = cached_wav[..., :target_samples]
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, _model.sampling_rate, f"mix_{seg_id}"))
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, cached_wav, backend.sample_rate, f"mix_{seg_id}"))
try:
del cached_wav
except Exception:
@@ -404,7 +417,7 @@ async def dub_generate(job_id: str, req: DubRequest):
# Fall through to a silent placeholder if the cached WAV
# is broken — cleaner than aborting the whole mix.
yield f"data: {json.dumps({'type': 'warning', 'segment': i, 'message': f'cached seg lost, padding silence: {str(e)[:120]}'})}\n\n"
sr = _model.sampling_rate
sr = backend.sample_rate
silence = torch.zeros(1, max(0, int(seg_duration * sr)))
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, silence, sr, f"mix_{seg_id}"))
try:
@@ -488,25 +501,23 @@ async def dub_generate(job_id: str, req: DubRequest):
torch.manual_seed(used_seed)
try:
audios = _model.generate(
audio_out = backend.generate(
text=text, language=lang if lang != "Auto" else None,
ref_audio=ref_audio, ref_text=ref_text,
instruct=instruct_str if instruct_str else None,
duration=dur_s, num_step=nstep, guidance_scale=cfg,
speed=spd, denoise=True, postprocess_output=True,
)
audio_out = audios[0]
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
sr = backend.sample_rate
# Apply per-segment DSP effect preset (default: broadcast)
seg_effect_preset = effect_preset or "broadcast"
if seg_effect_preset == "raw":
return audio_out
# TODO(#312): this route runs the OmniVoice model directly (not the active
# backend), so VoxCPM2 never reaches it. When these routes become
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
mastered_audio = audio_out
if not getattr(backend, "applies_own_mastering", False):
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
effect_chain = get_effect_chain(seg_effect_preset)
if effect_chain:
mastered_audio = apply_effects_chain(
@@ -537,24 +548,22 @@ async def dub_generate(job_id: str, req: DubRequest):
nstep, retry_steps,
)
try:
audios = _model.generate(
audio_out = backend.generate(
text=text, language=lang if lang != "Auto" else None,
ref_audio=ref_audio, ref_text=ref_text,
instruct=instruct_str if instruct_str else None,
duration=dur_s, num_step=retry_steps, guidance_scale=cfg,
speed=spd, denoise=True, postprocess_output=True,
)
audio_out = audios[0]
sr = _model.sampling_rate if hasattr(_model, 'sampling_rate') else 24000
sr = backend.sample_rate
seg_effect_preset = effect_preset or "broadcast"
if seg_effect_preset == "raw":
return audio_out
# TODO(#312): this route runs the OmniVoice model directly (not the active
# backend), so VoxCPM2 never reaches it. When these routes become
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
mastered_audio = audio_out
if not getattr(backend, "applies_own_mastering", False):
mastered_audio = apply_mastering(audio_out, sample_rate=sr)
effect_chain = get_effect_chain(seg_effect_preset)
if effect_chain:
mastered_audio = apply_effects_chain(
@@ -641,7 +650,7 @@ async def dub_generate(job_id: str, req: DubRequest):
yield f"data: {json.dumps({'type': 'cancelled', 'segments_processed': i + 1})}\n\n"
return
target_samples = int(seg_duration * _model.sampling_rate)
target_samples = int(seg_duration * backend.sample_rate)
current_samples = audio_tensor.shape[-1]
if strategy == "strict_slot":
@@ -659,7 +668,7 @@ async def dub_generate(job_id: str, req: DubRequest):
# trim, slip, stretch the video, or split audio/video
# retiming (smart_fit) to accommodate it.
generated_dur = audio_tensor.shape[-1] / _model.sampling_rate
generated_dur = audio_tensor.shape[-1] / backend.sample_rate
sync_ratio = round(generated_dur / max(seg_duration, 0.01), 3)
sync_scores.append(sync_ratio)
@@ -683,20 +692,20 @@ async def dub_generate(job_id: str, req: DubRequest):
except Exception as e:
logger.debug("seg fingerprint skipped for %s: %s", seg_id, e)
_pending_seg_writes.append((i, _model.sampling_rate, seg_id, _seg_fp, _num_step))
_pending_seg_writes.append((i, backend.sample_rate, seg_id, _seg_fp, _num_step))
# RVC needs the WAV on disk, so write it immediately only
# when RVC is active (uncommon path).
if rvc_is_enabled():
seg_wav_path = _seg_lang_path(seg_id)
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
try:
await loop.run_in_executor(_gpu_pool, apply_rvc, seg_wav_path)
rvc_wav, rvc_sr = torchaudio.load(seg_wav_path)
if rvc_sr == _model.sampling_rate:
if rvc_sr == backend.sample_rate:
audio_tensor = rvc_wav
target_samples = int(seg_duration * _model.sampling_rate)
target_samples = int(seg_duration * backend.sample_rate)
current_samples = audio_tensor.shape[-1]
if target_samples > current_samples:
audio_tensor = torch.nn.functional.pad(audio_tensor, (0, target_samples - current_samples))
@@ -713,25 +722,25 @@ async def dub_generate(job_id: str, req: DubRequest):
# no double-mark. Cached-reuse audio is already marked;
# silence/zero slots carry no speech to mark, so neither is
# re-watermarked.
audio_tensor = embed_watermark(audio_tensor, _model.sampling_rate)
audio_tensor = embed_watermark(audio_tensor, backend.sample_rate)
seg_wav_path = _seg_lang_path(seg_id)
try:
# Keep the existing per-segment WAV contract for previews
# and partial regeneration, but do not keep the tensor in RAM.
atomic_save_wav(seg_wav_path, audio_tensor, _model.sampling_rate)
atomic_save_wav(seg_wav_path, audio_tensor, backend.sample_rate)
except Exception as e:
logger.warning("seg write failed for %s: %s", seg_id, e)
# If the durable segment write fails, still preserve a mix
# copy so this generation can finish.
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, _model.sampling_rate, f"mix_{seg_id}"))
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, audio_tensor, backend.sample_rate, f"mix_{seg_id}"))
try:
del audio_tensor
except Exception:
pass
_release_audio_tensors()
else:
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, _model.sampling_rate))
all_segment_wavs.append((seg.start, seg.end, seg_wav_path, backend.sample_rate))
try:
del audio_tensor
except Exception:
@@ -739,7 +748,7 @@ async def dub_generate(job_id: str, req: DubRequest):
_release_audio_tensors()
except Exception as e:
yield f"data: {json.dumps({'type': 'error', 'segment': i, 'error': str(e)})}\n\n"
sr = _model.sampling_rate
sr = backend.sample_rate
all_segment_wavs.append(_store_mix_wav(seg.start, seg.end, torch.zeros(1, max(0, int(seg_duration * sr))), sr, f"mix_{seg_id}"))
sync_scores.append(1.0)
@@ -768,7 +777,7 @@ async def dub_generate(job_id: str, req: DubRequest):
_save_job(job_id, job)
_t_diskw = time.perf_counter() - _t_diskw_0
sr = _model.sampling_rate
sr = backend.sample_rate
slot_fit = (req.slot_fit or "time_stretch").lower()
overflow_budget_s = max(0.0, float(req.overflow_budget_s or 0.0))
@@ -1196,7 +1205,13 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
if not job:
raise HTTPException(status_code=404, detail="Job not found")
_model = await get_model()
# See the /dub/generate/{job_id} resolution above (issue #312 class) —
# a segment preview resolves ref_audio from the same auto-clone /
# voice-profile sources, so it needs the same cloning-capable gate.
try:
backend = await resolve_generation_backend(require_cloning=True)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
def _gen():
ref_audio = None
@@ -1231,7 +1246,7 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
instruct_str = row["instruct"]
lang = req.language if req.language != "Auto" else None
audios = _model.generate(
audio_out = backend.generate(
text=req.text,
language=lang,
ref_audio=ref_audio,
@@ -1244,21 +1259,15 @@ async def preview_segment(job_id: str, req: SegmentPreviewRequest):
denoise=True,
postprocess_output=True,
)
audio_out = audios[0]
# TODO(#312): this route runs the OmniVoice model directly (not the active
# backend), so VoxCPM2 never reaches it. When these routes become
# engine-aware, guard with `if not getattr(backend, "applies_own_mastering", False)`.
mastered = apply_mastering(
audio_out,
sample_rate=getattr(_model, "sampling_rate", 24000),
)
return normalize_audio(mastered, target_dBFS=-2.0)
if not getattr(backend, "applies_own_mastering", False):
audio_out = apply_mastering(audio_out, sample_rate=backend.sample_rate)
return normalize_audio(audio_out, target_dBFS=-2.0)
# Bounded + pool-reset on hang so a wedged preview generate can't starve the
# GPU pool and brick the backend (#730 class).
audio_tensor = await run_on_gpu_pool_guarded(_gen, what="Dub preview generate")
sr = getattr(_model, "sampling_rate", 24000)
sr = backend.sample_rate
buf = io.BytesIO()
_safe_torchaudio_save(buf, audio_tensor, sr, format="wav")
buf.seek(0)
+24
View File
@@ -16,6 +16,7 @@ Environment variables (`OMNIVOICE_TTS_BACKEND`, `OMNIVOICE_ASR_BACKEND`,
a backend without Settings silently undoing it.
"""
import os
import re
import threading
from time import perf_counter
@@ -429,6 +430,11 @@ def engine_selftest(engine_id: str):
class SelectEngineRequest(BaseModel):
family: str # "tts" | "asr" | "llm"
backend_id: str
# Only meaningful for family="tts", backend_id="mlx-audio" (#981) — picks
# which of mlx-audio's curated models is actually loaded. A curated key
# ("kokoro") or a raw HF repo id ("mlx-community/Kokoro-82M-bf16") — the
# same tolerance MLXAudioBackend.__init__ already has. Ignored otherwise.
model_id: str | None = None
class SelectEngineResponse(BaseModel):
@@ -471,6 +477,24 @@ def select_engine(req: SelectEngineRequest):
f"Backend {req.backend_id} can't run on this machine: {why}. "
f"Pick an engine with a CPU path, or one that supports this host's GPU.",
)
# #981: mlx-audio multiplexes 7+ curated models behind one backend id —
# persist the model pick alongside the backend id so the UI can actually
# select which curated model gets loaded (previously it always defaulted
# to Kokoro no matter what the user downloaded in Settings → Models).
if req.family == "tts" and req.backend_id == "mlx-audio" and req.model_id is not None:
known_keys = tts_backend.MLXAudioBackend.CURATED_MODELS
# Accept a curated key OR a raw HF repo id ("owner/name") — the same
# tolerance MLXAudioBackend.__init__ already has for power users.
# Anything else (typo'd key, malformed id) is rejected outright
# rather than silently persisted as a "custom repo" that then fails
# to resolve at load time.
if req.model_id not in known_keys and not re.fullmatch(r"[\w.-]+/[\w.-]+", req.model_id):
raise HTTPException(
400,
f"Unknown mlx-audio model: {req.model_id!r}. Expected one of "
f"{sorted(known_keys)} or a HF repo id like 'owner/name'.",
)
prefs.set_("mlx_audio_model_id", req.model_id)
prefs.set_(pref_key, req.backend_id)
return {
"family": req.family,
+26 -4
View File
@@ -105,8 +105,8 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
``skip_mastering`` honors a backend's ``applies_own_mastering`` flag
(issue #312): studio engines (e.g. VoxCPM2's native 48 kHz output)
opt out of the broadcast Compressor + Reverb chain that's tuned for
OmniVoice's 24 kHz clone output. Loudness normalization still runs —
opt out of the broadcast highpass + Compressor pre-stage that's tuned
for OmniVoice's 24 kHz clone output. Loudness normalization still runs —
it's a benign peak scale. Mirrors ``_run_tts`` in openai_compat.py.
"""
from services.audio_dsp import (
@@ -143,6 +143,28 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
return normalize_audio(audio_out, target_dBFS=-2.0)
def _safe_exc_text(e: BaseException) -> str:
"""``f"{type(e).__name__}: {e}"`` — the house style used for
unrecognized-error formatting throughout the backend (grep
``type(e).__name__`` in settings.py / asr_backend.py / model_manager.py
/ engines.py) with a guard against leaking a raw container repr.
#977: an AssertionError raised deep inside a vendored dependency
(mlx-audio's Kokoro pipeline) had ``.args`` shaped like
``('du', {'a': 'American English', ...})`` a tuple containing a dict.
``str(e)`` on that renders the WHOLE table straight into the user-facing
message. Any engine's ``generate()`` can raise something shaped like
this (not just Kokoro), so guard generically: if any element of
``e.args`` is a container rather than a plain string, don't interpolate
``str(e)`` at all name the exception type and point at the log
instead.
"""
args = getattr(e, "args", ())
if any(isinstance(a, (dict, list, tuple, set, frozenset)) for a in args):
return f"{type(e).__name__} — see Settings → Logs → Backend for details"
return f"{type(e).__name__}: {e}"
def _exception_chain(e):
"""Yield ``e`` plus every ``__cause__``/``__context__`` beneath it
(cycle-safe). Engines and hub libraries routinely wrap the original
@@ -412,7 +434,7 @@ def _oom_friendly_reraise(e):
raise RuntimeError(
f"TTS engine stopped mid-generation with an error OmniVoice doesn't "
f"recognize. Retry once; if it keeps failing, please report it with "
f"the full trace. Underlying error: {e}"
f"the full trace. Underlying error: {_safe_exc_text(e)}"
) from e
@@ -934,7 +956,7 @@ async def generate_speech(
status_code=500,
detail=(
f"Couldn't synthesize audio. See Settings → Logs → Backend for the full trace. "
f"Underlying error: {e}"
f"Underlying error: {_safe_exc_text(e)}"
),
)
finally:
+4 -4
View File
@@ -237,10 +237,10 @@ def _run_tts(backend, text: str, kw: dict):
sr = backend.sample_rate
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
# That chain's Compressor + 8% Reverb is tuned for OmniVoice's 24 kHz clone
# output; applied to a studio engine it adds an audible level pump and a
# reverb tail that degrade the very output we want clean. Loudness
# normalisation still runs — it's a benign peak scale, not dynamics.
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
# output; applied to a studio engine it adds an audible level pump that
# degrades the very output we want clean. Loudness normalisation still
# runs — it's a benign peak scale, not dynamics.
if not getattr(backend, "applies_own_mastering", False):
wav = apply_mastering(wav, sample_rate=sr)
wav = normalize_audio(wav, target_dBFS=-2.0)
+11
View File
@@ -73,6 +73,17 @@ async def create_profile(
raise ValueError("not an object")
except ValueError:
raise HTTPException(status_code=422, detail="vd_states must be a JSON object")
# Root-cause close for #983: a design profile must never be PERSISTED
# with a partial vd_states shape, regardless of which client (older
# frontend build, hand-edited payload, third-party API caller) created
# it — a missing category key crashes DesignMethodPanel's render on
# every future client that selects this profile. CATEGORY_ORDER is the
# same single source of truth the frontend's CATEGORIES keys mirror
# (core/describe_voice.py), so this can't drift from the picker.
from core.describe_voice import CATEGORY_ORDER
for _cat in CATEGORY_ORDER:
parsed.setdefault(_cat, "Auto")
vd_states = _json.dumps(parsed)
# An all-Auto design (every category left on "Auto") yields an empty
# instruct — that's still a valid, saveable voice: synthesis falls back
# to neutral instruct-only conditioning (see generation.py design path).
+45
View File
@@ -750,6 +750,51 @@ def set_hf_mirror(body: _HFMirrorBody):
return {"configured": url, "restart_required": changed, "presets": _HF_MIRROR_PRESETS}
# ── OpenAI-compatible remote ASR (#877) ─────────────────────────────────────
# A path to Qwen3-ASR/FunASR/SenseVoice — or OpenAI's own Whisper API — today,
# without waiting on transformers to ship a direct Qwen3-ASR integration.
# base_url/model are plain settings_store text rows; the key is encrypted via
# settings_store.set_secret — same convention as /llm-providers, never
# returned to the client, '' clears it, omitted/None leaves it unchanged.
class _ASROpenAICompatBody(BaseModel):
base_url: str | None = None
model: str | None = None
api_key: str | None = Field(None, description="'' clears it, None leaves unchanged")
@router.get("/asr-openai-compat")
def get_asr_openai_compat():
from services import asr_backend
return {
"base_url": asr_backend.resolve_openai_compat_asr_base_url(),
"model": asr_backend.resolve_openai_compat_asr_model(),
"has_key": asr_backend.openai_compat_asr_has_key(),
}
@router.put("/asr-openai-compat")
def set_asr_openai_compat(body: _ASROpenAICompatBody):
from services import asr_backend, settings_store
if body.base_url is not None:
url = body.base_url.strip().rstrip("/")
if url and not url.startswith(("http://", "https://")):
raise HTTPException(status_code=400, detail="Base URL must start with http(s)://")
settings_store.set_text(asr_backend._ASR_OPENAI_COMPAT_BASE_URL_KEY, url)
if body.model is not None:
settings_store.set_text(
asr_backend._ASR_OPENAI_COMPAT_MODEL_KEY, body.model.strip() or "whisper-1"
)
if body.api_key is not None:
settings_store.set_secret(
asr_backend._ASR_OPENAI_COMPAT_SECRET_NAME, body.api_key.strip()
)
return get_asr_openai_compat()
# ── Updates panel: shipped changelog + pre-migration DB backup state ────────
# (feat/safe-updates). Both are read-only, local-first surfaces for
# Settings → Updates: the "What's new" viewer reads the CHANGELOG.md that
+68 -10
View File
@@ -168,16 +168,40 @@ def _detect_gpu() -> dict:
return info
def _probe_network(host: str = "huggingface.co", timeout: float = 2.0) -> bool:
def _probe_network(host: str = "huggingface.co", port: int = 443, timeout: float = 2.0) -> bool:
"""Tiny TCP connect test."""
import socket
try:
with socket.create_connection((host, 443), timeout=timeout):
with socket.create_connection((host, port), timeout=timeout):
return True
except Exception:
return False
def _hf_endpoint_host() -> tuple[str, int]:
"""Host/port of the Hugging Face endpoint actually in effect.
Mirror-aware: restricted-network users (e.g. behind the Great Firewall)
point HF_ENDPOINT at a mirror via Settings Models Hugging Face
mirror. Probing hardcoded huggingface.co would fail them even when their
configured mirror works fine.
"""
try:
from core.failure import configured_hf_mirror
mirror = configured_hf_mirror()
except Exception:
mirror = ""
if mirror:
try:
from urllib.parse import urlsplit
u = urlsplit(mirror)
if u.hostname:
return u.hostname, u.port or (80 if u.scheme == "http" else 443)
except Exception:
pass
return "huggingface.co", 443
def _ram_gb() -> float:
try:
import psutil
@@ -400,15 +424,49 @@ def preflight():
"status": r_status, "detail": r_detail, "fix": r_fix,
})
# ── Network
net_ok = _probe_network()
# ── Network — probes the HF endpoint actually in effect (mirror-aware),
# and a dead network is a WARNING, not a blocker. The app is local-first:
# already-downloaded models work offline, and a hard fail here dead-ends
# restricted-network users (e.g. China, where huggingface.co is blocked)
# on the very first screen — before they can reach the mirror setting
# that fixes it. Model downloads surface their own actionable errors.
net_host, net_port = _hf_endpoint_host()
net_ok = _probe_network(net_host, net_port)
mirror_reachable = False
if not net_ok and net_host == "huggingface.co":
# Official endpoint blocked — if the community mirror is reachable,
# tell the user exactly which switch unblocks them.
mirror_reachable = _probe_network("hf-mirror.com")
if net_ok:
net_fix = None
elif mirror_reachable:
net_fix = (
"huggingface.co is blocked on this network, but the hf-mirror.com "
"community mirror is reachable — apply it below and re-check. "
"Model downloads will use the mirror immediately."
)
elif net_host != "huggingface.co":
net_fix = (
f"Your configured Hugging Face mirror ({net_host}) is unreachable "
"— it may be down or blocked. Pick another mirror or the official "
"endpoint below, or continue offline: models already downloaded "
"keep working."
)
else:
net_fix = (
"Check internet connection, VPN, or corporate firewall whitelist "
"for huggingface.co. You can continue — models already downloaded "
"keep working offline; new downloads need a connection or a "
"mirror (configurable below)."
)
checks.append({
"id": "network", "label": "Network (huggingface.co)",
"status": "pass" if net_ok else "fail",
"detail": "Reachable" if net_ok else "Unreachable on port 443",
"fix": None if net_ok else
"Check internet connection, VPN, or corporate firewall "
"whitelist for huggingface.co.",
"id": "network", "label": f"Network ({net_host})",
"status": "pass" if net_ok else "warn",
"detail": "Reachable" if net_ok else f"Unreachable on port {net_port}",
"fix": net_fix,
# Frontend affordance hint: the wizard offers the mirror quick-pick
# when the endpoint is unreachable (PreflightCheck allows extras).
"mirror_reachable": mirror_reachable,
})
# Aggregate
+15
View File
@@ -42,6 +42,7 @@ _HINTS: dict[str, str] = {
"TRANSFORMERS_IMPORT": "Your transformers install is incomplete. Reinstall it (`uv pip install --reinstall transformers`) or switch ASR to faster-whisper (Settings → Models).",
"OS_INVALID_ARGUMENT": "The OS rejected a file operation (Errno 22 / invalid argument) — in the transcribe path this is the temporary WAV write before ASR. It's almost always the temp directory: missing, read-only, on a full or removed drive, or blocked by antivirus. Check that your system TEMP/TMP folder exists and is writable and the drive has free space (add an OmniVoice antivirus exclusion if you use one), then retry.",
"SOCKS_PROXY_SUPPORT_MISSING": "A SOCKS proxy is configured in your environment (ALL_PROXY/HTTPS_PROXY=socks5://…) and the backend's HTTP client is missing SOCKS support. Newer OmniVoice builds ship SOCKS support (the socksio package) — update the app. If you still see this, unset ALL_PROXY/HTTPS_PROXY for OmniVoice, or run `uv pip install 'httpx[socks]'` in the backend venv, then restart.",
"SSL_HANDSHAKE_FAILURE": "A corporate or antivirus proxy is intercepting HTTPS traffic and re-signing certificates with its own CA — your OS trusts that CA, but Python's bundled certifi CA list doesn't, so the TLS handshake fails even though the connection reached the server. Newer OmniVoice builds trust the OS certificate store at startup (the truststore package), which should already fix this — update the app and retry. If you still see this, add an HTTPS-scanning exclusion for OmniVoice/Python in your antivirus, or ask IT for the proxy's CA bundle and set SSL_CERT_FILE to it, then restart.",
"UNSUPPORTED_VIDEO_URL": "This link isn't a directly downloadable video. Paste a direct video page (e.g. a youtube.com/watch?v=… or douyin.com/video/<id> link), not a share/profile/feed link — or download the file and drop it in directly.",
"VIDEO_DOWNLOAD_NETWORK": "The connection to the video server dropped mid-download (often a transient CDN/network blip or a regional rate-limit). Just retry — OmniVoice already cleaned up the partial download. If it keeps failing, check your network/VPN.",
"BROKEN_VENV": "The Python backend environment was moved or damaged. OmniVoice rebuilds it automatically on the next launch; if it keeps failing, use Clean & Retry on the setup screen.",
@@ -177,6 +178,7 @@ def append_hf_mirror_hint(text: str) -> str:
# hint on a model-load timeout that leaks through the 500 handler.
_CONTEXT_FREE_HINT_CLASSES = frozenset({
"SOCKS_PROXY_SUPPORT_MISSING",
"SSL_HANDSHAKE_FAILURE",
})
@@ -257,6 +259,19 @@ def classify(reason: str) -> str:
# a message that also carries HF wording still names this class.
if "socks proxy" in low or "socksio" in low:
return "SOCKS_PROXY_SUPPORT_MISSING"
# #976: a TLS handshake failing AFTER the TCP connection succeeds — the
# signature of a corporate/antivirus proxy that TLS-inspects traffic and
# re-signs certificates with a CA the OS trusts but Python's bundled
# certifi list doesn't (a different failure mode from #984's TCP-level
# "can't reach the host at all"). Requires "ssl" plus a handshake/cert-
# verify marker so a generic connection error isn't mislabelled.
if "ssl" in low and (
"handshake" in low
or "certificate verify failed" in low
or "sslv3_alert" in low
or "sslcertverificationerror" in low
):
return "SSL_HANDSHAKE_FAILURE"
if ("huggingface" in low or "hf_token" in low or "401" in low or "unauthorized" in low) and (
"token" in low or "auth" in low or "401" in low or "unauthorized" in low
):
+1 -1
View File
@@ -24,7 +24,7 @@ from pathlib import Path
# tests/test_app_version.py::test_all_version_files_in_lockstep and bumped by
# release.yml's version-bump job, so it stays equal to
# pyproject/tauri.conf/Cargo/package.json.
_FALLBACK_VERSION = "0.3.11"
_FALLBACK_VERSION = "0.3.14"
def _fallback_version() -> str:
+1
View File
@@ -70,6 +70,7 @@ class Supertonic3Backend(SubprocessBackend):
id = "supertonic3"
display_name = "Supertonic-3 (31 langs, CPU ONNX, 7 preset voices, OpenRAIL-M)"
supports_voice_design = False # preset voices only
supports_cloning = False # preset voices only; generate() never reads ref_audio
# TTS-04: honest hardware reporting. Supertonic-3 has no CUDA / MPS
# path in the SDK ‑‑ ONNX Runtime CPU EP only.
gpu_compat: tuple[str, ...] = ("cpu",)
+79 -8
View File
@@ -139,6 +139,24 @@ os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
os.environ.setdefault("HF_HUB_ETAG_TIMEOUT", "15")
os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "30")
# ── OS trust store for TLS (#976) ───────────────────────────────────────────
# Users behind a corporate/antivirus proxy that TLS-inspects HTTPS traffic get
# a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure"
# on every model install — the TCP connection succeeds (a different failure
# mode from #984's TCP-level blocked-host case), but the proxy re-signs the
# certificate with its own root CA, which the OS trusts (Windows CryptoAPI/
# SChannel) and Python's bundled `certifi` CA list does not. `inject_into_ssl`
# patches `ssl.SSLContext` process-wide to verify against the OS trust store
# instead, which is the actual fix (not just a nicer error message). Must run
# here — at MODULE level, before huggingface_hub/requests/httpx do any network
# I/O — not inside lifespan(), which runs too late. Not platform-gated: it's a
# correctness improvement everywhere. Best-effort: never block startup.
try:
import truststore
truststore.inject_into_ssl()
except Exception:
pass
# Prevent torchaudio from lazy-importing torchcodec (broken on some installs).
# Proper fix = exclude torchcodec in pyproject.toml; this is a belt-and-braces guard.
@@ -486,6 +504,35 @@ async def _start_mcp_session_manager(session_manager, *, timeout: float):
return task, stop, mounted
async def _cancel_and_await_tasks(*tasks, timeout: float = 3.0) -> None:
"""Cancel each background task and give it a bounded chance to actually
finish before shutdown proceeds ``None`` entries are skipped (a task
that's conditionally created, e.g. ``capture_preload_task``, may not
exist).
``task.cancel()`` alone is not enough for a task awaiting
``run_in_executor()``: once the underlying OS thread is inside blocking
native/import work, cancellation can't stop it, so cancel-and-move-on lets
shutdown finish while that thread is still running invisible to
asyncio, but very much alive when the interpreter starts tearing down
module state under it (#1000 class). Awaiting with a bound (instead of
just cancelling) gives an early-stage task a real chance to exit cleanly
first; a task that's genuinely still deep in blocking work times out here
same as before, and the caller's own GPU-pool reset handles that case.
"""
for t in tasks:
if t is None:
continue
t.cancel()
for t in tasks:
if t is None:
continue
try:
await asyncio.wait_for(t, timeout=timeout)
except (asyncio.CancelledError, asyncio.TimeoutError):
pass
@asynccontextmanager
async def lifespan(app: FastAPI):
# Startup watchdog (#632): a silent hang during startup (e.g. a model-load /
@@ -560,6 +607,7 @@ async def lifespan(app: FastAPI):
# lean and the first dictation is instant instead of a cold model load.
# OMNIVOICE_PRELOAD_CAPTURE_ASR=0 opts out; the warm-up is also skipped
# under 4 GB free RAM (checked at warm time, not boot time).
capture_preload_task = None # only assigned when the preload actually runs (#1000 class)
if _env_flag("OMNIVOICE_PRELOAD_CAPTURE_ASR", default=True):
async def _preload_capture_asr():
await asyncio.sleep(_capture_preload_delay_s())
@@ -628,14 +676,33 @@ async def lifespan(app: FastAPI):
pass
except Exception:
pass
idle_task.cancel()
worker_task.cancel()
# Wait for tasks to finish their current iteration
for t in (idle_task, worker_task):
try:
await asyncio.wait_for(t, timeout=3.0)
except (asyncio.CancelledError, asyncio.TimeoutError):
pass
# preload_task/capture_preload_task matter most here (#1000 class): a quit
# mid-preload used to fall straight through to "Shutdown: done." while the
# model load was still running on a GPU-pool thread — cancel() can't stop
# a thread already inside blocking import/load work, so the process
# reported a clean exit while that background thread was still mid-
# `import transformers`, and got torn down by interpreter finalization
# instead. That surfaced as a misleading "Could not import module
# 'AutoFeatureExtractor'" — transformers' own generic lazy-import wrapper,
# not a real dependency problem. Awaiting here lets an early-stage load
# (still importing, not yet mid weight-download) finish cleanly before we
# report done; a load that's genuinely deep into a multi-GB download still
# times out — _reset_gpu_pool() below abandons it either way.
#
# 20s, not the original 3s (code-review finding post-merge): a cold
# transformers import alone can take longer than 3s on a slow disk or a
# first-ever launch, so the original bound left a real residual window —
# cancellation detaches the asyncio task, but the underlying OS thread
# keeps running past it, and shutdown could still report "done" while
# that thread was alive. Python cannot forcibly kill a running thread, so
# no finite bound eliminates this outright — 20s just shrinks the window
# from "any preload" to "an unusually slow cold-import," which is the
# practical ceiling before a longer shutdown itself becomes the
# complaint. A thread that's still running past 20s was never going to
# finish in a shutdown-appropriate timeframe regardless.
await _cancel_and_await_tasks(
idle_task, worker_task, preload_task, capture_preload_task, timeout=20.0,
)
# Unload the model and free GPU memory
try:
import services.model_manager as mm
@@ -643,6 +710,10 @@ async def lifespan(app: FastAPI):
mm.model = None
logger.info("Shutdown: model unloaded.")
mm.free_vram()
# Abandon a still-running preload's GPU-pool thread (Python can't kill
# a thread mid blocking call) so it can't outlive this shutdown block
# holding a reference into module state that's about to be torn down.
mm._reset_gpu_pool()
except Exception:
pass
# Run GC to release any remaining references
+174 -1
View File
@@ -29,6 +29,7 @@ import os
import re
import threading
from abc import ABC, abstractmethod
from typing import Optional
logger = logging.getLogger("omnivoice.asr")
@@ -1634,6 +1635,161 @@ class FunASRBackend(ASRBackend):
pass
# ── OpenAI-compatible remote transcription (#877 — Qwen3-ASR / FunASR / any
# compatible server, today, without waiting on transformers to catch up) ──
#
# transformers doesn't yet ship a stable Qwen3-ASR integration (issue #877),
# but a self-hosted Qwen3-ASR/FunASR/SenseVoice server exposing an
# OpenAI-compatible `POST /v1/audio/transcriptions` endpoint — or OpenAI's own
# Whisper API — is usable right now. This backend is a pure network client:
# no model runs locally, so it needs no install and claims no GPU.
#
# Settings mirror the LLM-providers convention exactly (services/
# llm_providers.py): base_url/model are plain settings_store text rows; the
# API key is Fernet-encrypted via settings_store.set_secret/get_secret — never
# a .env row, never echoed back to the client. Optional: some self-hosted
# servers (vLLM, LM Studio-style) don't check the key at all.
_ASR_OPENAI_COMPAT_BASE_URL_KEY = "asr.openai_compat.base_url"
_ASR_OPENAI_COMPAT_MODEL_KEY = "asr.openai_compat.model"
_ASR_OPENAI_COMPAT_SECRET_NAME = "asr_openai_compat_key"
def resolve_openai_compat_asr_base_url() -> str:
from services import settings_store
return (
os.environ.get("ASR_OPENAI_COMPAT_BASE_URL")
or settings_store.get_text(_ASR_OPENAI_COMPAT_BASE_URL_KEY)
or ""
)
def resolve_openai_compat_asr_model() -> str:
from services import settings_store
return (
os.environ.get("ASR_OPENAI_COMPAT_MODEL")
or settings_store.get_text(_ASR_OPENAI_COMPAT_MODEL_KEY)
or "whisper-1"
)
def resolve_openai_compat_asr_api_key() -> Optional[str]:
"""Env → encrypted stored key → None. Unlike LLM providers, no 'local'
sentinel: many self-hosted transcription servers accept an empty/omitted
Authorization header outright, so the OpenAI SDK is constructed with
``api_key="not-needed"`` (a non-empty placeholder the SDK requires) when
this returns None, rather than treating a keyless server as unconfigured.
"""
from services import settings_store
return os.environ.get("ASR_OPENAI_COMPAT_API_KEY") or settings_store.get_secret(
_ASR_OPENAI_COMPAT_SECRET_NAME
)
def openai_compat_asr_has_key() -> bool:
"""Whether a key is configured, without ever decrypting it — mirrors
llm_providers.has_key()'s no-plaintext-round-trip contract."""
from services import settings_store
if os.environ.get("ASR_OPENAI_COMPAT_API_KEY"):
return True
return _ASR_OPENAI_COMPAT_SECRET_NAME in settings_store.list_secret_names()
class OpenAICompatASRBackend(ASRBackend):
"""Remote transcription via any OpenAI-compatible server.
Adapts whatever the server returns into this module's expected shape.
Prefers `response_format="verbose_json"` for real per-segment timestamps
(OpenAI's own API and most compatible servers support it); falls back to
plain text with rough single-segment bounds mirroring
MoonshineASRBackend's degraded shape — for minimal servers that reject it.
"""
id = "openai-compat-asr"
display_name = "OpenAI-compatible (remote server)"
gpu_compat = ("cpu",) # network client only — no local compute
def __init__(self):
self._base_url = resolve_openai_compat_asr_base_url()
self._model = resolve_openai_compat_asr_model()
@classmethod
def is_available(cls) -> tuple[bool, str]:
if not resolve_openai_compat_asr_base_url():
return False, "Configure a server endpoint in Settings → Engines"
try:
import openai # noqa: F401
except ImportError:
return False, "openai package not installed. Install with: uv pip install openai"
return True, "ready"
def _client(self):
from openai import OpenAI
api_key = resolve_openai_compat_asr_api_key() or "not-needed"
# max_retries=0: mirrors llm_skills.resolve_skill_client — a
# rate-limited/slow server retrying inside the SDK would blow past
# whatever bounded timeout the caller (dub transcribe, dictation)
# expects from a single call.
return OpenAI(base_url=self._base_url, api_key=api_key, max_retries=0)
def transcribe(self, audio_path: str, *, word_timestamps: bool = True) -> dict:
logger.info(
"OpenAI-compat ASR transcribing %s (base_url=%s, model=%s)",
audio_path, self._base_url, self._model,
)
client = self._client()
try:
with open(audio_path, "rb") as f:
try:
resp = client.audio.transcriptions.create(
file=f, model=self._model, response_format="verbose_json",
)
except Exception:
# Minimal/older compatible servers reject verbose_json
# outright — retry plain before treating it as a real
# failure. Re-open: the SDK may have partially consumed
# the file handle on the first attempt.
f.seek(0)
resp = client.audio.transcriptions.create(
file=f, model=self._model, response_format="json",
)
except Exception as exc:
# Never leak a raw SDK/httpx exception object (auth headers,
# connection internals) straight into a user-facing message —
# same convention as generation.py's _safe_exc_text (#977 class).
raise RuntimeError(
f"OpenAI-compatible ASR server at {self._base_url!r} failed: "
f"{type(exc).__name__}: {exc}"
) from exc
return self._adapt_response(resp)
@staticmethod
def _adapt_response(resp) -> dict:
segments_out = []
# verbose_json: resp.segments is a list of objects with start/end/text.
raw_segments = getattr(resp, "segments", None)
if raw_segments:
for seg in raw_segments:
seg_dict = seg if isinstance(seg, dict) else seg.model_dump()
segments_out.append({
"text": (seg_dict.get("text") or "").strip(),
"start": seg_dict.get("start", 0.0),
"end": seg_dict.get("end", 0.0),
"words": [], # word-level timing isn't part of this API
})
else:
# Plain text response (json/text format) — single-segment shape,
# matching MoonshineASRBackend's degraded fallback exactly.
text = (getattr(resp, "text", None) or "").strip()
if text:
segments_out.append({"text": text, "start": 0.0, "end": None, "words": []})
chunks = [
{"text": seg["text"], "timestamp": (seg["start"], seg["end"])}
for seg in segments_out
]
language = getattr(resp, "language", None) or "en"
return {"chunks": chunks, "segments": segments_out, "language": language}
def _isolated_faster_whisper():
"""Lazy import so the subprocess_asr → subprocess_backend chain isn't
pulled in at registry definition time."""
@@ -1689,6 +1845,7 @@ _REGISTRY: dict[str, type[ASRBackend]] = _LazyASRRegistry({
"moonshine": MoonshineASRBackend,
"funasr": FunASRBackend,
"sherpa-onnx-asr": SherpaDictationBackend,
"openai-compat-asr": OpenAICompatASRBackend,
# "faster-whisper-isolated": resolved lazily (crash-isolated subprocess).
})
@@ -1700,10 +1857,26 @@ _INSTALL_HINTS: dict[str, str] = {
"faster-whisper": "pip install faster-whisper (CTranslate2; cross-platform, CUDA or CPU)",
"mlx-whisper": "pip install mlx-whisper (Apple Silicon only)",
"pytorch-whisper": "Bundled with transformers — no extra install (CUDA/MPS/CPU)",
"nemo-parakeet": "pip install nemo_toolkit[asr] (NVIDIA Parakeet; CUDA or CPU)",
"nemo-parakeet": (
"No safe install path in this app yet — nemo_toolkit's ASR extras pin "
"transformers>=4.57,<4.58, which conflicts with OmniVoice's own "
"transformers>=5.3 requirement and WILL break the backend "
"(ImportError on startup) if installed into this shared venv. Do NOT "
"install nemo_toolkit here. If you want to try Parakeet TDT, set it "
"up in a separate/dedicated Python environment — not the one "
"OmniVoice manages; in-app isolation for this engine is tracked "
"separately."
),
"moonshine": "pip install useful-moonshine (edge/CPU-optimized ASR)",
"funasr": "pip install funasr (SenseVoiceSmall + FSMN-VAD; CUDA or CPU)",
"sherpa-onnx-asr": "uv add sherpa-onnx (ONNX live dictation; CPU, cross-platform)",
"openai-compat-asr": (
"No install needed — configure a server endpoint in Settings → "
"Engines. Points OmniVoice at any OpenAI-compatible transcription "
"server (a self-hosted Qwen3-ASR/FunASR/SenseVoice server, OpenAI's "
"own Whisper API, or similar) — a path to Qwen3-ASR today, without "
"waiting on a direct transformers integration."
),
"faster-whisper-isolated": (
"No extra install (reuses faster-whisper). Escape hatch for hanging "
"transcribes: runs ASR in a separate process that can be force-killed "
+23 -18
View File
@@ -1,9 +1,12 @@
"""
Audio DSP pipeline broadcast-grade mastering + configurable effects chain.
The default `apply_mastering()` is the same chain shipped since v0.1.0
(highpass + compressor + light reverb). The new `apply_effects_chain()`
lets callers build custom pipelines from a list of named effects.
`apply_mastering()` is the shared pre-stage that runs before the user's
effect preset: highpass + gentle compression only (see `MASTERING_CHAIN`).
Reverb is deliberately NOT part of it it is preset-declared only (e.g.
cinematic, warm); a hidden reverb here used to bake echo into every non-raw
synthesis, which field reports flagged. `apply_effects_chain()` lets callers
build custom pipelines from a list of named effects.
All effects use Spotify's `pedalboard` library. When pedalboard isn't
installed, every function degrades gracefully (returns audio unmodified).
@@ -97,24 +100,26 @@ def get_effect_chain(preset_id: str) -> list[dict]:
# ── Core DSP functions ──────────────────────────────────────────────────
#: Shared pre-preset mastering stage: highpass + gentle compression ONLY.
#: Reverb must never live here — a hidden Reverb in this chain baked echo
#: into every non-raw synthesis regardless of the chosen preset (field
#: reports of echoey voices; the podcast preset even promises "no reverb").
#: Reverb is preset-declared only (see EFFECT_PRESETS: cinematic, warm).
MASTERING_CHAIN = [
{"type": "highpass", "cutoff_hz": 60},
{"type": "compressor", "threshold_db": -15, "ratio": 1.5, "attack_ms": 2.0, "release_ms": 100},
]
def apply_mastering(audio_tensor, sample_rate=24000):
"""Applies professional Broadcast-grade DSP (EQ, Compressor, light Reverb) to the clone voice."""
"""Applies the broadcast pre-stage (highpass + gentle compression) to the clone voice.
Reverb is intentionally absent only user-chosen effect presets declare
it. Degrades gracefully: pedalboard missing or any DSP error returns the
input unmodified.
"""
try:
from pedalboard import Pedalboard, Compressor, Reverb, HighpassFilter
import numpy as np
board = Pedalboard([
HighpassFilter(cutoff_frequency_hz=60),
Compressor(threshold_db=-15, ratio=1.5, attack_ms=2.0, release_ms=100),
Reverb(room_size=0.10, wet_level=0.08, dry_level=0.95)
])
audio_np = audio_tensor.cpu().numpy()
if audio_np.ndim == 1:
audio_np = audio_np[np.newaxis, :]
effected = board(audio_np, sample_rate, reset=False)
return torch.from_numpy(effected).to(audio_tensor.device)
except ImportError:
return audio_tensor # Fail gracefully if pedalboard isn't installed
return apply_effects_chain(audio_tensor, sample_rate, MASTERING_CHAIN)
except Exception as e:
logger.warning("Mastering DSP Error: %s", e)
return audio_tensor
+24
View File
@@ -45,11 +45,33 @@ def _asr_device() -> str:
return "cpu"
def _active_tts_id() -> Optional[str]:
"""Configured TTS engine id, or None if it can't be resolved. Attribution
is advisory a prefs/import hiccup must never break /model/loaded."""
try:
from services.tts_backend import active_backend_id
return active_backend_id()
except Exception:
return None
def _tts_attribution(engine_id: str, active: Optional[str]) -> dict:
"""Per-entry engine attribution for TTS-family models. A model can stay
resident in VRAM after the user switches engines (freed only by unload/
idle-evict), so the panel needs to know which entry synthesis actually
routes to. ``is_active_engine`` is None when the active id is unknown."""
return {
"engine_id": engine_id,
"is_active_engine": (engine_id == active) if active is not None else None,
}
def list_loaded() -> dict:
"""Enumerate every currently-loaded model. Shape: ``{"models": [...],
"count": n}`` with per-model id/name/checkpoint/device/vram_mb/unloadable
(+ optional ``note``)."""
models: list[dict] = []
active_tts = _active_tts_id()
# 1. In-process TTS model (OmniVoice)
if mm.model is not None:
@@ -64,6 +86,7 @@ def list_loaded() -> dict:
"device": device,
"vram_mb": round(_tts_vram_mb(), 1),
"unloadable": True,
**_tts_attribution("omnivoice", active_tts),
})
# 2. ASR (WhisperX) — co-loaded with and released alongside the TTS model.
@@ -105,6 +128,7 @@ def list_loaded() -> dict:
"device": get_best_device(),
"vram_mb": round(float(s.get("vram_mb") or 0), 1),
"unloadable": True,
**_tts_attribution(s["id"], active_tts),
})
except Exception:
pass
+17 -5
View File
@@ -277,9 +277,11 @@ def _timeout_guidance(what: str, timeout: float) -> str:
)
return common + (
"most often the GPU is VRAM-starved (a resident model and this job "
"contend for memory). For a durable fix try shorter text, a lighter "
"engine, or set the engine to CPU in Settings → Models. (Raise "
"OMNIVOICE_GENERATE_TIMEOUT_S for very long single generations.)"
"contend for memory). For a durable fix, Flush caches / Unload the "
"resident model (top toolbar or Settings → Models) before retrying, "
"try shorter text, a lighter engine, or set the engine to CPU in "
"Settings → Models. (Raise OMNIVOICE_GENERATE_TIMEOUT_S for very "
"long single generations.)"
)
@@ -955,7 +957,13 @@ def _load_model_sync():
except Exception: # never let failure-formatting mask the real error
err_msg = str(exc)
_set_loading("error", "Model loading failed", error=err_msg)
logger.error("Model loading failed: %s", str(exc))
# #1000 class: transformers' lazy-import machinery wraps ANY disruption
# to an inner import (including one interrupted by process teardown)
# in a generic "Could not import module X. Are this object's
# requirements defined correctly?" — logging only str(exc) discarded
# the real cause in __cause__/__context__ and made a shutdown race
# look like a broken install. exc_info surfaces the full chain.
logger.error("Model loading failed: %s", str(exc), exc_info=exc)
raise
finally:
unregister_listener(lid)
@@ -1087,7 +1095,11 @@ async def preload_model():
model = await _load_model_with_timeout()
logger.info("Preload complete — model ready.")
except Exception as e:
logger.warning("Model preload failed (non-fatal): %s", e)
# See the matching exc_info note on the _load_model_sync handler above
# (#1000 class) — the full chain, not just str(e), is what actually
# distinguishes a real dependency problem from a shutdown-interrupted
# import.
logger.warning("Model preload failed (non-fatal): %s", e, exc_info=e)
def get_model_status():
is_loaded = model is not None
+54
View File
@@ -223,6 +223,60 @@ def extract_segment_refs(
return out
def refine_ref_text(ref_audio_path: str, asr_backend, fallback_text: str) -> str:
"""Re-transcribe a written reference clip and return that transcript.
`extract_speaker_clones`/`extract_segment_refs` pair each audio slice with
the ASR segment's OWN text field, on the assumption that the segment's
timestamps and its transcribed text agree. They routinely don't — Whisper
(and friends) frequently drift on segment boundaries: a trailing word
audible in `[start, end]` but missing from `text`, or vice versa. When the
(ref_audio, ref_text) pair disagrees, zero-shot TTS prompt-priming breaks
down and the clone can speak the mismatched reference text itself instead
of the target-language text it was given to synthesize (issue #1004).
Re-transcribing the *actual written clip* guarantees the pair matches by
construction the model doesn't care whether the original ASR text was
right, only that ref_text is what's really in ref_audio. `asr_backend` is
the caller's already-loaded active backend (duck-typed:
`.transcribe(path, word_timestamps=...) -> dict` with a `chunks` list of
`{"text": ...}`); the model is already warm, so this costs one more short
transcribe call, not a fresh load. Falls back to `fallback_text` never
raises so a re-transcribe failure is a strict no-op, never a regression
from the original (matching) behavior.
"""
if asr_backend is None:
return fallback_text
try:
result = asr_backend.transcribe(ref_audio_path, word_timestamps=False)
text = " ".join(
(c.get("text") or "").strip() for c in (result.get("chunks") or [])
).strip()
return text or fallback_text
except Exception as e:
logger.warning(
"speaker_clone: re-transcribe of %s failed, keeping original ref_text: %s",
ref_audio_path, e,
)
return fallback_text
def refine_ref_texts(clones: dict[str, dict], asr_backend) -> dict[str, dict]:
"""Apply `refine_ref_text` to every entry's `ref_text` in place.
Batches the whole dict (per-speaker `clones` from `extract_speaker_clones`
or per-segment `seg_clones` from `extract_segment_refs`) into the single
executor round-trip the caller submits to the GPU pool, rather than one
dispatch per reference. Mutates and returns `clones` for a convenient
call-and-reassign at the call site.
"""
for entry in clones.values():
entry["ref_text"] = refine_ref_text(
entry["ref_audio"], asr_backend, entry.get("ref_text", "")
)
return clones
# ── Internals ───────────────────────────────────────────────────────────────
+265 -9
View File
@@ -143,12 +143,19 @@ class TTSBackend(ABC):
supports_voice_design: bool = False
#: Whether this engine already emits mastered, studio-grade audio and should
#: therefore skip the shared apply_mastering() chain (Compressor + Reverb,
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
#: 48 kHz) set this True so their clean output isn't pumped/reverbed. Loudness
#: 48 kHz) set this True so their clean output isn't pumped. Loudness
#: normalisation is applied regardless — it's a benign peak scale.
applies_own_mastering: bool = False
#: Whether this engine can clone an arbitrary voice from reference audio
#: (`ref_audio=`), as opposed to only offering a fixed set of preset
#: voices. Default True — most engines clone. Dub/batch gate on this
#: (issue #312 class) before committing to a job that needs it, instead
#: of silently falling back to OmniVoice or mis-cloning per segment.
supports_cloning: bool = True
#: GPU/accelerator targets the engine can run on. Surfaced via the
#: Engine Compatibility Matrix (Plan 02-04 / ENGINE-06) so users can
#: tell at a glance which engines will use their hardware. Defaults to
@@ -603,6 +610,7 @@ class KittenTTSBackend(TTSBackend):
display_name = "KittenTTS (English, 8 preset voices, CPU realtime)"
# KittenTTS ships as an ONNX CPU graph; no CUDA/MPS path today.
gpu_compat = ("cpu",)
supports_cloning = False # fixed preset voices only; ref_audio is ignored
PRESET_VOICES = [
"expr-voice-2-m", "expr-voice-2-f",
@@ -683,6 +691,54 @@ class KittenTTSBackend(TTSBackend):
# ── MLX-Audio (mac-ARM engine multiplexer) ──────────────────────────────────
# #977: Kokoro's own ALIASES table (mlx_audio.tts.models.kokoro.pipeline) only
# recognizes ISO-ish tokens ("en", "es", "fr-fr", "pt-br", …) — it has no idea
# what a full language name is. OmniVoice's `language` kwarg is normally a
# full display name from frontend/src/languages.json (e.g. "Dutch",
# "Spanish"), forwarded verbatim by the frontend and by
# `OmniVoiceBackend.generate()`. Translate the subset Kokoro actually
# supports to the ISO token its own ALIASES expects; a caller that already
# passes an ISO code (or one of Kokoro's own single-letter codes) is
# resolved unchanged by `resolve_kokoro_lang_code()` below.
_KOKORO_ISO_BY_FULL_NAME = {
"english": "en",
"spanish": "es",
"french": "fr",
"hindi": "hi",
"italian": "it",
"portuguese": "pt",
"japanese": "ja",
"chinese": "zh",
}
def resolve_kokoro_lang_code(language: str) -> str:
"""Map a full language name / ISO code to Kokoro's single-letter
`lang_code`, against the AUTHORITATIVE table read from the installed
mlx-audio package (never a hardcoded guess the vendored table is the
only source of truth and can change across mlx-audio versions).
Raises ``ValueError`` for anything Kokoro doesn't support, naming what
it *does* support instead of forwarding a bogus code into Kokoro's
`assert lang_code in LANG_CODES`, which crashes with an unreadable
``(lang_code, LANG_CODES)`` tuple/dict repr (#977).
"""
from mlx_audio.tts.models.kokoro.pipeline import ALIASES, LANG_CODES
key = language.strip().lower()
iso = _KOKORO_ISO_BY_FULL_NAME.get(key, key)
code = ALIASES.get(iso, iso)
if code not in LANG_CODES:
supported = ", ".join(sorted(name.title() for name in _KOKORO_ISO_BY_FULL_NAME))
raise ValueError(
f"mlx-audio's Kokoro model (mlx-community/Kokoro-82M-bf16) doesn't "
f"support language={language!r}. Kokoro supports: {supported}. "
f"Pick one of those, leave language as 'Auto', or switch to a "
f"multilingual engine (e.g. OmniVoice) for other languages."
)
return code
class MLXAudioBackend(TTSBackend):
"""Blaizzy/mlx-audio — Apple-Silicon-only wrapper over 14+ TTS engines
(Kokoro, CSM, Dia, Qwen3-TTS, Chatterbox, MeloTTS, OuteTTS, Spark,
@@ -721,7 +777,16 @@ class MLXAudioBackend(TTSBackend):
def __init__(self):
self._model = None
self._sr = 24000 # most mlx-audio engines emit 24 kHz mono
key = os.environ.get("OMNIVOICE_MLX_AUDIO_MODEL", self.DEFAULT_MODEL_KEY)
# Env var > persisted UI choice (#981 — Settings → Engines curated-
# model picker) > default. Mirrors active_backend_id()'s resolution
# order exactly so power-users can still pin a model without the UI
# silently undoing it.
from core import prefs
key = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=self.DEFAULT_MODEL_KEY,
)
# Accept either a curated key ("kokoro") or a full HF repo id
# ("mlx-community/Kokoro-82M-bf16") — flexibility for power users.
self._model_id = self.CURATED_MODELS.get(key, key)
@@ -759,6 +824,18 @@ class MLXAudioBackend(TTSBackend):
# silently ignores languages it doesn't know.
return ["multi"]
@property
def supports_cloning(self) -> bool:
"""Model-dependent — this adapter multiplexes 7+ curated models and
only some take a reference-audio speaker prompt. `generate()` passes
`ref_audio` through when present (~kwargs below) but silently retries
without it on a TypeError, so an engine picked for cloning that's
actually running Kokoro/Qwen3-TTS/etc. would clone nothing. Of the
curated set, only CSM (`mlx-community/csm-1b-8bit`) is confirmed to
accept a reference prompt default False for every other model,
curated or user-supplied, until positively confirmed."""
return self._model_id == self.CURATED_MODELS.get("csm")
def _ensure_loaded(self):
if self._model is not None:
return
@@ -772,6 +849,7 @@ class MLXAudioBackend(TTSBackend):
voice = kw.get("voice")
ref_audio = kw.get("ref_audio")
ref_text = kw.get("ref_text")
language = kw.get("language")
speed = float(kw.get("speed", 1.0))
@@ -782,7 +860,31 @@ class MLXAudioBackend(TTSBackend):
kwargs = {"text": text, "speed": speed}
if voice: kwargs["voice"] = voice
if ref_audio: kwargs["ref_audio"] = ref_audio
if language: kwargs["lang_code"] = language[:2].lower()
# CSM (sesame.py) only builds its cloning context when BOTH ref_audio
# AND ref_text are present — with ref_text missing, its context list
# stays empty and indexing into it raises an opaque
# "IndexError: list index out of range" deep inside mlx-audio,
# instead of ever attempting the clone. Community-diagnosed (#1012).
if ref_audio and ref_text: kwargs["ref_text"] = ref_text
if language and language != "Auto":
if self._model_id == self.CURATED_MODELS.get("kokoro"):
# Kokoro's vendored pipeline hard-asserts `lang_code` against
# its own single-letter table — a bogus code crashes with an
# unreadable AssertionError instead of failing cleanly
# (#977). Resolve against the authoritative installed table
# instead of guessing via `language[:2]`.
kwargs["lang_code"] = resolve_kokoro_lang_code(language)
else:
# `lang_code`-as-2-letter-truncation is Kokoro's own
# convention, not mlx-audio's in general — other curated
# models either ignore unrecognized kwargs (CSM/Dia/OuteTTS
# accept **kwargs and drop it) or expect something else
# entirely (Qwen3-TTS's own docstring: "lang_code: Language
# code (auto, chinese, english, etc.)" — a full name, not a
# 2-letter code). Kokoro's strict validation doesn't apply to
# them, so don't reject a language that's valid for whatever
# model is actually active.
kwargs["lang_code"] = language[:2].lower()
pieces = []
try:
@@ -1111,6 +1213,7 @@ class SherpaOnnxBackend(TTSBackend):
# Sherpa-ONNX uses the onnxruntime providers — CPU is the universal
# baseline; CUDA provider is available on Linux/Windows installs.
gpu_compat = ("cuda", "cpu")
supports_cloning = False # VITS speaker-id only; no ref_audio support
def __init__(self):
self._tts = None
@@ -1366,6 +1469,21 @@ _SETUP_SNIPPETS: dict[str, str] = {
}
# Short, readable labels for mlx-audio's curated models (#981) — surfaced in
# the Settings → Engines model picker so users see more than a bare key.
# Single-sourced here rather than on MLXAudioBackend.CURATED_MODELS itself so
# the class dict stays a plain key → repo-id map (what __init__ needs).
_MLX_AUDIO_MODEL_LABELS: dict[str, str] = {
"kokoro": "Kokoro (default, fast)",
"csm": "CSM (voice cloning)",
"qwen3-tts": "Qwen3-TTS (voice design)",
"dia": "Dia",
"chatterbox": "Chatterbox",
"melotts": "MeloTTS (lightweight)",
"outetts": "OuteTTS",
}
def list_backends() -> list[dict]:
"""Enumerate every registered backend with its availability state.
@@ -1449,6 +1567,22 @@ def list_backends() -> list[dict]:
# effective_device / routing_status / routing_reason (scrubbed):
**routing_fields(gpu_compat, caps),
})
# #981: mlx-audio multiplexes 7+ curated models behind one backend id
# — surface the roster + the currently-active pick so Settings can
# render a model picker instead of always defaulting to Kokoro.
# mlx-audio ONLY; every other backend loads a single fixed model.
if bid == "mlx-audio":
from core import prefs
active_model = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=cls.DEFAULT_MODEL_KEY,
)
out[-1]["curated_models"] = [
{"key": key, "label": _MLX_AUDIO_MODEL_LABELS.get(key, key), "repo_id": repo_id}
for key, repo_id in cls.CURATED_MODELS.items()
]
out[-1]["active_model_id"] = active_model
return out
@@ -1458,6 +1592,29 @@ def get_backend_class(backend_id: str) -> type[TTSBackend]:
return _REGISTRY[backend_id]
def cloning_capable_engine_ids() -> list[str]:
"""Engine ids that support reference-audio voice cloning — used to build
an actionable error when the active engine can't (dub/batch gating).
Iterates the same registry ``list_backends()`` uses, via ``.items()`` so
lazy entries resolve through ``_LazyRegistry``'s snapshot-safe iteration
(see ``_LazyRegistry.__iter__``) exactly like every other registry scan
in this module.
A class-level ``getattr`` on a *property* returns the descriptor object
itself (always truthy) rather than its computed value so a
model-dependent adapter like ``MLXAudioBackend`` (only some of its 7+
curated models can clone) would always show up here regardless of which
model is actually configured. Excluded rather than falsely recommended:
``isinstance(..., bool)`` is False for a descriptor, True for a plain
class attribute.
"""
return [
bid for bid, cls in _REGISTRY.items()
if isinstance((v := getattr(cls, "supports_cloning", True)), bool) and v
]
def active_routing() -> dict | None:
"""Routing verdict for the currently-active TTS engine, or ``None`` if it
can't be determined (no engine / probe failure).
@@ -1526,15 +1683,21 @@ def active_backend_id() -> str:
# call the outgoing engine's unload() before switching.
_active_instance: "TTSBackend | None" = None
_active_instance_id: "str | None" = None
# mlx-audio multiplexes 7+ curated models behind one backend id — a model-only
# switch (same "mlx-audio" id, different curated model) must also invalidate
# the cache, or picking a different model in Settings has no effect until the
# app restarts (#981). Only meaningful when _active_instance_id == "mlx-audio".
_active_mlx_model_key: "str | None" = None
def reset_active_backend() -> None:
"""Unload + clear the cached active backend. For app shutdown and tests.
Idempotent and best-effort a raising unload() never propagates."""
global _active_instance, _active_instance_id
global _active_instance, _active_instance_id, _active_mlx_model_key
inst = _active_instance
_active_instance = None
_active_instance_id = None
_active_mlx_model_key = None
if inst is not None:
try:
inst.unload()
@@ -1552,13 +1715,32 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
``model=`` (caller already holds a loaded model), we return a fresh view
over the shared singleton rather than caching it but a switch *away from*
a different engine still triggers that engine's unload().
For mlx-audio specifically, the backend id alone doesn't capture *which*
curated model is loaded (#981) — so we also track the resolved model key
and treat a model-only change as a switch, reusing the exact same
unload-and-reconstruct path as an id switch.
"""
global _active_instance, _active_instance_id
global _active_instance, _active_instance_id, _active_mlx_model_key
bid = active_backend_id()
# Switching engines: release the outgoing one first. Best-effort so a bad
# unload() can never block the switch.
if _active_instance is not None and _active_instance_id != bid:
mlx_model_key = None
if bid == "mlx-audio":
from core import prefs
mlx_model_key = prefs.resolve(
"mlx_audio_model_id",
env="OMNIVOICE_MLX_AUDIO_MODEL",
default=MLXAudioBackend.DEFAULT_MODEL_KEY,
)
# Switching engines (or, for mlx-audio, switching curated models): release
# the outgoing one first. Best-effort so a bad unload() can never block
# the switch.
switching = _active_instance is not None and (
_active_instance_id != bid
or (bid == "mlx-audio" and mlx_model_key != _active_mlx_model_key)
)
if switching:
try:
_active_instance.unload()
except Exception as exc: # noqa: BLE001
@@ -1566,6 +1748,7 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
type(_active_instance).__name__, exc)
_active_instance = None
_active_instance_id = None
_active_mlx_model_key = None
cls = get_backend_class(bid)
if cls is OmniVoiceBackend and model is not None:
@@ -1577,9 +1760,82 @@ def get_active_tts_backend(*, model=None) -> TTSBackend:
if _active_instance is None or _active_instance_id != bid:
_active_instance = OmniVoiceBackend(model=model) if cls is OmniVoiceBackend else cls()
_active_instance_id = bid
_active_mlx_model_key = mlx_model_key
return _active_instance
# ── Shared generation-time engine resolution (issue #312 class) ───────────
#
# dub_generate.py and batch.py used to call services.model_manager.get_model()
# directly, hardcoding OmniVoice regardless of the engine selected in
# Settings → Engines — a SILENT fallback: pick VoxCPM2, dub anyway with
# OmniVoice, no error. This is the single resolution path both routers now
# call instead, mirroring generation.py's /generate resolution (engine id →
# is_available() → routing gate) plus a voice-cloning capability gate that
# /generate doesn't need (OmniVoice's native path always clones).
async def resolve_generation_backend(
*, require_cloning: bool = False, cloning_purpose: str = "dubbing",
) -> TTSBackend:
"""Resolve + validate the active TTS engine for a generation call.
Returns the live backend instance (:func:`get_active_tts_backend`)
cached, and properly unload()ed on an engine switch. Raises ``ValueError``
with an actionable message (never silently falls back to OmniVoice) when:
* the configured engine id is unknown (bad env var / stale pref),
* the engine reports itself unavailable (``is_available()``),
* the engine needs an accelerator this host lacks and has no CPU path
(``routing_status == "unavailable"``),
* ``require_cloning`` is True and the resolved backend can't clone
from reference audio (``supports_cloning`` False) checked on the
live *instance*, not the class, so a model-dependent adapter like
MLX-Audio (Kokoro vs. CSM) is judged by what's actually loaded.
"""
engine_id = active_backend_id()
try:
backend_cls = get_backend_class(engine_id)
except ValueError as e:
raise ValueError(
f"Active TTS engine '{engine_id}' is not a recognized backend ({e}). "
"Check Settings → Engines or the OMNIVOICE_TTS_BACKEND env var."
) from e
try:
ok, msg = backend_cls.is_available()
except Exception as exc: # noqa: BLE001 — surface as an actionable ValueError
ok, msg = False, f"{type(exc).__name__}: {exc}"
if not ok:
raise ValueError(f"TTS engine '{engine_id}' is not available: {_mask_hf_tokens(msg)}")
from core.device_caps import detect_host_caps
from services.engine_routing import resolve_routing
routing = resolve_routing(getattr(backend_cls, "gpu_compat", ("cpu",)), detect_host_caps())
if routing["routing_status"] == "unavailable":
raise ValueError(routing["routing_reason"])
_model = None
if backend_cls is OmniVoiceBackend:
# OmniVoice needs its model pre-loaded before construction: called
# from an async context, OmniVoiceBackend._ensure_loaded() refuses to
# bootstrap its own event loop (see its docstring) — same reason
# generation.py's /generate special-cases this backend.
from services.model_manager import get_model
_model = await get_model()
backend = get_active_tts_backend(model=_model)
if require_cloning and not getattr(backend, "supports_cloning", True):
raise ValueError(
f"The active TTS engine '{engine_id}' doesn't support voice cloning, "
f"so {cloning_purpose} can't preserve speaker voices. Switch to one "
f"of: {', '.join(cloning_capable_engine_ids())} in Settings → "
"Engines, or use OmniVoice for this job."
)
return backend
# ── PEP 562 lazy attribute re-export ───────────────────────────────────────
#
# Allows ``from services.tts_backend import IndexTTS2Backend`` to keep
+6 -2
View File
@@ -82,8 +82,12 @@ If `huggingface.co` is slow or blocked, point the client at a mirror:
HF_ENDPOINT=https://hf-mirror.com
```
Set it as an environment variable (or in **Settings → environment**) before
downloading. Caveats:
Set it in **Settings → Models → Hugging Face mirror** (quick-pick presets
included), or as an environment variable before launching. On first run, the
setup wizard offers the same mirror quick-pick right on the system-check
screen when the endpoint is unreachable — the network check is a warning, not
a blocker, so an offline or firewalled machine can still finish setup once
models are available (mirror, or manual download below). Caveats:
- A mirror serves the **classic** download path, **not Xet** — you lose
chunk-dedup and Xet's parallel fetch, but you gain reachability. On the
+43
View File
@@ -0,0 +1,43 @@
# OmniVoice Studio — OpenAI-Compatible Remote ASR
A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own
Whisper API — today, without waiting on `transformers` to ship a direct
Qwen3-ASR integration (tracked separately). Unlike every other ASR engine,
this one runs no model locally: it's a pure network client that calls any
server exposing an OpenAI-compatible `POST /v1/audio/transcriptions`
endpoint.
## Setup
No install step — configure it directly:
1. Open **Settings → Models** and find **OpenAI-compatible ASR (remote
server)**.
2. Set **Server URL** to your server's base URL (e.g.
`http://localhost:8000/v1` for a local Qwen3-ASR/FunASR server, or
`https://api.openai.com/v1` for OpenAI's own API).
3. Set **Model** to whatever your server expects (`whisper-1` for OpenAI's
API; check your self-hosted server's docs otherwise).
4. **API key** is optional — many self-hosted servers accept requests
without one. Set it if your server requires auth, or if you're using
OpenAI's own API.
5. Activate the engine in **Settings → Engines** — click **Use** on
**OpenAI-compatible ASR** in the ASR Engines table (the same picker TTS
engines have). Power users can pin it instead by setting
`OMNIVOICE_ASR_BACKEND=openai-compat-asr` before launching — the env var
always wins over the Settings pick.
## Response format
The backend prefers `response_format=verbose_json` for real per-segment
timestamps (OpenAI's API and most compatible servers support it) and falls
back to plain text automatically if your server rejects that format. Neither
path returns word-level timestamps — that's not part of this API.
## Privacy note
Unlike every other ASR engine in OmniVoice, audio sent through this backend
leaves your machine — to whatever server you configured. If that's a
self-hosted server on your own network, nothing leaves your control; if
it's a third-party API (OpenAI's, or someone else's), review their data
handling before sending anything sensitive.
+2
View File
@@ -74,6 +74,8 @@ asr_engines:
readme: FunASR
- id: sherpa-onnx-asr
readme: "**sherpa-onnx** (live dictation)"
- id: openai-compat-asr
readme: "**OpenAI-compatible** ⚠️ remote"
# Doc files that must exist (the install path users are sent to).
docs:
+2
View File
@@ -56,6 +56,8 @@ Priority: `duration` > `speed`.
| `preprocess_prompt` | bool | True | Whether to apply preprocessing to the voice-clone prompt audio (remove long silences in reference audio, add punctuation in the end of reference text). |
| `postprocess_output` | bool | True | Apply post-processing to generated audio (remove long silences). |
> **Tip — reference-clip quality transfers.** Zero-shot cloning mirrors the acoustics of the reference clip, not just the voice: a clip recorded in an echoey room clones echoey. Record dry and close-mic for clean output. No effect preset adds reverb unless you choose one that declares it (Cinematic, Warm).
## Long-Form Generation
To support stable long-form speech generation with low VRAM consumption, the text is automatically split into smaller segments when the estimated duration of the generated speech exceeds `audio_chunk_duration`, with each segment producing approximately `audio_chunk_duration` seconds of audio. This approach allows the model to accept arbitrarily long text and generate arbitrarily long speech with near-constant VRAM consumption.
+31 -15
View File
@@ -5,13 +5,17 @@ working OmniVoice Studio install on a Debian / Ubuntu / Fedora / Arch host.
## Prerequisites
### Using the AppImage or .deb
### Using the AppImage
- **Linux x86_64** with a desktop session (X11 or Wayland) capable of running
a Tauri / WebKitGTK app.
- **~10 GB free disk** for the app, its Python environment, and model weights.
- Optional: an **NVIDIA driver** for CUDA GPU acceleration — the app runs
CPU-only without one. For AMD GPUs see [AMD GPU (ROCm)](#amd-gpu-rocm).
- Optional: **yt-dlp** for downloading YouTube/video clips directly in the
Voice Gallery and Dub tabs — `sudo apt install yt-dlp` (Debian/Ubuntu),
`sudo dnf install yt-dlp` (Fedora), or `sudo pacman -S yt-dlp` (Arch).
Without it those downloads fail; everything else works fine.
That's it — Python, FFmpeg, and the model weights are bundled or bootstrapped
by the app itself on first launch. No toolchain needed.
@@ -73,12 +77,15 @@ No FUSE? Use `--appimage-extract-and-run`:
./OmniVoice.Studio_*.AppImage --appimage-extract-and-run
```
## Install (.deb)
## .deb package
```bash
sudo apt install ./OmniVoice.Studio_*.amd64.deb
omnivoice-studio
```
Not currently published: `.deb` bundling is disabled in the release pipeline
because of a `tauri-cli` bug (`Failed to create control scripts`) — see the
comment in `.github/workflows/release.yml` for the tracking note. The
AppImage above is the supported Linux install path until a `tauri-cli`
version resolves it. `apt install`-able `.deb`s shipped before v0.3 (see
[.deb ffprobe conflict](#deb-ffprobe-conflict) below) if you're upgrading
from one of those.
The desktop app uses these canonical paths (kept in sync with
`scripts/desktop-prod.sh` by the docs-drift CI gate):
@@ -190,26 +197,35 @@ Three ways to opt in, in order of preference:
with an AMD GPU but no ROCm runtime it stays offered-but-unselected — install
ROCm first (or continue on CPU). Choosing ROCm makes the bootstrap reinstall
`torch`/`torchaudio` from the ROCm wheel index
(`https://download.pytorch.org/whl/rocm6.2` by default) right after the
dependency sync.
(`https://download.pytorch.org/whl/rocm6.4` by default) right after the
dependency sync — matched to the app's pinned `torch==2.8.0` (the rocm6.2
index only ever published up to torch 2.5.1, so it silently failed the
reinstall and left the CPU-only CUDA build in place).
**2. Environment variable (existing installs / headless).** Set
`OMNIVOICE_TORCH_VARIANT=rocm` before launching — the next bootstrap performs
the same ROCm reinstall. `OMNIVOICE_TORCH_INDEX=<url>` overrides the wheel
index when you need a different ROCm version
([pytorch.org](https://pytorch.org/get-started/locally/) lists available
wheels). If the reinstall fails (network, unsupported card), OmniVoice keeps
the default torch build and warns instead of breaking the install.
index when you need a different ROCm version — e.g. AMD publishes newer
driver-matched builds (7.2.x) at `repo.radeon.com` as a `--find-links` page
rather than a PyPI-style index:
```bash
uv pip install --reinstall torch==2.8.0 torchaudio==2.8.0 \
--find-links https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2.4/
```
run that manually if you want a specific ROCm point release; the
`OMNIVOICE_TORCH_INDEX` env var only accepts a PEP 503 index URL, not a
find-links page. If the reinstall fails (network, unsupported card), OmniVoice
keeps the default torch build and warns instead of breaking the install.
**3. Manual wheel swap (fallback).** Replace torch with the ROCm wheel
**after** the first-run install populates the venv:
```bash
# From the project directory (source install), into OmniVoice's uv venv.
# Current stable is ROCm 6.2 — match your installed ROCm/driver version
# (https://pytorch.org/get-started/locally/ lists available wheels).
# Matches the app's torch==2.8.0 pin — a different ROCm point release
# (e.g. rocm6.2, rocm7.x) may not carry that exact torch build.
uv pip install --reinstall torch torchaudio \
--index-url https://download.pytorch.org/whl/rocm6.2
--index-url https://download.pytorch.org/whl/rocm6.4
```
Once a ROCm build of PyTorch is in the venv, detection is automatic —
+48 -2
View File
@@ -242,6 +242,15 @@ for the dedicated CosyVoice path.
**Linked issue:** [#55](https://github.com/debpalash/OmniVoice-Studio/issues/55)
**Same class, ASR side:** the `nemo-parakeet` ASR engine has the identical
problem and currently has **no safe install path** at all — `nemo_toolkit[asr]`
hard-pins `transformers>=4.57,<4.58`, which is unsatisfiable alongside
OmniVoice's own `transformers>=5.3` requirement. Installing it into the
shared venv breaks the backend outright. Do not `pip install nemo_toolkit`
into OmniVoice's environment; if you want to try it, use a separate Python
environment. Isolated-venv support for this engine (matching CosyVoice/
dots-tts) is tracked in [#974](https://github.com/debpalash/OmniVoice-Studio/issues/974).
## 12. CUDA PyTorch wheel download fails on first run
**Symptom:** first-run setup stops at **Installing dependencies** with a failure
@@ -314,8 +323,10 @@ order:
files are a common false-positive quarantine), then re-enable.
- **Connection** — use a stable, direct connection; pause any VPN; avoid
corporate/school networks.
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, set a
mirror **before** launching and relaunch:
- **Region mirror** — if `huggingface.co` is slow/blocked where you are, pick a
mirror in-app (**Settings → Models → Hugging Face mirror**, or the quick-pick
the first-run system check offers when the endpoint is unreachable), or set
it as an env var before launching and relaunch:
- macOS/Linux: `export HF_ENDPOINT=https://hf-mirror.com`
- Windows (PowerShell): `[Environment]::SetEnvironmentVariable("HF_ENDPOINT","https://hf-mirror.com","User")`
@@ -434,6 +445,41 @@ quit OmniVoice Studio, delete the folder below, then start the app again.
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\com.debpalash.omnivoice-studio\EBWebView"
```
## 16. macOS: microphone permission never prompts, OmniVoice never appears in System Settings
**Symptom:** clicking record shows "Microphone access denied. macOS: open
System Settings → Privacy & Security → Microphone and enable OmniVoice" —
but OmniVoice never appears in that list, so there's nothing to enable.
`NSMicrophoneUsageDescription` is present in the app's `Info.plist`, and
resetting the permission (`tccutil reset Microphone
com.debpalash.omnivoice-studio`) followed by a relaunch changes nothing — no
system prompt ever appears.
**Cause:** the app bundle was missing the Hardened Runtime *entitlement* for
microphone access. An earlier revision of this section blamed an upstream
Tauri/WebKit limitation — that was wrong (a community contributor,
[@MahdiHedhli](https://github.com/MahdiHedhli), read the sources more
carefully and found the real gap). wry's `WKUIDelegate` already grants the
WebKit-layer media-capture request; but Tauri's macOS bundler enables
Hardened Runtime by default, and Hardened Runtime blocks microphone hardware
access unless `com.apple.security.device.audio-input` is present in the
signed binary's entitlements — regardless of `Info.plist`'s
`NSMicrophoneUsageDescription` (that only supplies the prompt *text*).
Without the entitlement, macOS's TCC layer never registers a request, which
is exactly why the app never appears in the System Settings list.
**Fix:** ships in the release after v0.3.12 (the bundle now carries
`src-tauri/entitlements.plist` — [#1016](https://github.com/debpalash/OmniVoice-Studio/pull/1016),
contributed by the same person who diagnosed it). Update and live recording
works, with a normal macOS permission prompt on first use.
**Workaround on older builds (≤ v0.3.12):** record your voice sample in any
other app (Voice Memos, QuickTime, etc.) and upload the resulting file in
OmniVoice instead of using live recording — upload-based cloning is
unaffected and works normally.
**Linked issue:** [#1013](https://github.com/debpalash/OmniVoice-Studio/issues/1013)
## Dub: "translation engine needs the optional … package"
**Symptom:** in the Dub tab, translating fails with e.g. *"The 'google'
+22
View File
@@ -77,6 +77,28 @@ Download the latest MSI from the
run it, follow the wizard. The shortcut lands in the Start menu as
**OmniVoice Studio**.
### Installing to a different drive
<a id="install-other-drive"></a>
The wizard's **directory picker** lets you install the app to any **local**
drive (D:, E:, …). Two caveats:
- **Mapped network drives (Z: → a share) are not supported** — this is a
Windows Installer limitation, not an OmniVoice bug: MSI custom actions run
as a service account that doesn't see per-user drive mappings, so the
install fails or rolls back. Install to a local drive instead.
- The install location only moves the ~200 MB app itself. The big data
(models, voices, projects — tens of GB) lives in the **data directory**,
which you move independently: **Settings → Storage → Models directory**
in-app, or `OMNIVOICE_DATA_DIR` / [Portable mode](#portable-install) for
the whole data tree.
If an install to a local non-C: drive fails anyway, capture a log with
`msiexec /i OmniVoice*.msi /L*V install.log` and
[open an issue](https://github.com/debpalash/OmniVoice-Studio/issues) with it
— that log shows exactly which step rolled back.
## Portable install (Windows)
<a id="portable-install"></a>
@@ -6,7 +6,7 @@ Today the longform renderer (Audiobook + Stories) applies a **single-pass** `lou
Upgrade to **two-pass** `loudnorm`: a first **measure** pass (`print_format=json`, output to `-f null -`) parses the clip's `input_i / input_tp / input_lra / input_thresh / target_offset`, then a second **apply** pass feeds those measured values back as `measured_*` + `offset` + `linear=true`. This lands the output accurately on the preset target. The change is a **runner enhancement** layered over the existing pure builders — the pure `build_loudnorm_filter()` and `LOUDNESS_PRESETS` stay; we add a measure-filter builder, a measured-apply-filter builder, a JSON parser, a measure-cmd argv builder, and an async two-pass orchestrator that runs in `_render_longform_sse` (`backend/api/routers/audiobook.py:345`) between the chapter renders and the final mux. Loudness stays **opt-in** (`loudness: None` default on both `AudiobookRequest` `:151` and `LongformRenderRequest` `:510`), so default cross-platform behavior is unchanged.
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard EQ/Compressor/Reverb chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard highpass/Compressor chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
## Problem
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omnivoice-studio",
"version": "0.3.11",
"version": "0.3.14",
"private": true,
"license": "AGPL-3.0-only",
"type": "module",
+2 -1
View File
@@ -2941,7 +2941,7 @@ dependencies = [
[[package]]
name = "omnivoice-studio"
version = "0.3.11"
version = "0.3.14"
dependencies = [
"arboard",
"dirs-next",
@@ -2969,6 +2969,7 @@ dependencies = [
"walkdir",
"webkit2gtk",
"webview2-com",
"windows 0.61.3",
"windows-core 0.61.2",
"zip 2.4.2",
]
+8 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "omnivoice-studio"
version = "0.3.11"
version = "0.3.14"
description = "OmniVoice Studio AI voice cloning & dubbing desktop app"
authors = ["Debpalash"]
license = "AGPL-3.0-only"
@@ -71,6 +71,13 @@ zip = { version = "2", default-features = false, features = ["deflate"] }
# Versions match what wry already locks — no new native code is pulled in.
webview2-com = "0.38"
windows-core = "0.61"
# Raw HWND access (GetWindowLongPtrW/SetWindowLongPtrW/ShowWindow) to mark the
# dictation pill WS_EX_NOACTIVATE so showing it doesn't steal Win32 foreground
# activation (#982 — Windows counterpart of #287's macOS focus-steal fix).
# Version pinned to match what `tauri` itself already resolves to (0.61.x) so
# `WebviewWindow::hwnd()`'s return type and our syscalls share the exact same
# `HWND` type — no second copy of the crate enters the dependency graph.
windows = { version = "0.61", features = ["Win32_Foundation", "Win32_UI_WindowsAndMessaging"] }
[target.'cfg(unix)'.dependencies]
libc = "0.2"
+19 -1
View File
@@ -22,9 +22,27 @@ HERE="$(dirname -- "$(readlink -f -- "$0")")"
# Sourced by AppRun.test.sh — keep this function pure so unit tests can stub
# `pkg-config`, source the file, call _detect_webkit_workaround, and inspect
# the resulting environment without exec'ing the binary.
#
# Version source (#961 follow-up): the WebKitGTK that actually RUNS is the
# BUNDLED copy (LD_LIBRARY_PATH below puts $HERE/usr/lib first) — NOT the
# host's. Asking the host's pkg-config therefore reads the wrong number
# whenever host and bundle diverge (e.g. a user who builds from source has
# dev packages installed, so pkg-config answers with their system's healthy
# 2.48 while the bundle runs an older lib — skipping a workaround the running
# library needs). inject-apprun.sh stamps the bundled version into
# .bundled-webkitgtk-version at build time, where it is knowable by
# construction; the host pkg-config path survives only as a fallback for
# bundles predating the stamp. OMNIVOICE_APPRUN_WK_MARKER exists for the
# unit tests to point at a fixture marker.
_detect_webkit_workaround() {
local wk_version="0.0"
if command -v pkg-config >/dev/null 2>&1; then
local marker="${OMNIVOICE_APPRUN_WK_MARKER:-$HERE/.bundled-webkitgtk-version}"
if [ -r "$marker" ]; then
# Empty/unreadable marker content → "0.0" (unknown) → fail-safe workaround,
# same philosophy as the missing-pkg-config branch below.
wk_version="$(cat "$marker" 2>/dev/null | tr -d '[:space:]')"
[ -n "$wk_version" ] || wk_version="0.0"
elif command -v pkg-config >/dev/null 2>&1; then
wk_version="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|| echo "0.0")"
@@ -72,6 +72,56 @@ run_case "2.46 (broken)" "2.46.1" "1"
run_case "2.48 (healthy)" "2.48.0" "unset"
run_case "pkg-config absent" "0.0" "1" "no"
# ── Bundled-version marker cases (#961 follow-up) ───────────────────────────
# inject-apprun.sh stamps the bundle's actual WebKitGTK version into
# .bundled-webkitgtk-version at build time; AppRun must prefer that marker
# over the host's pkg-config (which reports the SYSTEM version — wrong
# whenever it diverges from the bundled copy, e.g. on a machine with newer
# dev packages installed).
run_marker_case() {
local label="$1" marker_content="$2" pkg_output="$3" expected="$4"
local marker_file
marker_file="$(mktemp)"
printf '%s\n' "$marker_content" > "$marker_file"
local actual
actual=$(
bash -c '
set +e
pkg_output="'"$pkg_output"'"
export OMNIVOICE_APPRUN_WK_MARKER="'"$marker_file"'"
pkg-config() { echo "$pkg_output"; }
export -f pkg-config
exec() { :; }
export -f exec
# shellcheck disable=SC1090
source "'"$THIS_DIR"'/AppRun" >/dev/null 2>&1 || true
echo "${WEBKIT_DISABLE_COMPOSITING_MODE:-unset}"
'
)
rm -f "$marker_file"
if [[ "$actual" == "$expected" ]]; then
echo "PASS [$label]"
PASS_COUNT=$((PASS_COUNT + 1))
else
echo "FAIL [$label]: expected '$expected' got '$actual'" >&2
FAIL_COUNT=$((FAIL_COUNT + 1))
fi
}
# Marker says broken → workaround applies, even though host pkg-config says healthy.
run_marker_case "marker 2.46 beats host 2.48" "2.46.1" "2.48.0" "1"
# Marker says healthy → no workaround, even though host pkg-config says broken
# (the exact #961 inversion: from-source user with old system lib, new bundle).
run_marker_case "marker 2.48 beats host 2.44" "2.48.0" "2.44.3" "unset"
# Empty marker → treated as unknown → fail-safe workaround.
run_marker_case "empty marker fails safe" "" "2.48.0" "1"
echo
echo "─── AppRun test summary: $PASS_COUNT pass / $FAIL_COUNT fail ───"
if [[ $FAIL_COUNT -ne 0 ]]; then
+28
View File
@@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<!--
Tauri's macOS bundle defaults `hardenedRuntime` to true. Hardened
Runtime blocks camera/microphone hardware access unless the matching
entitlement is present here — regardless of Info.plist's
NSMicrophoneUsageDescription and regardless of wry's own WKUIDelegate
already granting the request at the WebKit/JS layer
(WryWebViewUIDelegate::request_media_capture_permission unconditionally
calls WKPermissionDecision::Grant). Without this entitlement, TCC
never even registers a request for the app — nothing shows up in
System Settings → Privacy & Security → Microphone to enable, because
the OS never saw a legitimately-entitled process ask.
-->
<key>com.apple.security.device.audio-input</key>
<true/>
<!--
Matches Info.plist's forward-looking NSCameraUsageDescription — no
current feature uses the camera, but ship the entitlement now so a
future getUserMedia({video: true}) call doesn't hit this same bug.
-->
<key>com.apple.security.device.camera</key>
<true/>
</dict>
</plist>
+12 -4
View File
@@ -704,9 +704,14 @@ fn sync_failure_is_torch_download(tail: &str) -> bool {
|| (low.contains("torch") && (low.contains("failed to download") || low.contains("failed to fetch")))
}
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124). ROCm 6.2 is
/// the current stable wheel set; overridable via OMNIVOICE_TORCH_INDEX.
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.2";
/// Default PyTorch ROCm wheel index for the opt-in AMD path (#124).
/// ROCm 6.4, not 6.2: the app's pinned `torch==2.8.0` (pyproject.toml) has no
/// build on the rocm6.2 index (it tops out at 2.5.1), so that index silently
/// failed the reinstall and left the default CUDA build in place — which runs
/// on CPU on an AMD GPU (#972). rocm6.4 carries a matching 2.8.0 build.
/// Overridable via OMNIVOICE_TORCH_INDEX (e.g. a `--find-links` URL for
/// distro-matched ROCm builds torch's own index doesn't carry).
const ROCM_TORCH_INDEX: &str = "https://download.pytorch.org/whl/rocm6.4";
/// `uv pip install` args that replace the default CUDA torch build with the AMD
/// ROCm wheel (#124). Opt-in (gated on OMNIVOICE_TORCH_VARIANT=rocm by the
@@ -1817,7 +1822,10 @@ mod tests {
assert!(args.iter().any(|a| a == "torch"));
assert!(args.iter().any(|a| a == "torchaudio"));
let i = args.iter().position(|a| a == "--index-url").expect("has --index-url");
assert!(args[i + 1].contains("rocm6.2"), "default index is the rocm6.2 wheel set");
// rocm6.4, not rocm6.2: rocm6.2's index tops out at torch 2.5.1 and
// can't satisfy the app's torch==2.8.0 pin (#972) — a regression to
// rocm6.2 here would silently resurrect the CPU-fallback bug.
assert!(args[i + 1].contains("rocm6.4"), "default index is the rocm6.4 wheel set (matches torch==2.8.0)");
}
#[test]
+153 -12
View File
@@ -79,9 +79,18 @@ pub const TRAY_ICON_RECORDING: &[u8] = include_bytes!("../icons/tray-recording.p
// applies on top.
// - Linux (WebKitGTK): media-stream must be enabled per-WebView and the
// permission request answered programmatically.
// - macOS (WKWebView): nothing to do here — wry grants media-capture to the
// app origin and the user-visible consent is the system TCC prompt driven
// by NSMicrophoneUsageDescription in src-tauri/Info.plist.
// - macOS (WKWebView): nothing to do here in code — wry's own WKUIDelegate
// (WryWebViewUIDelegate::request_media_capture_permission) already grants
// every media-capture request unconditionally at the WebKit/JS layer. But
// that alone isn't sufficient (#1013): Tauri's macOS bundle defaults
// `hardenedRuntime` to true, and Hardened Runtime blocks camera/microphone
// hardware access unless the matching entitlement is present — without it,
// TCC never even registers a request, so the app never appears in System
// Settings → Privacy & Security → Microphone for the user to enable. See
// src-tauri/entitlements.plist (wired in via tauri.conf.json's
// bundle.macOS.entitlements) for the actual grant; NSMicrophoneUsageDescription
// in Info.plist only supplies the *prompt text* TCC shows, it doesn't
// substitute for the entitlement.
/// True for origins the app itself serves: the Tauri custom-protocol origin
/// in production and the Vite dev server / loopback in `tauri dev`.
@@ -205,6 +214,120 @@ mod media_permission_tests {
}
}
// ── Windows: dictation pill must never take foreground focus (#982) ────────
//
// Windows counterpart of #287 (macOS auto-paste — don't steal focus). The
// pill is `.always_on_top(true).skip_taskbar(true)` and is documented above
// (see `grant_webview_media_permissions`) as "deliberately unfocused so the
// auto-paste lands in the target app" — true on macOS, but on Windows,
// showing an always-on-top top-level window gives it Win32 foreground
// activation by default (ordinary Windows window-manager behavior; macOS
// doesn't force-activate a shown window the same way). Nothing marked the
// pill non-activating, so on Windows it stole foreground on every show —
// the synthesized Ctrl+V from `simulate_paste` landed back in the pill
// instead of the app the user was dictating into, and because the pill
// wrongly held focus for the whole session the target app never got it back
// until the pill's auto-dismiss timer eventually hid it.
//
// Two pieces, both required (verified by reading how `.show()` is used at
// the call sites below — several are followed by an explicit `set_focus()`
// that would fight the style bit on its own):
// 1. WS_EX_NOACTIVATE on the HWND, applied once right after creation, so
// the OS never grants this window foreground activation implicitly.
// 2. `ShowWindow(SW_SHOWNOACTIVATE)` in place of `WebviewWindow::show()` at
// the pill's dictation-trigger call sites, and the explicit
// `set_focus()` calls at those same sites are skipped on Windows (the
// same way they already are on macOS below).
//
// The flag math (`with_noactivate_style`) is a plain function so it's
// unit-testable on every platform — the actual Win32 syscalls that use it
// are Windows-only and can't run under `cargo test` on a non-Windows runner.
/// `WS_EX_NOACTIVATE` (winuser.h: `#define WS_EX_NOACTIVATE 0x08000000L`).
/// Hardcoded rather than imported from the `windows` crate so `with_noactivate_style`
/// below stays free of the Windows-only dependency and is testable everywhere.
/// Only consumed by Windows-only code (or the platform-agnostic test module
/// below) — `#[allow(dead_code)]` elsewhere, same as `is_app_origin` above.
#[cfg_attr(not(windows), allow(dead_code))]
const WS_EX_NOACTIVATE_BIT: isize = 0x0800_0000;
/// OR `WS_EX_NOACTIVATE` into an existing extended window style, preserving
/// every other bit already set (topmost, layered, etc. — the pill's
/// `always_on_top(true)` sets one of these). Pure so it's unit-testable
/// without a real HWND. See module comment above for why this exists.
#[cfg_attr(not(windows), allow(dead_code))]
fn with_noactivate_style(current_ex_style: isize) -> isize {
current_ex_style | WS_EX_NOACTIVATE_BIT
}
/// Mark the pill's HWND `WS_EX_NOACTIVATE`, once, right after creation — this
/// holds for every later `.show()` regardless of call site (belt-and-braces
/// alongside `show_pill_noactivate` below, which some call sites also need
/// because they pair `.show()` with an explicit `set_focus()`).
#[cfg(target_os = "windows")]
fn mark_pill_noactivate(win: &tauri::WebviewWindow) {
use windows::Win32::UI::WindowsAndMessaging::{
GetWindowLongPtrW, SetWindowLongPtrW, GWL_EXSTYLE,
};
let Ok(hwnd) = win.hwnd() else {
log::warn!("pill: could not resolve HWND to apply WS_EX_NOACTIVATE (#982)");
return;
};
unsafe {
let current = GetWindowLongPtrW(hwnd, GWL_EXSTYLE);
SetWindowLongPtrW(hwnd, GWL_EXSTYLE, with_noactivate_style(current));
}
}
/// Show the pill without granting it foreground activation. Used instead of
/// `WebviewWindow::show()` at the pill's dictation-trigger call sites on
/// Windows — `.show()` maps to plain `ShowWindow(SW_SHOW)`, which relies on
/// the NOACTIVATE style alone to suppress activation; `SW_SHOWNOACTIVATE` is
/// the explicit, documented way to show a window without activating it and
/// costs nothing extra now that the style bit is also set (#982).
#[cfg(target_os = "windows")]
fn show_pill_noactivate(win: &tauri::WebviewWindow) {
use windows::Win32::UI::WindowsAndMessaging::{ShowWindow, SW_SHOWNOACTIVATE};
let Ok(hwnd) = win.hwnd() else {
log::warn!("pill: could not resolve HWND for non-activating show (#982)");
return;
};
unsafe {
let _ = ShowWindow(hwnd, SW_SHOWNOACTIVATE);
}
}
#[cfg(test)]
mod pill_noactivate_tests {
use super::{with_noactivate_style, WS_EX_NOACTIVATE_BIT};
#[test]
fn adds_noactivate_bit_without_clobbering_existing_style() {
// Stand-in for whatever bits the pill's always_on_top/skip_taskbar
// window already carries (e.g. WS_EX_TOPMOST = 0x00000008) —
// NOACTIVATE must be added on top, never replace them.
let topmost = 0x0000_0008isize;
let updated = with_noactivate_style(topmost);
assert_eq!(
updated & WS_EX_NOACTIVATE_BIT,
WS_EX_NOACTIVATE_BIT,
"NOACTIVATE bit must be set"
);
assert_eq!(updated & topmost, topmost, "pre-existing style bits must survive");
}
#[test]
fn idempotent_if_already_noactivate() {
assert_eq!(with_noactivate_style(WS_EX_NOACTIVATE_BIT), WS_EX_NOACTIVATE_BIT);
}
#[test]
fn matches_documented_win32_value() {
// winuser.h: #define WS_EX_NOACTIVATE 0x08000000L
assert_eq!(WS_EX_NOACTIVATE_BIT, 0x0800_0000);
}
}
// ── Tauri entry ───────────────────────────────────────────────────────────
#[cfg_attr(mobile, tauri::mobile_entry_point)]
@@ -334,9 +457,15 @@ pub fn run() {
.skip_taskbar(true)
.center()
.build();
if let Err(e) = result {
if let Err(e) = &result {
log::error!("Failed to create widget window: {e:?}");
}
// Windows: mark the pill non-activating right away so it holds
// for every later `.show()` regardless of call site (#982).
#[cfg(target_os = "windows")]
if let Ok(win) = &result {
mark_pill_noactivate(win);
}
}
app.manage(AppFlags {
@@ -370,12 +499,17 @@ pub fn run() {
if win.move_window(Position::BottomCenter).is_err() {
let _ = win.center();
}
// Windows: show without granting foreground activation
// (#982) — `.show()` on other platforms is unaffected.
#[cfg(target_os = "windows")]
show_pill_noactivate(&win);
#[cfg(not(target_os = "windows"))]
let _ = win.show();
// Don't steal focus on macOS: the simulated ⌘V from
// simulate_paste() must land in the app the user is
// dictating into — focusing the widget would swallow
// it (#287).
#[cfg(not(target_os = "macos"))]
// Don't steal focus on macOS or Windows: the simulated
// ⌘V/Ctrl+V from simulate_paste() must land in the app
// the user is dictating into — focusing the widget would
// swallow it (#287 macOS, #982 Windows).
#[cfg(not(any(target_os = "macos", target_os = "windows")))]
let _ = win.set_focus();
}
let _ = app_handle.emit("tray-dictate", ());
@@ -526,7 +660,9 @@ pub fn run() {
// + focus the widget BEFORE emitting tray-dictate so
// the user sees the pill instead of silent recording.
// Positioning mirrors the global-shortcut handler:
// bottom-center (WhisperFlow style).
// bottom-center (WhisperFlow style). Windows skips the
// focus (and uses a non-activating show) for the same
// reason the global-shortcut handler does — see #982.
if let Some(win) = app.get_webview_window("widget") {
if win.is_visible().unwrap_or(false) {
let _ = app.emit("tray-dictate-stop", ());
@@ -534,8 +670,13 @@ pub fn run() {
if win.move_window(Position::BottomCenter).is_err() {
let _ = win.center();
}
let _ = win.show();
let _ = win.set_focus();
#[cfg(target_os = "windows")]
show_pill_noactivate(&win);
#[cfg(not(target_os = "windows"))]
{
let _ = win.show();
let _ = win.set_focus();
}
let _ = app.emit("tray-dictate", ());
}
} else {
+2 -1
View File
@@ -85,7 +85,8 @@
],
"macOS": {
"minimumSystemVersion": "12.0",
"signingIdentity": "-"
"signingIdentity": "-",
"entitlements": "entitlements.plist"
}
},
"plugins": {
+8 -1
View File
@@ -46,8 +46,15 @@ export async function listEngines(): Promise<AllEnginesResponse> {
export async function selectEngine(
family: EngineFamily,
backendId: string,
modelId?: string,
): Promise<SelectEngineResponse> {
return apiPost<SelectEngineResponse>('/engines/select', { family, backend_id: backendId });
return apiPost<SelectEngineResponse>('/engines/select', {
family,
backend_id: backendId,
// Only mlx-audio's curated-model picker (#981) sets this — omit
// entirely rather than send `undefined`/null for every other call site.
...(modelId ? { model_id: modelId } : {}),
});
}
/**
+13
View File
@@ -40,6 +40,19 @@ interface EngineBackend {
effective_device?: EffectiveDevice;
routing_status?: RoutingStatus;
routing_reason?: string | null;
// #981 — mlx-audio ONLY: it multiplexes 7+ curated models behind one
// backend id, so its entry also carries the roster + current pick so
// Settings can render a model picker. Absent on every other backend.
curated_models?: CuratedModel[];
active_model_id?: string;
}
// #981 — one of mlx-audio's curated models (see backend
// MLXAudioBackend.CURATED_MODELS / _MLX_AUDIO_MODEL_LABELS).
export interface CuratedModel {
key: string;
label: string;
repo_id: string;
}
interface EngineFamilyResponse {
@@ -17,7 +17,7 @@ import { useTranslation } from 'react-i18next';
import { listEngines, getEngineHealth, selfTestEngine } from '../api/engines';
import { copyText } from '../utils/copyText';
import { ChevronRight } from 'lucide-react';
import { Badge, Button, Segmented, Table } from '../ui';
import { Badge, Button, Segmented, Select, Table } from '../ui';
import { cn } from '@/lib/utils';
import SupertonicLicenseDialog from './SupertonicLicenseDialog';
@@ -63,12 +63,18 @@ function reasonMentionsLicense(reason) {
*
* Props:
* - family: 'tts' | 'asr' | 'llm' default 'tts'
* - onSelect?: (family, backendId) => Promise<void> optional when
* provided, a "Use" button appears next to "Test engine" for
* - onSelect?: (family, backendId, modelId?) => Promise<void> optional
* when provided, a "Use" button appears next to "Test engine" for
* available, non-active rows. Lets the matrix double as an engine
* picker so Settings doesn't need a parallel table.
* picker so Settings doesn't need a parallel table. The optional third
* arg is set only by mlx-audio's curated-model picker (#981).
* - activeId?: string the currently-active backend id for this
* family. Used to render the "active" badge.
* - showFamilyTabs?: boolean default true. When false, the matrix is
* pinned to `family` no TTS/ASR/LLM switcher, and the header names
* the family ("ASR Engines") instead of the generic matrix title.
* Settings Engines stacks one pinned matrix per family so the ASR
* and LLM pickers are visible instead of tucked behind a tab.
*/
const FAMILY_META = {
tts: { label: 'TTS', icon: Cpu },
@@ -139,6 +145,10 @@ function normalizeEntry(entry) {
effective_device: entry.effective_device || null,
routing_status: entry.routing_status || null,
routing_reason: entry.routing_reason || null,
// #981 mlx-audio ONLY: the curated-model roster + current pick.
// null/absent on every other backend, which never renders a picker.
curated_models: Array.isArray(entry.curated_models) ? entry.curated_models : null,
active_model_id: entry.active_model_id || null,
};
}
@@ -153,8 +163,10 @@ export default function EngineCompatibilityMatrix({
family = 'tts',
onSelect = null,
activeId = null,
// Test-friendly overrides let the RTL suite mock the API layer
// without resorting to module-level vi.mock incantations.
showFamilyTabs = true,
// Injectable API layer lets the RTL suite mock it without module-level
// vi.mock incantations, and lets EnginesTab share one in-flight
// GET /engines across its stacked per-family matrices.
apiListEngines = listEngines,
apiGetEngineHealth = getEngineHealth,
apiSelfTestEngine = selfTestEngine,
@@ -291,6 +303,18 @@ export default function EngineCompatibilityMatrix({
setTimeout(() => setCopiedId((c) => (c === id ? null : c)), 1500);
}, []);
// #981 mlx-audio's curated-model picker. Reuses the same onSelect the
// "Use" button calls, with the curated model key as the optional third
// arg, then reloads so active_model_id reflects the new pick immediately.
const changeModel = useCallback(
async (id, modelId) => {
if (!onSelect || !modelId) return;
await onSelect(activeFamily, id, modelId);
reload();
},
[onSelect, activeFamily, reload],
);
const COLUMNS = [
{ key: 'name', label: t('engines.matrixTitle').split(' ')[0] || 'Engine', flex: 3 },
{ key: 'status', label: t('engines.status'), width: 130, align: 'center' },
@@ -330,12 +354,19 @@ export default function EngineCompatibilityMatrix({
// TTS-05: the license dialog registered for the engine awaiting acceptance
// (or null). Capitalized so JSX renders it as a component below.
const LicenseDialog = licenseDialogFor ? LICENSE_DIALOGS[licenseDialogFor] : null;
// Pinned mode: the header names the family (with its icon) since there is
// no switcher to say which family this table is.
const familyMeta = FAMILY_META[activeFamily] || FAMILY_META.tts;
const TitleIcon = showFamilyTabs ? Layers : familyMeta.icon;
return (
<section className="engine-matrix flex flex-col gap-[var(--space-3,8px)]">
<header className="engine-matrix__head flex items-center justify-between gap-[12px]">
<h3 className="engine-matrix__title inline-flex items-center gap-[6px] m-0 text-[13px] font-semibold text-[color:var(--chrome-fg,currentColor)]">
<Layers size={14} /> {t('engines.matrixTitle')}
<TitleIcon size={14} />{' '}
{showFamilyTabs
? t('engines.matrixTitle')
: t('engines.familyMatrixTitle', { family: familyMeta.label })}
</h3>
<Button
size="sm"
@@ -348,7 +379,7 @@ export default function EngineCompatibilityMatrix({
</Button>
</header>
{families.length > 1 && (
{showFamilyTabs && families.length > 1 && (
<Segmented
size="sm"
value={activeFamily}
@@ -413,6 +444,35 @@ export default function EngineCompatibilityMatrix({
<code className="engine-matrix__id font-mono text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
{b.id}
</code>
{/* #981 mlx-audio multiplexes 7+ curated models behind this
one backend id (Kokoro, CSM, OuteTTS, ); without this
picker there's no way to load anything but the default
(Kokoro) even after downloading a different model's
weights in Settings Models. Disabled while the row
itself isn't available/selectable, matching the "Use"
button's gating. */}
{b.curated_models && b.curated_models.length > 0 && (
<div className="engine-matrix__model-picker flex items-center gap-[6px] mt-[2px]">
<span className="text-[11px] text-[color:var(--chrome-fg-muted,#888)]">
{t('engines.curatedModelLabel')}
</span>
<Select
size="sm"
className="w-auto min-w-[150px]"
value={b.active_model_id || ''}
disabled={!onSelect || !b.available}
onChange={(e) => changeModel(b.id, e.target.value)}
aria-label={t('engines.curatedModelAria', { engine: b.display_name })}
data-testid={`curated-model-select-${b.id}`}
>
{b.curated_models.map((m) => (
<option key={m.key} value={m.key}>
{m.label}
</option>
))}
</Select>
</div>
)}
{/* For available rows, show install_hint inline (one line usually
a parenthetical like "(bundled — no extra install needed)").
For unavailable rows, collapse reason + install_hint + last_error
+7
View File
@@ -376,6 +376,13 @@ export default function Header({
<div className="flex flex-col gap-[1px] min-w-0">
<span className="text-[12px] text-[var(--color-fg)] font-medium">
{m.name}
{/* Resident-but-not-routed engine (e.g. OmniVoice still in
VRAM after switching to another backend) say so. */}
{m.is_active_engine === false && (
<span className="ml-[6px] text-[10px] font-normal text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
{t('header.model_not_active')}
</span>
)}
</span>
<span className="text-[10px] text-[var(--color-fg-subtle)] [font-family:var(--font-mono)]">
{m.device} {m.vram_mb > 0 ? `· ${m.vram_mb.toFixed(0)} MB` : ''}
-27
View File
@@ -73,9 +73,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
[t],
);
const donateLabel = t('donate.pill', { defaultValue: 'Support OmniVoice' });
const donateActive = mode === 'donate';
// `nav-rail` is retained purely as the layout hook the (out-of-scope)
// `.app-container > .nav-rail` grid rules position by; all visual styling now
// lives in the utilities below. Border flips to the inner edge when on the right.
@@ -84,17 +81,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
? '[border-left:1px_solid_var(--chrome-border)]'
: '[border-right:1px_solid_var(--chrome-border)]';
// Quiet "Support" pill (was `.rail-btn.donate-pill`): neutral at rest, warms to
// the accent on hover/active.
const donateState = donateActive
? 'text-[var(--chrome-accent)] bg-[var(--chrome-accent-bg)] [border:1px_solid_var(--chrome-accent-border)]'
: 'bg-transparent text-[var(--chrome-fg-dim)] [border:1px_solid_transparent] hover:bg-[color-mix(in_srgb,var(--chrome-accent)_10%,transparent)] hover:text-[var(--chrome-accent)]';
const heartBase =
'text-[16px] leading-none [transition:filter_0.16s,opacity_0.16s,transform_0.16s] group-hover:[transform:scale(1.1)] motion-reduce:[transition:none] motion-reduce:group-hover:[transform:none]';
const heartState = donateActive
? 'opacity-100 [filter:grayscale(0)]'
: 'opacity-75 [filter:grayscale(0.55)] group-hover:opacity-100 group-hover:[filter:grayscale(0)]';
return (
<aside
className={`nav-rail z-50 flex select-none flex-col items-center gap-[6px] bg-[var(--chrome-bg)] py-[8px] ${asideBorder}`}
@@ -111,19 +97,6 @@ export default function NavRail({ mode, setMode, side = 'left', onFlipSide }) {
))}
</div>
<div className="flex flex-col items-center gap-[4px]">
{/* Quiet "Support" pill warms to the accent on hover, opens the
donate page. Sits with the footer nav (Settings / flip). (#007) */}
<button
onClick={() => setMode('donate')}
title={donateLabel}
aria-label={donateLabel}
className={`${RAIL_BTN_BASE} ${donateState}`}
>
<span className={`${heartBase} ${heartState}`} aria-hidden="true">
🩷
</span>
<span className={railLabelCls(side)}>{donateLabel}</span>
</button>
{footerItems.map((it) => (
<RailBtn
key={it.id}
+17 -3
View File
@@ -351,7 +351,14 @@ function WaveformTimeline(
console.warn('WebKit audio decode not supported, using media element directly');
try {
const emptyPeaks = new Float32Array(1000).fill(0);
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
// Don't rely solely on the 'ready' event firing again for this
// recovery load the play button stayed permanently disabled
// when it didn't (the waveform still rendered from the peaks, so
// there was no visible sign anything was wrong). Confirm
// readiness explicitly once this load settles either way.
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
.then(() => setReady(true))
.catch(() => setReady(true));
} catch (_) {
setReady(true);
}
@@ -372,7 +379,12 @@ function WaveformTimeline(
})
.then((audioBuffer) => {
const channelData = audioBuffer.getChannelData(0);
ws.load(undefined, [channelData], audioBuffer.duration);
// Same explicit-readiness guard as the NotSupportedError branch
// above don't depend on the 'ready' event re-firing for this
// manually-decoded recovery load.
Promise.resolve(ws.load(undefined, [channelData], audioBuffer.duration))
.then(() => setReady(true))
.catch(() => setReady(true));
})
.catch((decodeErr) => {
// HTTP 404 on the companion audio means the source file is
@@ -391,7 +403,9 @@ function WaveformTimeline(
console.warn('Audio decode fallback failed, loading with empty peaks:', decodeErr);
try {
const emptyPeaks = new Float32Array(1000).fill(0);
ws.load(undefined, [emptyPeaks], mediaEl.duration || 60);
Promise.resolve(ws.load(undefined, [emptyPeaks], mediaEl.duration || 60))
.then(() => setReady(true))
.catch(() => setReady(true));
} catch (_) {
setLoadError(true);
}
@@ -0,0 +1,43 @@
import { describe, it, expect } from 'vitest';
import { readFileSync } from 'node:fs';
import path from 'node:path';
// Regression guard: the dub editor's play button stayed permanently disabled
// (disabled={!ready}) whenever the initial WaveSurfer decode failed and the
// component fell back to a peaks-only ws.load(undefined, [peaks], duration)
// call — the waveform still rendered from those peaks (so nothing looked
// visibly broken), but `ready` was only ever set from the 'ready' event
// re-firing on that recovery load, which this component's own error-handling
// code never actually confirmed. Each fallback load must now explicitly
// confirm readiness once it settles, instead of assuming the event fires.
//
// Driving WaveSurfer + a real decode-failure/recovery sequence through jsdom
// is brittle (see WaveformTimeline.unlock.test.js), so this is a
// source-level contract guard, same house pattern: every `ws.load(undefined,
// ...)` recovery call inside the `ws.on('error', ...)` handler must be
// followed by an explicit setReady(true) confirmation.
const src = readFileSync(
path.resolve(process.cwd(), 'src/components/WaveformTimeline.jsx'),
'utf8',
);
describe('WaveformTimeline error-recovery ready confirmation', () => {
it("confirms readiness explicitly after every fallback ws.load() call, not just via the 'ready' event", () => {
const errorHandler = /ws\.on\('error', \(err\) => \{([\s\S]*?)\n \}\);/.exec(src)?.[1];
expect(errorHandler, "ws.on('error', ...) handler not found").toBeTruthy();
// Every recovery load in this handler passes peaks explicitly
// (`ws.load(undefined, [...], ...)`) — each occurrence must be
// immediately confirmed ready via a .then()/.catch() pair (or an
// unconditional setReady in a synchronous catch), not left to hope the
// 'ready' event re-fires on its own.
const loadCalls = [...errorHandler.matchAll(/ws\.load\(undefined, \[[^\]]*\][^)]*\)/g)];
expect(loadCalls.length).toBeGreaterThanOrEqual(3);
for (const match of loadCalls) {
const tail = errorHandler.slice(match.index, match.index + 220);
expect(tail, `no readiness confirmation after: ${match[0]}`).toMatch(/setReady\(true\)/);
}
});
});
@@ -147,6 +147,13 @@ export default function DesignMethodPanel({
{Object.entries(CATEGORIES).map(([key, options]) => {
const many = options.length > 6;
const optLabel = (val) => {
// #983: a profile/localStorage-restored vdStates can carry a
// partial shape (missing category keys) val is then undefined
// here even though the 'Auto' check above only catches the
// literal sentinel. Guard before .replace() rather than crash;
// 'Auto' matches how the rest of the component (the ternary
// above, the chip/select fallbacks) treats an unset category.
if (typeof val !== 'string' || !val) return 'Auto';
const tKey = `clone.opt_${val.replace(/[ -]/g, '_')}`;
const tl = t(tKey);
return tl !== tKey ? tl : val;
@@ -180,9 +187,13 @@ export default function DesignMethodPanel({
aria-label={t(`clone.cat_${key}`)}
>
{options.map((opt, i) => {
const optTKey = `clone.opt_${opt.replace(/[ -]/g, '_')}`;
// `opt` is always a hardcoded CATEGORIES string today,
// never undefined guarded anyway for consistency with
// the identical pattern above (#983).
const safeOpt = typeof opt === 'string' && opt ? opt : 'Auto';
const optTKey = `clone.opt_${safeOpt.replace(/[ -]/g, '_')}`;
const optTl = t(optTKey);
const optLabel = optTl !== optTKey ? optTl : opt;
const optLabel = optTl !== optTKey ? optTl : safeOpt;
const checked = vdStates[key] === opt;
// Roving tabindex: the checked chip is the group's
// single tab stop (first chip if nothing matches).
@@ -0,0 +1,68 @@
import React from 'react';
import { describe, it, expect, vi } from 'vitest';
import { render, screen } from '@testing-library/react';
import DesignMethodPanel from './DesignMethodPanel';
// #983: "Cannot read properties of undefined (reading 'replace')" the
// identity panel crashed whenever vdStates was missing one of the 6
// CATEGORIES keys (a design profile saved by an older/foreign client, or a
// stale localStorage shape). The label helper called `val.replace(...)` on
// an undefined category value. This regression-tests the render guard added
// to DesignMethodPanel.jsx directly, independent of the upstream data-shape
// fixes in useProfiles.js / useAppData.js / profiles.py.
// A minimal i18next-compatible mock: returns the defaultValue if given, else
// echoes the key back (mirrors i18next's behavior for a missing translation,
// which is what `optLabel`'s `tl !== tKey` check relies on).
const t = (key, opts) => opts?.defaultValue ?? key;
function setup(vdStates, props = {}) {
return render(
<DesignMethodPanel
t={t}
describeText=""
onDescribeChange={vi.fn()}
describeMatchedAny={false}
describeUnmatched={[]}
chipPersonalities={[]}
activePersonality={null}
applyPersonality={vi.fn()}
applyPreset={vi.fn()}
identityOpen={true}
setIdentityOpen={vi.fn()}
identityRecipe="test recipe"
vdStates={vdStates}
setVdStates={vi.fn()}
onChipKeyDown={vi.fn()}
showSaveProfile={false}
setShowSaveProfile={vi.fn()}
profileName=""
setProfileName={vi.fn()}
handleSaveDesignProfile={vi.fn()}
instruct=""
language="Auto"
{...props}
/>,
);
}
describe('DesignMethodPanel — #983 partial vdStates crash', () => {
it('does not throw when vdStates is missing 5 of the 6 CATEGORIES keys', () => {
// Only Gender is set Age, Pitch, Style, EnglishAccent, ChineseDialect
// are all undefined, exercising both the chip-based and <select>-based
// ("many" options) render paths.
expect(() => setup({ Gender: 'male' })).not.toThrow();
});
it('does not throw when vdStates is a fully empty object', () => {
expect(() => setup({})).not.toThrow();
});
it('still renders category labels and the identity recipe with a partial shape', () => {
const { container } = setup({ Gender: 'male' });
expect(screen.getByText('test recipe')).toBeInTheDocument();
// The label text sits alongside a sibling <span> kicker, so assert via
// textContent rather than getByText's exact-node matching.
expect(container.textContent).toContain('clone.cat_Gender');
});
});
@@ -91,8 +91,13 @@ export default function CommunityZone({
name: r.name,
}),
);
} catch {
flash(t('gallery.use_failed', { defaultValue: 'Could not add that voice.' }));
} catch (e) {
flash(
t('gallery.use_failed', {
message: e?.message || String(e),
defaultValue: 'Could not create that voice: {{message}}',
}),
);
}
}}
onDesign={(item) =>
@@ -110,7 +110,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
const r = await searchYoutube(q, 'import', 10);
setResults(r.results || []);
} catch (e) {
flash(t('gallery.search_failed', { defaultValue: 'Search failed.' }));
flash(
t('gallery.search_failed', {
message: e?.message || String(e),
defaultValue: 'Search failed: {{message}}',
}),
);
} finally {
setIsSearching(false);
}
@@ -149,7 +154,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
await uploadVoiceClip(fd);
reload();
} catch (err) {
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
flash(
t('gallery.upload_failed', {
message: err?.message || String(err),
defaultValue: 'Upload failed: {{message}}',
}),
);
} finally {
if (fileRef.current) fileRef.current.value = '';
}
@@ -165,7 +175,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
}),
);
} catch (e) {
flash(t('gallery.save_failed', { defaultValue: 'Could not save profile.' }));
flash(
t('gallery.save_failed', {
message: e?.message || String(e),
defaultValue: 'Could not save profile: {{message}}',
}),
);
}
};
@@ -179,8 +194,13 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
try {
await deleteGalleryVoice(v.id);
reload();
} catch {
/* noop */
} catch (e) {
flash(
t('gallery.delete_failed', {
message: e?.message || String(e),
defaultValue: 'Could not delete: {{message}}',
}),
);
}
};
@@ -191,7 +211,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
const file = new File([blob], `${v.name}.wav`, { type: 'audio/wav' });
setTrimming({ voice: v, file });
} catch (e) {
flash(t('gallery.trim_load_failed', { defaultValue: 'Could not load audio for trimming.' }));
flash(
t('gallery.trim_load_failed', {
message: e?.message || String(e),
defaultValue: 'Could not load audio for trimming: {{message}}',
}),
);
}
};
@@ -209,7 +234,12 @@ export default function ImportsZone({ t, playingId, loadingPreviewId, onPlayGall
reload();
setTrimming(null);
} catch (e) {
flash(t('gallery.upload_failed', { defaultValue: 'Upload failed.' }));
flash(
t('gallery.upload_failed', {
message: e?.message || String(e),
defaultValue: 'Upload failed: {{message}}',
}),
);
}
};
@@ -0,0 +1,152 @@
/**
* Settings Models tab OpenAI-compatible remote ASR panel (#877).
*
* A path to Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's
* own Whisper API today, without waiting on transformers to ship a direct
* Qwen3-ASR integration. Configures the `openai-compat-asr` backend's
* base_url/model/api_key; activating it as the active ASR engine still needs
* `OMNIVOICE_ASR_BACKEND=openai-compat-asr` (no in-app ASR engine picker
* exists yet for any ASR backend this panel only configures this one).
*
* Endpoints (loopback-only):
* GET /api/settings/asr-openai-compat {base_url, model, has_key}
* PUT /api/settings/asr-openai-compat body {base_url?, model?, api_key?}
* ('' clears api_key; omitted/null leaves it unchanged never returned)
*/
import React, { useCallback, useEffect, useState } from 'react';
import { Mic } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { apiJson, apiFetch } from '../../api/client';
import { SettingsSection, SettingRow, SettingsInput } from './primitives';
import { Button } from '../../ui';
export default function AsrOpenAICompatPanel() {
const { t } = useTranslation();
const [baseUrl, setBaseUrl] = useState('');
const [model, setModel] = useState('');
const [apiKey, setApiKey] = useState('');
const [hasKey, setHasKey] = useState(false);
const [saving, setSaving] = useState(false);
const [error, setError] = useState(null);
const refresh = useCallback(async () => {
setError(null);
try {
const d = await apiJson('/api/settings/asr-openai-compat');
setBaseUrl(d?.base_url || '');
setModel(d?.model || '');
setHasKey(Boolean(d?.has_key));
setApiKey(''); // the key is never returned the field always starts blank
} catch (e) {
setError(e?.message || t('models.asrOpenAICompatLoadError'));
}
}, [t]);
useEffect(() => {
refresh();
}, [refresh]);
const save = async () => {
setSaving(true);
setError(null);
try {
const res = await apiFetch('/api/settings/asr-openai-compat', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
base_url: baseUrl,
model,
// Only send api_key when the user actually typed something
// an untouched field must leave the stored key unchanged, not
// clear it (the field is always blank on load, so "unchanged"
// and "empty" would otherwise be indistinguishable).
...(apiKey ? { api_key: apiKey } : {}),
}),
});
const d = await res.json();
setBaseUrl(d.base_url || '');
setModel(d.model || '');
setHasKey(Boolean(d.has_key));
setApiKey('');
} catch (e) {
setError(e?.message || t('models.asrOpenAICompatSaveError'));
} finally {
setSaving(false);
}
};
return (
<SettingsSection
icon={Mic}
title={t('models.asrOpenAICompatTitle')}
description={t('models.asrOpenAICompatDescription')}
>
{error && (
<div className="perfpanel__error" role="alert">
{error}
</div>
)}
<SettingRow
stack
title={t('models.asrOpenAICompatBaseUrlTitle')}
hint={t('models.asrOpenAICompatBaseUrlHint')}
control={
<SettingsInput
mono
type="text"
value={baseUrl}
onChange={(e) => setBaseUrl(e.target.value)}
placeholder="http://localhost:8000/v1"
data-testid="asr-openai-compat-base-url"
/>
}
/>
<SettingRow
stack
title={t('models.asrOpenAICompatModelTitle')}
control={
<SettingsInput
mono
type="text"
value={model}
onChange={(e) => setModel(e.target.value)}
placeholder="whisper-1"
data-testid="asr-openai-compat-model"
/>
}
/>
<SettingRow
stack
title={t('models.asrOpenAICompatApiKeyTitle')}
hint={
hasKey ? t('models.asrOpenAICompatKeyConfigured') : t('models.asrOpenAICompatApiKeyHint')
}
control={
<>
<SettingsInput
mono
type="password"
value={apiKey}
onChange={(e) => setApiKey(e.target.value)}
placeholder={hasKey ? '••••••••' : t('models.asrOpenAICompatApiKeyOptional')}
data-testid="asr-openai-compat-api-key"
/>
<Button
variant="subtle"
size="sm"
onClick={save}
loading={saving}
disabled={saving}
data-testid="asr-openai-compat-save"
>
{t('common.save')}
</Button>
</>
}
/>
</SettingsSection>
);
}
@@ -1,12 +1,19 @@
import React, { useCallback } from 'react';
import React, { useCallback, useRef } from 'react';
import { toast } from 'react-hot-toast';
import { useTranslation } from 'react-i18next';
import { addBreadcrumb } from '../../utils/breadcrumbs';
import { selectEngine } from '../../api/engines';
import { listEngines, selectEngine } from '../../api/engines';
import { notifyEngineSelected } from '../../utils/engineSelectToast';
import EngineCompatibilityMatrix from '../EngineCompatibilityMatrix';
import { SETTINGS_SECTION_SURFACE } from './primitives';
/** One pinned matrix per family, stacked in this order. ASR used to be
* reachable only through the matrix's family tabs, which read as a
* TTS-only table README even promised a Settings ASR picker that
* didn't exist (UX gap found during #877). Every family now gets a
* visible picker; `OMNIVOICE_*_BACKEND` env vars still win over any pick. */
const FAMILIES = ['tts', 'asr', 'llm'];
export default function EnginesTab() {
const { t } = useTranslation();
@@ -17,10 +24,12 @@ export default function EnginesTab() {
//
// Review mode (the staged-checkpoint nudges) moved to Settings General.
const onSelect = useCallback(
async (family, backendId) => {
// modelId is only ever set by mlx-audio's curated-model picker (#981)
// every other call site (the "Use" button) omits it.
async (family, backendId, modelId) => {
try {
addBreadcrumb(`engine:${family}=${backendId}`);
const r = await selectEngine(family, backendId);
const r = await selectEngine(family, backendId, modelId);
// Consume the routing echo: warn (not a bare success) when the pick
// lands on a CPU fallback on this host. See notifyEngineSelected.
notifyEngineSelected(r, t, family);
@@ -31,9 +40,32 @@ export default function EnginesTab() {
[t],
);
// The stacked matrices all consume the same GET /engines payload share
// one in-flight request so opening the tab probes every engine once, not
// once per family. A per-matrix Refresh after the shared promise settles
// still triggers a fresh fetch.
const inflightList = useRef(null);
const listEnginesShared = useCallback(() => {
if (!inflightList.current) {
inflightList.current = listEngines().finally(() => {
inflightList.current = null;
});
}
return inflightList.current;
}, []);
return (
<section className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
<EngineCompatibilityMatrix family="tts" onSelect={onSelect} />
</section>
<>
{FAMILIES.map((family) => (
<section key={family} className={SETTINGS_SECTION_SURFACE} data-slot="settings-section">
<EngineCompatibilityMatrix
family={family}
showFamilyTabs={false}
onSelect={onSelect}
apiListEngines={listEnginesShared}
/>
</section>
))}
</>
);
}
@@ -0,0 +1,88 @@
import React from 'react';
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
// Keep toast side-channels out of the test (timers, portals).
vi.mock('react-hot-toast', () => ({
default: { error: vi.fn(), success: vi.fn() },
toast: Object.assign(vi.fn(), { error: vi.fn(), success: vi.fn() }),
}));
vi.mock('../../api/engines', () => ({
listEngines: vi.fn(),
selectEngine: vi.fn(),
getEngineHealth: vi.fn(),
selfTestEngine: vi.fn(),
}));
import { listEngines, selectEngine } from '../../api/engines';
import EnginesTab from './EnginesTab';
function entry(id, name) {
return {
id,
display_name: name,
available: true,
reason: null,
install_hint: null,
last_error: null,
isolation_mode: 'in-process',
gpu_compat: ['cpu'],
};
}
const ENGINES = {
tts: { active: 'omnivoice', backends: [entry('omnivoice', 'OmniVoice (test)')] },
asr: {
active: 'whisperx',
backends: [
entry('whisperx', 'WhisperX (test)'),
entry('openai-compat-asr', 'OpenAI-compatible ASR (test)'),
],
},
llm: { active: 'off', backends: [entry('off', 'Off (test)')] },
};
describe('EnginesTab', () => {
beforeEach(() => {
vi.clearAllMocks();
listEngines.mockResolvedValue(ENGINES);
});
it('renders a pinned picker per family — TTS, ASR and LLM all visible at once', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('WhisperX (test)'));
// One named section per family (the ASR picker used to be tucked behind
// a family tab inside a single TTS-titled matrix no picker to find).
expect(screen.getByText('TTS Engines')).toBeInTheDocument();
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
expect(screen.getByText('LLM Engines')).toBeInTheDocument();
// Pinned matrices render no family switcher.
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
});
it('the stacked matrices share one GET /engines on mount', async () => {
render(<EnginesTab />);
await waitFor(() => screen.getByText('WhisperX (test)'));
expect(listEngines).toHaveBeenCalledTimes(1);
});
it('clicking Use on an ASR engine selects it with family="asr"', async () => {
selectEngine.mockResolvedValue({
family: 'asr',
active: 'openai-compat-asr',
env_override: false,
routing_status: 'cpu_only',
effective_device: 'cpu',
routing_reason: null,
});
render(<EnginesTab />);
await waitFor(() => screen.getByText('OpenAI-compatible ASR (test)'));
fireEvent.click(screen.getByRole('button', { name: /use openai-compatible asr \(test\)/i }));
await waitFor(() => {
expect(selectEngine).toHaveBeenCalledWith('asr', 'openai-compat-asr', undefined);
});
});
});
+7 -1
View File
@@ -8,6 +8,7 @@ import { listExportHistory } from '../api/exports';
import { modelStatus as apiModelStatus } from '../api/system';
import { useModelStatus } from '../api/hooks';
import useRealtimeEvents from './useRealtimeEvents';
import { mergeDescribedAttrs } from '../utils/voiceInstruct';
/**
* Encapsulates all data-loading effects, localStorage persistence,
@@ -174,7 +175,12 @@ export default function useAppData() {
setDefineMethod('design');
} else if (saved.mode) setMode(saved.mode);
if (saved.defineMethod) setDefineMethod(saved.defineMethod);
if (saved.vdStates) setVdStates(saved.vdStates);
// #983: legacy localStorage state had no shape validation at all — a
// partial/corrupt saved.vdStates crashed DesignMethodPanel on restore.
// Mirror useProfiles.js's guard: require a plain object, then complete
// it to the full CATEGORIES shape (missing/unknown keys → 'Auto').
if (saved.vdStates && typeof saved.vdStates === 'object')
setVdStates(mergeDescribedAttrs(saved.vdStates));
if (saved.language) setLanguage(saved.language);
if (saved.isSidebarCollapsed !== undefined) setIsSidebarCollapsed(saved.isSidebarCollapsed);
if (saved.sidebarTab) setSidebarTab(saved.sidebarTab);
+40 -4
View File
@@ -11,7 +11,11 @@ import { generateSpeech, audioUrlWithCacheBust } from '../api/generate';
import { apiFetch } from '../api/client';
import { playBlobAudio } from '../utils/media';
import { PRESETS } from '../utils/constants';
import { instructToFormValue } from '../utils/voiceInstruct';
import {
instructToFormValue,
mergeDescribedAttrs,
buildDesignInstruct,
} from '../utils/voiceInstruct';
import { askConfirm } from '../utils/dialog';
import { toast } from 'react-hot-toast';
import { recordValueMoment } from '../utils/donationMoments';
@@ -55,7 +59,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
const safeBlob = new Blob([arrBuf], { type: refAudio.type });
formData.append('ref_audio', safeBlob, refAudio.name || 'profile.wav');
formData.append('ref_text', refText);
formData.append('instruct', instruct);
// #1010: the backend only sanitizes instruct on save for kind='design'
// profiles — a clone profile (this call always creates kind='clone')
// would silently persist an unsupported free-text instruct and then
// 400 every single time it's used to generate. Filter here too.
const { instruct: safeInst } = buildDesignInstruct({}, instruct);
formData.append('instruct', safeInst);
formData.append('language', language);
try {
await createProfile(formData);
@@ -96,7 +105,12 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
if (profile.kind === 'design' && profile.vd_states) {
try {
const parsed = JSON.parse(profile.vd_states);
if (parsed && typeof parsed === 'object') setVdStates(parsed);
// #983: a profile saved by an older/foreign client (or hand-edited)
// can carry a partial shape — mergeDescribedAttrs (already used for
// the "describe your voice" restore path) guarantees every
// CATEGORIES key is present, defaulting missing/unknown ones to
// 'Auto', so DesignMethodPanel never sees an undefined category.
if (parsed && typeof parsed === 'object') setVdStates(mergeDescribedAttrs(parsed));
} catch {
/* malformed stored state — sliders keep their current values */
}
@@ -199,6 +213,25 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
fin_prof = '';
}
// #1010: this instruct string comes straight from segment/preset data,
// never through the validator-safe builder — a preset's raw attrs or a
// free-text style field can carry phrases outside the active engine's
// supported instruct vocabulary, 400ing instead of previewing. Same
// client-side guard useTTS.js already applies to the clone path.
if (fin_inst) {
const { instruct: safeInst, unsupported, duplicates } = buildDesignInstruct({}, fin_inst);
if (unsupported.length) {
toast(t('tts_errors.ignored_unsupported', { items: unsupported.join(', ') }), {
icon: '⚠️',
});
}
if (duplicates.length) {
toast(t('tts_errors.ignored_duplicate', { items: duplicates.join(', ') }), {
icon: '⚠️',
});
}
fin_inst = safeInst;
}
if (fin_prof) formData.append('profile_id', fin_prof);
if (fin_inst) formData.append('instruct', fin_inst);
const fin_lang = seg.target_lang || dubLang;
@@ -240,7 +273,10 @@ export default function useProfiles({ loadHistory, loadProfiles }) {
: item.text
: '';
formData.append('ref_text', extractedText);
formData.append('instruct', item.instruct || '');
// #1010: same guard as handleSaveProfile — this always creates a
// kind='clone' profile, which the backend never sanitizes on save.
const { instruct: safeHistInst } = buildDesignInstruct({}, item.instruct || '');
formData.append('instruct', safeHistInst);
formData.append('language', item.language || 'Auto');
if (item.seed !== undefined && item.seed !== null) {
formData.append('seed', item.seed);
+29 -9
View File
@@ -1188,8 +1188,8 @@
"no_matches": "No voices match these filters.",
"load_more": "Load more",
"saved_as_profile": "Added \"{{name}}\" to your voices.",
"use_failed": "Could not create that voice — the engine may be loading.",
"preview_failed": "Preview unavailable — the voice engine may still be loading.",
"use_failed": "Could not create that voice: {{message}}",
"preview_failed": "Preview unavailable: {{message}}",
"import_explainer": "Paste a URL you have the rights to (or upload a file), trim the part you need, and save it as a voice. You are responsible for the licensing of anything you import.",
"import_placeholder": "Paste a video/audio URL, or type to search…",
"imported_clip": "Imported clip",
@@ -1199,11 +1199,12 @@
"no_imports": "Nothing imported yet. Paste a URL above to get started.",
"search_results": "{{count}} results",
"download_failed": "Download failed: {{msg}}",
"search_failed": "Search failed.",
"upload_failed": "Upload failed.",
"save_failed": "Could not save profile.",
"search_failed": "Search failed: {{message}}",
"upload_failed": "Upload failed: {{message}}",
"save_failed": "Could not save profile: {{message}}",
"confirm_delete": "Delete \"{{name}}\"?",
"trim_load_failed": "Could not load audio for trimming.",
"delete_failed": "Could not delete: {{message}}",
"trim_load_failed": "Could not load audio for trimming: {{message}}",
"delete": "Delete",
"community_empty": "No community voices loaded yet — connect to the internet and reopen, or be the first to submit one.",
"community_explainer": "Designed presets and recorded voices shared by the community, loaded from the omnivoice-gallery.",
@@ -1291,7 +1292,11 @@
"try_dictation": "Try dictation",
"next_try_dictation": "Next: Try dictation",
"step_aria": "Step {{num}}: {{label}}",
"step_completed": "completed"
"step_completed": "completed",
"mirror_rescue_title": "Can't reach Hugging Face? Use a mirror",
"mirror_rescue_hint": "Model downloads switch to the mirror immediately — no restart needed during setup. hf-mirror.com is the community mirror commonly used in China.",
"mirror_apply": "Apply & re-check",
"mirror_apply_error": "Could not save the mirror setting."
},
"donate": {
"title": "Support",
@@ -1549,6 +1554,7 @@
"loading": "Loading engines…",
"refresh": "Refresh",
"matrixTitle": "Engine Compatibility Matrix",
"familyMatrixTitle": "{{family}} Engines",
"loadFailed": "Failed to load engines: {{message}}",
"couldNotLoad": "Could not load engines: {{message}}",
"retry": "Retry",
@@ -1588,7 +1594,9 @@
"routingUnknown": "Unknown",
"routingEffectiveChip": "Runs on {{device}} on this machine",
"routingCaveatTitle": "GPU selected, but: {{reason}}",
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}"
"selectCpuFallback": "{{engine}}: running on CPU — {{reason}}",
"curatedModelLabel": "Model",
"curatedModelAria": "Model for {{engine}}"
},
"errors": {
"title": "This tab hit a snag.",
@@ -1759,6 +1767,7 @@
"flush": "Flush",
"loaded_models": "Loaded Models",
"no_models": "No models loaded",
"model_not_active": "not active — safe to unload",
"unload": "Unload",
"flush_caches": "Flush caches",
"unload_all_flush": "Unload all + flush"
@@ -2007,7 +2016,18 @@
"mirror_preset_hint": "On a restricted network, route model downloads through a mirror. Leave empty for the official endpoint.",
"mirror_restart_note": "Model Store downloads use the new mirror immediately. Only model loads (transformers) pick it up after a restart.",
"mirror_load_error": "Failed to load mirror setting",
"mirror_save_error": "Failed to save"
"mirror_save_error": "Failed to save",
"asrOpenAICompatTitle": "OpenAI-compatible ASR (remote server)",
"asrOpenAICompatDescription": "Point transcription at Qwen3-ASR, a self-hosted FunASR/SenseVoice server, or OpenAI's own API.",
"asrOpenAICompatBaseUrlTitle": "Server URL",
"asrOpenAICompatBaseUrlHint": "The base URL of an OpenAI-compatible transcription server. To use this engine, also set OMNIVOICE_ASR_BACKEND=openai-compat-asr — there's no in-app engine picker for ASR yet.",
"asrOpenAICompatModelTitle": "Model",
"asrOpenAICompatApiKeyTitle": "API key",
"asrOpenAICompatApiKeyHint": "Optional — many self-hosted servers don't require one.",
"asrOpenAICompatApiKeyOptional": "optional",
"asrOpenAICompatKeyConfigured": "A key is saved. Leave blank to keep it, or type a new one to replace it.",
"asrOpenAICompatLoadError": "Failed to load ASR server setting",
"asrOpenAICompatSaveError": "Failed to save"
},
"enterprise_faq": {
"q_internal_tools": "Do I need a license for internal tools?",
+9 -2
View File
@@ -515,7 +515,9 @@
"routingRemote": "远程",
"routingUnknown": "未知",
"routingEffectiveChip": "在此机器上的 {{device}} 上运行",
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}"
"routingCaveatTitle": "已选择 GPU,但是:{{reason}}",
"curatedModelLabel": "模型",
"curatedModelAria": "{{engine}} 的模型"
},
"capture": {
"desc": "全局热键仅在桌面应用中有效。网页界面使用页面内快捷键 <1>Ctrl+Shift+Space</1>(窗口聚焦时可用)。",
@@ -1144,7 +1146,11 @@
"next_try_dictation": "下一步:试用听写",
"step_aria": "第 {{num}} 步:{{label}}",
"step_completed": "已完成",
"cache_label": "模型缓存"
"cache_label": "模型缓存",
"mirror_rescue_title": "无法连接 Hugging Face?使用镜像",
"mirror_rescue_hint": "模型下载会立即切换到镜像地址,安装过程无需重启。hf-mirror.com 是国内常用的社区镜像。",
"mirror_apply": "应用并重新检查",
"mirror_apply_error": "镜像设置保存失败。"
},
"donate": {
"title": "支持项目",
@@ -1504,6 +1510,7 @@
"flush": "冲洗",
"loaded_models": "加载模型",
"no_models": "没有加载模型",
"model_not_active": "非当前引擎 — 可安全卸载",
"unload": "卸载",
"flush_caches": "刷新缓存",
"unload_all_flush": "全部卸载+冲洗"
+2
View File
@@ -22,6 +22,7 @@ import StoragePanel from '../components/settings/StoragePanel';
import StorageTab from '../components/settings/StorageTab';
import StorageUsagePanel from '../components/settings/StorageUsagePanel';
import HFMirrorPanel from '../components/settings/HFMirrorPanel';
import AsrOpenAICompatPanel from '../components/settings/AsrOpenAICompatPanel';
import SharingPanel from '../components/settings/SharingPanel';
import RemoteBackendPanel from '../components/settings/RemoteBackendPanel';
import MCPBindingsPanel from '../components/settings/MCPBindingsPanel';
@@ -367,6 +368,7 @@ export default function Settings() {
<>
<StoragePanel />
<HFMirrorPanel />
<AsrOpenAICompatPanel />
<ModelStoreTab info={info} modelBadge={modelBadge} />
</>
);
+101
View File
@@ -3,6 +3,7 @@ import { useTranslation } from 'react-i18next';
import { Loader, RotateCw } from 'lucide-react';
import { cn } from '@/lib/utils';
import { useSetupStatus, usePreflight } from '../api/hooks';
import { apiJson, apiFetch } from '../api/client';
import WizardLibrary from '../components/WizardLibrary';
import HfTokenCard from '../components/HfTokenCard';
import DictationDemo from '../components/DictationDemo';
@@ -133,6 +134,101 @@ function PreflightPanel({ report, loading, onRecheck }) {
);
}
/* ── Mirror rescue — restricted-network escape hatch on step 0 ─────────── */
/**
* Shown when the network preflight check can't reach the Hugging Face
* endpoint. Users behind restricted networks (e.g. China, where
* huggingface.co is blocked) can't reach Settings yet the wizard gates the
* studio so the mirror quick-pick has to live right here. PUT /hf-mirror
* takes effect immediately for downloads (no restart during setup).
*/
function MirrorRescue({ onApplied }) {
const { t } = useTranslation();
const [presets, setPresets] = useState([]);
const [url, setUrl] = useState('');
const [saving, setSaving] = useState(false);
const [error, setError] = useState(null);
useEffect(() => {
let alive = true;
apiJson('/api/settings/hf-mirror')
.then((d) => {
if (!alive) return;
setPresets((d?.presets || []).filter((p) => p.url));
setUrl(d?.configured || '');
})
.catch(() => {
/* endpoint unavailable — keep the free-text input usable */
});
return () => {
alive = false;
};
}, []);
const apply = async (value) => {
setSaving(true);
setError(null);
try {
await apiFetch('/api/settings/hf-mirror', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url: value }),
});
setUrl(value);
onApplied();
} catch (e) {
setError(e?.message || t('setup.mirror_apply_error'));
} finally {
setSaving(false);
}
};
return (
<div className="mt-3 flex flex-col gap-1.5 rounded-md border border-border px-3 py-2.5">
<span className="text-sm font-semibold">{t('setup.mirror_rescue_title')}</span>
<span className="text-xs leading-snug text-fg-muted">{t('setup.mirror_rescue_hint')}</span>
{error && (
<span className="text-xs text-danger" role="alert">
{error}
</span>
)}
<div className="mt-1 flex flex-wrap items-center gap-2">
{presets.map((p) => (
<Button
key={p.url}
variant="preset"
size="sm"
disabled={saving}
onClick={() => apply(p.url)}
data-testid={`wizard-mirror-${p.url}`}
>
{p.label}
</Button>
))}
<input
type="text"
value={url}
onChange={(e) => setUrl(e.target.value)}
placeholder="https://hf-mirror.com"
className="min-w-[220px] flex-1 rounded border border-border bg-transparent px-2 py-1 font-mono text-xs text-fg"
data-testid="wizard-mirror-url"
/>
<Button
variant="subtle"
size="sm"
loading={saving}
disabled={saving || !url.trim()}
onClick={() => apply(url.trim())}
data-testid="wizard-mirror-apply"
>
{t('setup.mirror_apply')}
</Button>
</div>
</div>
);
}
/* ── LED stepper rail ──────────────────────────────────────────────────── */
function StepperNav({ step, maxUnlockedStep, onStep }) {
@@ -241,6 +337,10 @@ export default function SetupWizard({ onReady }) {
const modelsReady = !!status?.models_ready;
const preflightOk = !!pre?.ok;
// Offer the mirror quick-pick whenever the HF endpoint probe didn't pass
// the wizard is the only surface these users can reach (Settings is gated
// behind setup), so the escape hatch must live here.
const networkDown = (pre?.checks || []).some((c) => c.id === 'network' && c.status !== 'pass');
const cachePath = status?.hf_cache_dir || '~/.cache/huggingface';
@@ -288,6 +388,7 @@ export default function SetupWizard({ onReady }) {
<div className="flex min-h-0 flex-auto flex-col gap-3" key="step-0">
<div className="fr-rise min-h-0 flex-1 overflow-y-auto" style={{ '--rise': 1 }}>
<PreflightPanel report={pre} loading={preLoading} onRecheck={recheckPreflight} />
{networkDown && <MirrorRescue onApplied={recheckPreflight} />}
</div>
<div
className="fr-rise flex shrink-0 items-center justify-between gap-4 border-t border-border pt-3"
+4 -2
View File
@@ -138,7 +138,8 @@ export default function VoiceGallery() {
stopPlayback();
flash(
t('gallery.preview_failed', {
defaultValue: 'Preview unavailable — the voice engine may still be loading.',
message: e?.message || String(e),
defaultValue: 'Preview unavailable: {{message}}',
}),
);
} finally {
@@ -222,7 +223,8 @@ export default function VoiceGallery() {
} catch (e) {
flash(
t('gallery.use_failed', {
defaultValue: 'Could not create that voice — the engine may be loading.',
message: e?.message || String(e),
defaultValue: 'Could not create that voice: {{message}}',
}),
);
}
@@ -702,4 +702,189 @@ describe('EngineCompatibilityMatrix', () => {
// KittenTTS in the fixture carries no setup_snippet no snippet block.
expect(screen.queryByTestId('setup-snippet-kittentts')).not.toBeInTheDocument();
});
// #981 mlx-audio curated-model picker
function mlxAudioResponse({ activeModelId = 'kokoro' } = {}) {
return {
tts: {
active: 'mlx-audio',
backends: [
{
id: 'mlx-audio',
display_name: 'MLX-Audio (test)',
available: true,
reason: null,
install_hint: null,
last_error: null,
isolation_mode: 'in-process',
gpu_compat: ['mps', 'cpu'],
curated_models: [
{
key: 'kokoro',
label: 'Kokoro (default, fast)',
repo_id: 'mlx-community/Kokoro-82M-bf16',
},
{ key: 'csm', label: 'CSM (voice cloning)', repo_id: 'mlx-community/csm-1b-8bit' },
{
key: 'outetts',
label: 'OuteTTS',
repo_id: 'mlx-community/Llama-OuteTTS-1.0-1B-4bit',
},
],
active_model_id: activeModelId,
},
],
},
asr: { active: '', backends: [] },
llm: { active: 'off', backends: [] },
};
}
it('renders the curated-model picker for mlx-audio, pre-selected to the active model', async () => {
const apiListEngines = vi
.fn()
.mockResolvedValue(mlxAudioResponse({ activeModelId: 'outetts' }));
render(
<EngineCompatibilityMatrix
family="tts"
onSelect={vi.fn()}
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('MLX-Audio (test)'));
const select = screen.getByTestId('curated-model-select-mlx-audio');
expect(select).toHaveValue('outetts');
// All curated models are offered as options.
expect(
within(select).getByRole('option', { name: 'Kokoro (default, fast)' }),
).toBeInTheDocument();
expect(within(select).getByRole('option', { name: 'CSM (voice cloning)' })).toBeInTheDocument();
expect(within(select).getByRole('option', { name: 'OuteTTS' })).toBeInTheDocument();
});
it('picking a different curated model calls onSelect with the model key and refreshes', async () => {
let activeModelId = 'kokoro';
const apiListEngines = vi.fn(async () => mlxAudioResponse({ activeModelId }));
const onSelect = vi.fn(async (_family, _id, modelId) => {
activeModelId = modelId;
});
render(
<EngineCompatibilityMatrix
family="tts"
onSelect={onSelect}
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('MLX-Audio (test)'));
const select = screen.getByTestId('curated-model-select-mlx-audio');
fireEvent.change(select, { target: { value: 'csm' } });
await waitFor(() => {
expect(onSelect).toHaveBeenCalledWith('tts', 'mlx-audio', 'csm');
});
// Reloaded after the pick matrix reflects the new active_model_id.
await waitFor(() => {
expect(screen.getByTestId('curated-model-select-mlx-audio')).toHaveValue('csm');
});
expect(apiListEngines.mock.calls.length).toBeGreaterThanOrEqual(2);
});
it('does not render a curated-model picker for engines without curated_models', async () => {
const apiListEngines = vi.fn().mockResolvedValue(makeEnginesResponse());
render(
<EngineCompatibilityMatrix
family="tts"
onSelect={vi.fn()}
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('OmniVoice (test)'));
expect(screen.queryByTestId(/curated-model-select-/)).not.toBeInTheDocument();
});
it('disables the curated-model picker when no onSelect is provided', async () => {
const apiListEngines = vi.fn().mockResolvedValue(mlxAudioResponse());
render(
<EngineCompatibilityMatrix
family="tts"
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('MLX-Audio (test)'));
expect(screen.getByTestId('curated-model-select-mlx-audio')).toBeDisabled();
});
// showFamilyTabs={false} pinned per-family mount (Settings Engines)
function multiFamilyResponse() {
return {
tts: {
active: 'omnivoice',
backends: [
{
id: 'omnivoice',
display_name: 'OmniVoice (test)',
available: true,
reason: null,
install_hint: null,
last_error: null,
isolation_mode: 'in-process',
gpu_compat: ['cpu'],
},
],
},
asr: {
active: 'whisperx',
backends: [
{
id: 'whisperx',
display_name: 'WhisperX (test)',
available: true,
reason: null,
install_hint: null,
last_error: null,
isolation_mode: 'in-process',
gpu_compat: ['cpu'],
},
],
},
llm: { active: 'off', backends: [] },
};
}
it('pins to the given family and hides the TTS/ASR/LLM switcher when showFamilyTabs is false', async () => {
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
render(
<EngineCompatibilityMatrix
family="asr"
showFamilyTabs={false}
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('WhisperX (test)'));
// Pinned header names the family instead of the generic matrix title
expect(screen.getByText('ASR Engines')).toBeInTheDocument();
// the TTS family never leaks into the pinned table
expect(screen.queryByText('OmniVoice (test)')).not.toBeInTheDocument();
// and there is no family switcher to wander off to.
expect(document.querySelector('.engine-matrix__tab-family')).toBeNull();
});
it('keeps the family switcher by default (standalone mounts unchanged)', async () => {
const apiListEngines = vi.fn().mockResolvedValue(multiFamilyResponse());
render(
<EngineCompatibilityMatrix
family="tts"
apiListEngines={apiListEngines}
apiGetEngineHealth={vi.fn()}
/>,
);
await waitFor(() => screen.getByText('OmniVoice (test)'));
expect(screen.getByText('Engine Compatibility Matrix')).toBeInTheDocument();
expect(document.querySelectorAll('.engine-matrix__tab-family').length).toBe(3);
});
});
@@ -0,0 +1,26 @@
// Regression guard: VoiceGallery/CommunityZone/ImportsZone catch blocks used
// to discard the real error and show a hardcoded, often-wrong generic guess
// (e.g. "the engine may be loading" on ANY failure, including ones that had
// nothing to do with loading). Fixed to interpolate the real `e.message`
// (already a clean, user-facing string from api/client.js's ApiError),
// matching the `{{message}}` convention used everywhere else in this file.
// This test only pins the i18n keys, not the call sites, deliberately: it's
// a cheap net against reverting to a hardcoded string, not a full behavior test.
import { describe, it, expect } from 'vitest';
import en from '../i18n/locales/en.json';
describe('gallery error messages interpolate the real error', () => {
const keys = [
'use_failed',
'preview_failed',
'search_failed',
'upload_failed',
'save_failed',
'delete_failed',
'trim_load_failed',
];
it.each(keys)('gallery.%s contains {{message}}', (key) => {
expect(en.gallery[key]).toContain('{{message}}');
});
});
+13
View File
@@ -5,7 +5,20 @@
const MAX = 500;
const buf = [];
// Tauri's own internal IPC fallback (#975): on some Windows configurations
// the custom-protocol IPC probe fails once at startup and Tauri logs this
// exact console.warn before silently — and successfully — falling back to
// postMessage + WebSocket. It's benign and fires at most once per launch,
// but as a captured console.warn it spuriously flips the Logs footer's
// Frontend pill to "1 warning" on every affected launch. Filtered at the
// capture source (not the display layer) so it never enters the ring
// buffer or a copied diagnostic dump either.
const BENIGN_WARNING_PREFIXES = ['IPC custom protocol failed'];
function push(level, args) {
if (level === 'warn' && typeof args[0] === 'string') {
if (BENIGN_WARNING_PREFIXES.some((p) => args[0].startsWith(p))) return;
}
const msg = Array.from(args)
.map((a) => {
if (a instanceof Error) return `${a.name}: ${a.message}${a.stack ? '\n' + a.stack : ''}`;
+37
View File
@@ -0,0 +1,37 @@
import { beforeEach, describe, expect, it } from 'vitest';
import { clearFrontendLogs, getFrontendLogs, installConsoleCapture } from './consoleBuffer';
describe('consoleBuffer', () => {
// installConsoleCapture() wraps console.* exactly once per page load (a
// module-level `installed` guard) — it must NOT be re-installed or have
// console.warn restored between tests, or later tests run against the
// un-wrapped original. Install once; only the ring buffer resets per test.
installConsoleCapture();
beforeEach(() => {
clearFrontendLogs();
});
it('captures an ordinary warning', () => {
console.warn('something genuinely worth seeing');
expect(getFrontendLogs().some((l) => l.msg.includes('something genuinely worth seeing'))).toBe(
true,
);
});
it("#975: filters Tauri's benign IPC-fallback warning out of the captured buffer", () => {
console.warn(
'IPC custom protocol failed, Tauri will now use the postMessage interface instead',
);
expect(getFrontendLogs().some((l) => l.msg.includes('IPC custom protocol failed'))).toBe(false);
});
it('does not filter a different warning that merely mentions IPC', () => {
// Prefix match, not a substring match — only Tauri's exact known message
// is suppressed; anything else that happens to mention "IPC" is not.
console.warn('some other IPC warning entirely');
expect(getFrontendLogs().some((l) => l.msg.includes('some other IPC warning entirely'))).toBe(
true,
);
});
});
+16
View File
@@ -52,6 +52,22 @@ describe('buildDesignInstruct', () => {
expect(unsupported).toEqual(['sôi nổi']);
});
it('clone path (#980): RTL/non-Latin script (Hebrew) is dropped like any other unsupported free-text, not a crash', () => {
// #612 covered Latin-script-with-diacritics (Vietnamese); #980 was the same
// failure mode with a right-to-left script — a name typed into the Style
// field must degrade the same way (dropped client-side + unsupported bucket),
// never round-trip to the backend's 400.
const { instruct, unsupported } = buildDesignInstruct({}, 'שמואל');
expect(instruct).toBe('');
expect(unsupported).toEqual(['שמואל']);
});
it('clone path (#980): RTL script mixed with a valid tag keeps the tag, drops the rest', () => {
const { instruct, unsupported } = buildDesignInstruct({}, 'whisper, שמואל');
expect(instruct).toBe('whisper');
expect(unsupported).toEqual(['שמואל']);
});
it('buckets a valid tag outranked by a dropdown as a duplicate, not unsupported (#114)', () => {
const { instruct, unsupported, duplicates } = buildDesignInstruct(
{ Pitch: 'low pitch' },
+10 -1
View File
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "omnivoice"
version = "0.3.11"
version = "0.3.14"
description = "OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models"
readme = "README.md"
# Free and open-source under the GNU Affero General Public License v3 (see
@@ -154,6 +154,15 @@ dependencies = [
# httpx imports it lazily inside try/except, so PyInstaller's tracer
# misses it and frozen installers would stay broken without the entry.
"socksio>=1.0",
# OS trust store for TLS (#976). Users behind a corporate/antivirus proxy
# that TLS-inspects traffic get a raw "[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE]"
# on every model install — the TCP connection succeeds, but the proxy's
# re-signed certificate is trusted by the OS (Windows CryptoAPI/SChannel)
# and not by Python's bundled `certifi` CA list. `truststore` patches
# `ssl.SSLContext` to verify against the OS trust store instead. Pure-
# Python, MIT, PyPA-maintained, zero transitive deps — same class of fix
# as socksio above, identical on macOS/Windows/Linux.
"truststore>=0.9",
]
[project.optional-dependencies]
+17
View File
@@ -42,6 +42,23 @@ for stage_base in "$STAGE_BASE_RELEASE" "$STAGE_BASE_DEBUG"; do
echo "inject-apprun: replacing AppRun in $appdir"
cp -f "$APPRUN_SRC" "$appdir/AppRun"
chmod 755 "$appdir/AppRun"
# Stamp the bundled WebKitGTK version (#961 follow-up). The AppImage
# bundles THIS build host's libwebkit2gtk, so the host's pkg-config
# answer here is the version the shipped bundle will actually run —
# knowable by construction at bundle time, unknowable reliably at
# runtime (a user's pkg-config reports their SYSTEM's version, which
# LD_LIBRARY_PATH overrides with the bundled copy). AppRun's workaround
# auto-detection reads this marker first and only falls back to host
# pkg-config when the marker is absent (bundles predating the stamp).
wk_bundled="$(pkg-config --modversion webkit2gtk-4.1 2>/dev/null \
|| pkg-config --modversion webkit2gtk-4.0 2>/dev/null \
|| echo "")"
if [ -n "$wk_bundled" ]; then
printf '%s\n' "$wk_bundled" > "$appdir/.bundled-webkitgtk-version"
echo "inject-apprun: stamped bundled WebKitGTK version: $wk_bundled"
else
echo "inject-apprun: WARNING — could not read the bundled WebKitGTK version (pkg-config missing?); AppRun will use its runtime fallback" >&2
fi
found=1
fi
done
@@ -247,6 +247,213 @@ def test_select_llm_never_routing_gated(fresh_app, monkeypatch):
assert r.status_code == 200, r.text
# ── ASR selection via /engines/select (Settings → Engines ASR picker) ──────
#
# The ASR family was always wired in _FAMILIES on paper, but no UI called it
# and nothing exercised it — the Settings picker now does. Lock the contract:
# a pick persists to prefs["asr_backend"], `OMNIVOICE_ASR_BACKEND` still wins
# over the pick, and unknown / not-ready ids are 400s.
def _register_fake_asr(asr_mod, engine_id, *, available=True):
"""Register a light in-process ASR stub (CPU-only so a forced-CPU host
routes it `cpu_only`, never `unavailable`). Returns (cls, restore_fn)."""
_avail = available
class _FakeASR(asr_mod.ASRBackend):
id = engine_id
display_name = f"Fake {engine_id}"
gpu_compat = ("cpu",)
@classmethod
def is_available(cls):
return (True, "ready") if _avail else (False, "deps missing (test)")
def transcribe(self, audio_path, *, word_timestamps=True):
raise NotImplementedError
saved = dict(asr_mod._REGISTRY)
asr_mod._REGISTRY[engine_id] = _FakeASR
def restore():
asr_mod._REGISTRY.clear()
asr_mod._REGISTRY.update(saved)
return _FakeASR, restore
def test_select_asr_persists_pref_and_echoes_active(fresh_app, monkeypatch):
from core import prefs as _prefs
from services import asr_backend as asr_mod
_force_cpu_host(monkeypatch)
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
_, restore = _register_fake_asr(asr_mod, "fake-asr")
try:
r = _client(fresh_app).post(
"/engines/select", json={"family": "asr", "backend_id": "fake-asr"})
assert r.status_code == 200, r.text
body = r.json()
assert body["family"] == "asr"
assert body["active"] == "fake-asr"
assert body["env_override"] is False
assert _prefs.get("asr_backend") == "fake-asr"
finally:
restore()
def test_select_asr_env_var_still_wins(fresh_app, monkeypatch):
"""CRITICAL backward-compat: an existing `OMNIVOICE_ASR_BACKEND` pin keeps
winning over a Settings pick the pick persists to prefs (for when the
pin is lifted) but the active id stays the env value, and the response
says so via env_override."""
from core import prefs as _prefs
from services import asr_backend as asr_mod
_force_cpu_host(monkeypatch)
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "pytorch-whisper")
_, restore = _register_fake_asr(asr_mod, "fake-asr-pinned")
try:
r = _client(fresh_app).post(
"/engines/select", json={"family": "asr", "backend_id": "fake-asr-pinned"})
assert r.status_code == 200, r.text
body = r.json()
assert body["env_override"] is True
assert body["active"] == "pytorch-whisper" # env wins
assert _prefs.get("asr_backend") == "fake-asr-pinned"
finally:
restore()
def test_select_asr_unknown_backend_is_400(fresh_app):
r = _client(fresh_app).post(
"/engines/select", json={"family": "asr", "backend_id": "nope-not-real"})
assert r.status_code == 400
assert "Unknown asr backend" in r.json()["detail"]
def test_select_asr_unavailable_backend_is_400(fresh_app, monkeypatch):
from services import asr_backend as asr_mod
_force_cpu_host(monkeypatch)
_, restore = _register_fake_asr(asr_mod, "fake-asr-down", available=False)
try:
r = _client(fresh_app).post(
"/engines/select", json={"family": "asr", "backend_id": "fake-asr-down"})
assert r.status_code == 400
assert "not ready" in r.json()["detail"]
finally:
restore()
def test_get_engines_asr_family_shape(fresh_app):
"""GET /engines/asr — the ASR picker's data source: active id + one row
per registered backend with availability, reasons and install hints."""
r = _client(fresh_app).get("/engines/asr")
assert r.status_code == 200
body = r.json()
assert isinstance(body["active"], str) and body["active"]
by_id = {b["id"]: b for b in body["backends"]}
assert {"whisperx", "faster-whisper", "openai-compat-asr"}.issubset(by_id)
# Install hints power the picker's tooltips (parity with TTS).
assert by_id["openai-compat-asr"]["install_hint"]
for entry in by_id.values():
missing = _REQUIRED_KEYS - entry.keys()
assert not missing, f"asr entry {entry['id']!r} missing: {missing}"
# ── #981 — mlx-audio curated-model selection via /engines/select ───────────
#
# mlx-audio multiplexes 7+ curated models behind one backend id. Before this
# fix there was NO way anywhere in the UI/API to pick which curated model
# actually loads — it always defaulted to Kokoro even if the user had
# downloaded e.g. Llama-OuteTTS via Settings → Models.
def _make_mlx_audio_available(monkeypatch):
"""mlx-audio is Apple-Silicon-gated; force is_available()=True + a
CPU-friendly host so the routing gate doesn't block these tests on
non-mac CI runners."""
from services import tts_backend as tts_mod
monkeypatch.setattr(
tts_mod.MLXAudioBackend, "is_available",
classmethod(lambda cls: (True, "ready")),
)
_force_cpu_host(monkeypatch)
def test_select_mlx_audio_unknown_model_id_is_400(fresh_app, monkeypatch):
_make_mlx_audio_available(monkeypatch)
r = _client(fresh_app).post(
"/engines/select",
json={"family": "tts", "backend_id": "mlx-audio", "model_id": "not-a-real-model"},
)
assert r.status_code == 400
assert "Unknown mlx-audio model" in r.json()["detail"]
def test_select_mlx_audio_curated_key_persists(fresh_app, monkeypatch):
from core import prefs as _prefs
_make_mlx_audio_available(monkeypatch)
r = _client(fresh_app).post(
"/engines/select",
json={"family": "tts", "backend_id": "mlx-audio", "model_id": "outetts"},
)
assert r.status_code == 200, r.text
assert _prefs.get("mlx_audio_model_id") == "outetts"
assert _prefs.get("tts_backend") == "mlx-audio"
def test_select_mlx_audio_raw_repo_id_accepted(fresh_app, monkeypatch):
"""MLXAudioBackend already tolerates a raw HF repo id, not just a
curated key (tts_backend.py ~733) the API must too."""
from core import prefs as _prefs
_make_mlx_audio_available(monkeypatch)
r = _client(fresh_app).post(
"/engines/select",
json={
"family": "tts", "backend_id": "mlx-audio",
"model_id": "mlx-community/Some-Other-Model-4bit",
},
)
assert r.status_code == 200, r.text
assert _prefs.get("mlx_audio_model_id") == "mlx-community/Some-Other-Model-4bit"
def test_select_mlx_audio_without_model_id_does_not_touch_pref(fresh_app, monkeypatch):
"""Selecting mlx-audio without a model_id (e.g. an older frontend) must
leave any existing mlx_audio_model_id pref untouched."""
from core import prefs as _prefs
_prefs.set_("mlx_audio_model_id", "csm")
_make_mlx_audio_available(monkeypatch)
r = _client(fresh_app).post(
"/engines/select", json={"family": "tts", "backend_id": "mlx-audio"})
assert r.status_code == 200, r.text
assert _prefs.get("mlx_audio_model_id") == "csm"
def test_select_model_id_ignored_for_non_mlx_audio_backend(fresh_app):
"""model_id is only meaningful for mlx-audio; picking a different TTS
backend with a model_id set must not persist a stray pref."""
from core import prefs as _prefs
r = _client(fresh_app).post(
"/engines/select",
json={"family": "tts", "backend_id": "omnivoice", "model_id": "kokoro"},
)
assert r.status_code == 200, r.text
assert _prefs.get("mlx_audio_model_id") is None
def test_engines_response_curated_models_only_on_mlx_audio(fresh_app):
client = _client(fresh_app)
body = client.get("/engines").json()
by_id = {b["id"]: b for b in body["tts"]["backends"]}
assert "curated_models" in by_id["mlx-audio"]
assert "active_model_id" in by_id["mlx-audio"]
assert "curated_models" not in by_id["omnivoice"]
assert "active_model_id" not in by_id["omnivoice"]
# ── /engines/{id}/health round-trip ────────────────────────────────────────
@@ -141,7 +141,10 @@ def test_list_backends_resilient(registry_sandbox):
def test_list_backends_shape(registry_sandbox):
"""Every entry must contain exactly the documented keys — no more, no less."""
"""Every entry must contain exactly the documented keys — no more, no
less EXCEPT mlx-audio, which also carries `curated_models` +
`active_model_id` (#981): it multiplexes 7+ curated models behind one
backend id, so the Settings picker needs the roster + current pick."""
out = list_backends()
# `gpu_compat` joined the documented shape in Plan 02-04 alongside the
# Engine Compatibility Matrix UI (ENGINE-06). The three routing keys
@@ -154,14 +157,51 @@ def test_list_backends_shape(registry_sandbox):
# Copy-paste env-var line for path-gated opt-in engines (None otherwise).
"setup_snippet",
}
mlx_audio_extra = {"curated_models", "active_model_id"}
for entry in out:
assert set(entry.keys()) == required, (
expected = required | mlx_audio_extra if entry["id"] == "mlx-audio" else required
assert set(entry.keys()) == expected, (
f"entry {entry.get('id')} has wrong keys: "
f"missing {required - entry.keys()}, "
f"extra {entry.keys() - required}"
f"missing {expected - entry.keys()}, "
f"extra {entry.keys() - expected}"
)
def test_mlx_audio_curated_models_roster(registry_sandbox):
"""#981 — mlx-audio's entry carries the curated-model roster + the
currently-active pick, so Settings can render a model picker instead of
always silently defaulting to Kokoro."""
out = {entry["id"]: entry for entry in list_backends()}
entry = out["mlx-audio"]
assert entry["active_model_id"] == "kokoro" # DEFAULT_MODEL_KEY, no prefs set
keys = {m["key"] for m in entry["curated_models"]}
assert keys == set(tts_backend.MLXAudioBackend.CURATED_MODELS)
for m in entry["curated_models"]:
assert set(m.keys()) == {"key", "label", "repo_id"}
assert m["repo_id"] == tts_backend.MLXAudioBackend.CURATED_MODELS[m["key"]]
assert m["label"] # non-empty, readable
def test_mlx_audio_active_model_id_reflects_prefs(registry_sandbox, monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
_prefs.set_("mlx_audio_model_id", "outetts")
out = {entry["id"]: entry for entry in list_backends()}
assert out["mlx-audio"]["active_model_id"] == "outetts"
def test_curated_models_not_present_on_other_backends(registry_sandbox):
"""Only mlx-audio multiplexes multiple models behind one backend id — no
other entry should carry curated_models/active_model_id."""
out = list_backends()
for entry in out:
if entry["id"] == "mlx-audio":
continue
assert "curated_models" not in entry
assert "active_model_id" not in entry
def test_isolation_mode_in_process_vs_subprocess(registry_sandbox):
"""SubprocessBackend subclasses get isolation_mode='subprocess'; others 'in-process'."""
registry_sandbox["fake-sub"] = FakeSubBackend
+32
View File
@@ -37,6 +37,38 @@ if not os.environ.get("OMNIVOICE_ENV_FILE"):
import pytest
import warnings as _warnings
# ── torch default-dtype isolation (CI flaky trio) ───────────────────────────
# Three tests (test_effects_chain / test_generation_audio_guard /
# test_persona_bundle) fail intermittently on CI — never locally — with
# signatures that all trace to one cause: a leaked
# `torch.set_default_dtype(torch.float16)` from some earlier test. The
# smoking gun is test_generation_audio_guard's observed value
# 0.0999755859375, which is exactly float16(0.1): `torch.tensor([0.1, …])`
# built under a leaked fp16 default. The same leak collapses
# test_effects_chain's preset differences into identical quantized outputs,
# and hands test_persona_bundle's soundfile writer fp16 data libsndfile
# can't encode. The polluter only executes on CI-Linux (it never reproduces
# on macOS), so rather than chase it blind, this guard makes the whole leak
# class impossible — same philosophy as the LLM-state guard below — and
# names the offender in CI output when it fires, so it CAN be chased.
@pytest.fixture(autouse=True)
def _torch_default_dtype_guard(request):
yield
torch = sys.modules.get("torch")
if torch is None:
return
if torch.get_default_dtype() is not torch.float32:
_warnings.warn(
f"{request.node.nodeid} leaked torch default dtype "
f"{torch.get_default_dtype()} — resetting to float32. This is "
f"the polluter behind the CI flaky trio; fix it at the source.",
stacklevel=1,
)
torch.set_default_dtype(torch.float32)
# ── LLM-provider state isolation (issue #878) ──────────────────────────────
# LLM provider selection is process-global three ways: env vars (the
+2
View File
@@ -18,6 +18,7 @@ DELETE /profiles/{profile_id}/consent
DELETE /projects/{project_id}
DELETE /pronunciation/{entry_id}
GET /api/mcp/bindings
GET /api/settings/asr-openai-compat
GET /api/settings/changelog
GET /api/settings/db-backup
GET /api/settings/dictation-refinement
@@ -216,6 +217,7 @@ POST /v1/audio/transcriptions
POST /watermark/detect
POST /watermark/settings
PUT /api/mcp/bindings
PUT /api/settings/asr-openai-compat
PUT /api/settings/dictation-refinement
PUT /api/settings/hf-mirror
PUT /api/settings/llm-endpoint
+16
View File
@@ -61,3 +61,19 @@ test('empty / missing attrs yields all-Auto', () => {
for (const cat of ALL_CATS) assert.equal(out[cat], 'Auto');
}
});
test('#983 — a partial vdStates shape (as restored from a saved profile or '
+ 'localStorage) is completed to all 6 CATEGORIES keys', () => {
// Mirrors the exact partial shape from issue #983: only Gender survives
// (e.g. a design profile saved by an older client, or a hand-edited
// payload), the other 5 category keys are simply absent from the object.
// useProfiles.js/useAppData.js now run any restored vd_states through this
// helper before calling setVdStates, so DesignMethodPanel's render never
// sees an undefined category value.
const out = mergeDescribedAttrs({ Gender: 'male' });
assert.deepEqual(Object.keys(out).sort(), [...ALL_CATS].sort());
assert.equal(out.Gender, 'male');
for (const cat of ALL_CATS) {
if (cat !== 'Gender') assert.equal(out[cat], 'Auto');
}
});
+29
View File
@@ -0,0 +1,29 @@
"""Run the AppImage AppRun launcher's shell unit tests under pytest.
AppRun.test.sh existed but was wired into NO CI job the launcher's
workaround auto-detection (which decides whether shipped Linux builds get
WEBKIT_DISABLE_COMPOSITING_MODE) could regress silently. This wrapper rides
the standard "Tests (backend + frontend)" gate instead of needing its own
workflow step. Covers the #961 follow-up too: the build-time
.bundled-webkitgtk-version marker must beat the host's pkg-config answer.
"""
import os
import shutil
import subprocess
import pytest
_REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_SCRIPT = os.path.join(_REPO, "frontend", "src-tauri", "appimage", "AppRun.test.sh")
@pytest.mark.skipif(shutil.which("bash") is None, reason="bash not available")
def test_apprun_shell_suite_passes():
proc = subprocess.run(
["bash", _SCRIPT], capture_output=True, text=True, timeout=120,
)
assert proc.returncode == 0, (
f"AppRun.test.sh failed (exit {proc.returncode}):\n"
f"{proc.stdout}\n{proc.stderr}"
)
assert "0 fail" in proc.stdout
+239
View File
@@ -0,0 +1,239 @@
"""Generic OpenAI-compatible ASR backend (#877) — a path to Qwen3-ASR,
FunASR/SenseVoice self-hosted servers, or OpenAI's own Whisper API, today,
without waiting on transformers to ship a direct Qwen3-ASR integration.
settings_store backed by in-memory dicts, OpenAI client faked at the SDK
boundary (no network) house convention, same as test_llm_providers_router.py:
direct handler calls, no TestClient, so the loopback auth guard isn't in play.
"""
from __future__ import annotations
import os
import sys
import types
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
os.environ.setdefault("OMNIVOICE_MODEL", "test")
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
_HAS_OPENAI = __import__("importlib").util.find_spec("openai") is not None
pytestmark = pytest.mark.skipif(not _HAS_OPENAI, reason="openai package not installed")
@pytest.fixture
def ss(monkeypatch):
"""services.settings_store, resolved fresh (no module-level import — see
asr_mod's docstring for why staleness across sys.modules reimports is a
real risk in this suite) and patched to in-memory dicts (no SQLite)."""
from services import settings_store as _ss
text: dict[str, str] = {}
secrets: dict[str, str] = {}
monkeypatch.setattr(_ss, "get_text", lambda k, default=None: text.get(k, default))
monkeypatch.setattr(_ss, "set_text", lambda k, v: text.__setitem__(k, v))
monkeypatch.setattr(_ss, "get_secret", lambda n: secrets.get(n))
monkeypatch.setattr(
_ss, "set_secret", lambda n, v: secrets.__setitem__(n, v) if v else secrets.pop(n, None)
)
monkeypatch.setattr(_ss, "list_secret_names", lambda: list(secrets))
return _ss
@pytest.fixture
def asr_mod(ss, monkeypatch):
"""services.asr_backend with settings_store in-memory (no SQLite).
Resolved via importlib.import_module INSIDE the fixture (not a top-level
`import` in this file) so it's the module object actually live in
sys.modules at test-run time other test files in this ~2400-test suite
pop+reimport shared service modules (services.model_manager,
services.tts_backend), and a module-level import captured once at file
COLLECTION time can go stale by the time an individual test in this file
finally runs, hours of test-order later. A collection-time reference
calling .set_text() and a fixture-time reference reading via .get_text()
can silently be two different module objects the write and the read
land in different in-memory dicts, and the test fails with no obvious
cause. Every test below takes `ss` as a fixture (not a module-level
`from services import settings_store`) for the same reason.
"""
for var in ("ASR_OPENAI_COMPAT_BASE_URL", "ASR_OPENAI_COMPAT_MODEL", "ASR_OPENAI_COMPAT_API_KEY"):
monkeypatch.delenv(var, raising=False)
import importlib
return importlib.import_module("services.asr_backend")
@pytest.fixture
def settings_mod(asr_mod):
"""api.routers.settings sharing the same monkeypatched settings_store."""
import importlib
return importlib.import_module("api.routers.settings")
def _fake_openai_transcribe(monkeypatch, *, verbose_ok=True, response=None, raise_exc=None):
"""Fake openai.OpenAI whose audio.transcriptions.create() either returns
a canned response or raises. verbose_ok=False simulates a minimal server
that rejects response_format="verbose_json" on the first call, forcing
the plain-json fallback."""
captured_kwargs = []
calls = []
class _FakeClient:
def __init__(self, **kwargs):
captured_kwargs.append(kwargs)
self.audio = types.SimpleNamespace(
transcriptions=types.SimpleNamespace(create=self._create)
)
def _create(self, **kw):
calls.append(kw)
if raise_exc is not None:
raise raise_exc
if kw.get("response_format") == "verbose_json" and not verbose_ok:
raise RuntimeError("response_format not supported")
return response
import openai
monkeypatch.setattr(openai, "OpenAI", _FakeClient)
return captured_kwargs, calls
# ── is_available() gating ───────────────────────────────────────────────────
def test_unavailable_without_base_url(asr_mod):
ok, msg = asr_mod.OpenAICompatASRBackend.is_available()
assert ok is False
assert "Settings" in msg
def test_available_once_base_url_configured(asr_mod, ss):
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
ok, _ = asr_mod.OpenAICompatASRBackend.is_available()
assert ok is True
# ── response adaptation ─────────────────────────────────────────────────────
def test_transcribe_adapts_verbose_json_segments(asr_mod, ss, monkeypatch, tmp_path):
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
class _Seg:
def model_dump(self):
return {"text": "hello world", "start": 0.0, "end": 1.5}
resp = types.SimpleNamespace(segments=[_Seg()], language="en")
_fake_openai_transcribe(monkeypatch, response=resp)
audio = tmp_path / "seg.wav"
audio.write_bytes(b"RIFF....WAVEfmt ") # content is never read by the fake client
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
assert out["language"] == "en"
assert out["segments"] == [{"text": "hello world", "start": 0.0, "end": 1.5, "words": []}]
assert out["chunks"] == [{"text": "hello world", "timestamp": (0.0, 1.5)}]
def test_transcribe_falls_back_to_plain_text_when_verbose_json_rejected(asr_mod, ss, monkeypatch, tmp_path):
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
resp = types.SimpleNamespace(text="plain text only", segments=None, language=None)
_captured, calls = _fake_openai_transcribe(monkeypatch, verbose_ok=False, response=resp)
audio = tmp_path / "seg.wav"
audio.write_bytes(b"RIFF....WAVEfmt ")
out = asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
assert len(calls) == 2 # verbose_json attempt, then the plain fallback
assert calls[0]["response_format"] == "verbose_json"
assert calls[1]["response_format"] == "json"
assert out["segments"] == [{"text": "plain text only", "start": 0.0, "end": None, "words": []}]
assert out["language"] == "en" # default when the server doesn't report one
def test_transcribe_network_failure_does_not_leak_raw_exception(asr_mod, ss, monkeypatch, tmp_path):
"""Mirrors the #977 convention: a raw SDK/httpx exception must never reach
the caller unformatted only a clean, actionable RuntimeError."""
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
_fake_openai_transcribe(monkeypatch, raise_exc=ConnectionError("connection refused"))
audio = tmp_path / "seg.wav"
audio.write_bytes(b"RIFF....WAVEfmt ")
with pytest.raises(RuntimeError) as ei:
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
msg = str(ei.value)
assert "localhost:8080" in msg
assert "ConnectionError" in msg
def test_client_disables_sdk_retries(asr_mod, ss, monkeypatch, tmp_path):
"""max_retries=0 — mirrors llm_skills.resolve_skill_client: a slow/rate-
limited server retrying inside the SDK would blow past the caller's own
bounded timeout expectation for a single transcribe call."""
ss.set_text(asr_mod._ASR_OPENAI_COMPAT_BASE_URL_KEY, "http://localhost:8080/v1")
resp = types.SimpleNamespace(text="ok", segments=None, language="en")
captured_kwargs, _ = _fake_openai_transcribe(monkeypatch, response=resp)
audio = tmp_path / "seg.wav"
audio.write_bytes(b"RIFF....WAVEfmt ")
asr_mod.OpenAICompatASRBackend().transcribe(str(audio))
assert captured_kwargs[0]["max_retries"] == 0
# ── settings endpoints ───────────────────────────────────────────────────────
def test_get_default_empty(settings_mod):
st = settings_mod.get_asr_openai_compat()
assert st == {"base_url": "", "model": "whisper-1", "has_key": False}
def test_put_persists_and_never_echoes_the_key(settings_mod):
st = settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(
base_url="http://localhost:8080/v1/", model="qwen3-asr", api_key="sk-test-123",
)
)
assert st["base_url"] == "http://localhost:8080/v1" # trailing slash trimmed
assert st["model"] == "qwen3-asr"
assert st["has_key"] is True
assert "sk-test-123" not in str(st) # the key never round-trips
st2 = settings_mod.get_asr_openai_compat()
assert st2 == st
def test_empty_api_key_clears_it(settings_mod):
settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(api_key="sk-test-123")
)
assert settings_mod.get_asr_openai_compat()["has_key"] is True
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key=""))
assert settings_mod.get_asr_openai_compat()["has_key"] is False
def test_none_fields_leave_existing_values_unchanged(settings_mod):
settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(base_url="http://localhost:8080/v1", model="qwen3-asr")
)
# A save that only touches api_key must not clobber base_url/model.
settings_mod.set_asr_openai_compat(settings_mod._ASROpenAICompatBody(api_key="sk-abc"))
st = settings_mod.get_asr_openai_compat()
assert st["base_url"] == "http://localhost:8080/v1"
assert st["model"] == "qwen3-asr"
assert st["has_key"] is True
def test_rejects_a_base_url_without_scheme(settings_mod):
from fastapi import HTTPException
with pytest.raises(HTTPException):
settings_mod.set_asr_openai_compat(
settings_mod._ASROpenAICompatBody(base_url="localhost:8080/v1")
)
def test_registered_in_backend_list(asr_mod):
assert "openai-compat-asr" in asr_mod._REGISTRY
assert asr_mod._REGISTRY["openai-compat-asr"] is asr_mod.OpenAICompatASRBackend
assert "openai-compat-asr" in asr_mod._INSTALL_HINTS
+345
View File
@@ -0,0 +1,345 @@
"""Issue #312 class — dub generation and batch TTS must honor the active
engine selection (Settings Engines) instead of silently falling back to
OmniVoice via services.model_manager.get_model(), and must refuse with an
actionable error naming alternatives instead of mis-cloning when the
active engine can't do reference-audio voice cloning.
Covers:
- `cloning_capable_engine_ids()` excludes the fixed-preset-voice engines
(kittentts, supertonic3, sherpa-onnx) and includes the cloning ones.
- /dub/generate: a non-cloning active engine fails the whole job with one
actionable message (never falls back to OmniVoice, never mis-clones
per segment).
- /dub/generate: a cloning-capable non-OmniVoice active engine actually
runs the request (proves the engine selection is honored, not ignored).
- batch: an unpinned voice_id runs fine on a non-cloning active engine;
a pinned voice_id on the same engine fails fast, before any TTS runs.
- `applies_own_mastering` still skips the shared mastering chain for both
pipelines (mirrors test_generate_engine.py's coverage of the same knob
for /generate).
"""
from __future__ import annotations
import asyncio
import importlib
import os
os.environ.setdefault("OMNIVOICE_DISABLE_FILE_LOG", "1")
import pytest
import torch
from fastapi import HTTPException
from schemas.requests import DubRequest, DubSegment
def _tts_mod():
"""Resolve services.tts_backend at RUN time — see test_generate_engine.py's
docstring for why (sys.modules pre-pollution across the collected suite)."""
return importlib.import_module("services.tts_backend")
def _make_fake_engine(engine_id, *, supports_cloning=True, available=True,
own_mastering=False, gpu_compat=("cpu",)):
tb = _tts_mod()
# Class-body assignment can't read the same name from the enclosing
# function scope (class bodies don't close over locals) — alias first,
# matching test_generate_engine.py's _make_fake_engine convention.
_cloning, _mastering, _compat = supports_cloning, own_mastering, gpu_compat
class _FakeEngine(tb.TTSBackend):
id = engine_id
display_name = f"Fake {engine_id} (test)"
supports_cloning = _cloning
applies_own_mastering = _mastering
gpu_compat = _compat
calls: list = []
@property
def sample_rate(self) -> int:
return 24000
@property
def supported_languages(self) -> list[str]:
return ["multi"]
@classmethod
def is_available(cls):
if available:
return True, "ready"
return False, "fake engine deliberately unavailable (test)"
def generate(self, text, **kw) -> torch.Tensor:
type(self).calls.append((text, kw))
return torch.zeros(1, 24000)
return _FakeEngine
@pytest.fixture
def fake_registry(monkeypatch):
"""Register a fake engine in the REAL registry so resolve_generation_backend
exercises the actual lookup/is_available/routing/cloning chain, not a stub.
Resets the MM2-01 active-backend cache before/after (see
tests/test_mm2_lifecycle.py's convention) so one test's cached instance
can't leak into the next."""
tb = _tts_mod()
tb.reset_active_backend()
registered: list[str] = []
def _register(engine_id, **kw):
cls = _make_fake_engine(engine_id, **kw)
tb._REGISTRY[engine_id] = cls
registered.append(engine_id)
return cls
yield _register
tb.reset_active_backend()
for engine_id in registered:
tb._REGISTRY.pop(engine_id, None)
@pytest.fixture
def no_omnivoice_model_manager(monkeypatch):
"""Fail loudly if resolution falls back to OmniVoice's get_model() path."""
import services.model_manager as mm
async def _boom():
raise AssertionError(
"services.model_manager.get_model() was called — engine "
"selection was silently ignored (#312 class)"
)
monkeypatch.setattr(mm, "get_model", _boom)
# ── cloning_capable_engine_ids() ────────────────────────────────────────────
def test_cloning_capable_engine_ids_excludes_fixed_voice_engines():
tb = _tts_mod()
ids = set(tb.cloning_capable_engine_ids())
assert ids.isdisjoint({"kittentts", "supertonic3", "sherpa-onnx"})
assert {"omnivoice", "voxcpm2", "cosyvoice", "gpt-sovits"}.issubset(ids)
def test_cloning_capable_engine_ids_excludes_model_dependent_adapters():
# MLXAudioBackend.supports_cloning is an instance @property (only some of
# its curated models can clone) — a class-level getattr() returns the
# property descriptor itself, which is truthy, so a naive check would
# always recommend "mlx-audio" even when the configured model is Kokoro
# (can't clone). Must be excluded from the suggestion list rather than
# falsely recommended.
tb = _tts_mod()
assert isinstance(
vars(tb.MLXAudioBackend).get("supports_cloning"), property
), "this test assumes MLXAudioBackend.supports_cloning is a property"
assert "mlx-audio" not in set(tb.cloning_capable_engine_ids())
# ── /dub/generate/{job_id} ──────────────────────────────────────────────────
@pytest.fixture
def dub_job_env(monkeypatch, tmp_path):
"""Minimal hermetic environment for `dg.dub_generate()` — same stub set as
test_smart_fit_generate.py's fixture, but WITHOUT patching
resolve_generation_backend, so the real registry + capability gate run."""
import api.routers.dub_generate as dg
job = {"duration": 2.0, "dubbed_tracks": {}, "speaker_clones": {}}
job_dir = tmp_path / "jobX"
job_dir.mkdir()
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
monkeypatch.setattr(
dg, "dub_seg_path",
lambda job_id, seg_id: str(job_dir / f"seg_{seg_id}.wav"),
)
monkeypatch.setattr(dg, "rvc_is_enabled", lambda: False)
monkeypatch.setattr(dg, "embed_watermark", lambda wav, sr: wav)
monkeypatch.setattr(dg, "apply_mastering", lambda a, sample_rate=None: a)
monkeypatch.setattr(dg, "get_effect_chain", lambda preset: None)
monkeypatch.setattr(dg, "apply_effects_chain", lambda a, **k: a)
monkeypatch.setattr(dg, "normalize_audio", lambda a, target_dBFS=None: a)
class _StubTaskManager:
def is_cancelled(self, task_id):
return False
async def add_task(self, task_id, task_type, func, *args, **kwargs):
async for _ in func(*args):
pass
monkeypatch.setattr(dg, "task_manager", _StubTaskManager())
return dg, job
def _one_seg_request():
return DubRequest(
segments=[DubSegment(start=0.0, end=1.0, text="hola")],
segment_ids=["0"], language="Auto", language_code="es", num_step=4,
)
def test_dub_generate_fails_fast_for_non_cloning_engine(
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
):
"""Active engine can't clone → the job fails once, up front, with an
actionable message naming alternatives never a silent OmniVoice run."""
dg, job = dub_job_env
fake_registry("fake-nonclone", supports_cloning=False)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-nonclone")
with pytest.raises(HTTPException) as exc_info:
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
assert exc_info.value.status_code == 400
detail = exc_info.value.detail
assert "fake-nonclone" in detail
assert "voice cloning" in detail
assert "omnivoice" in detail # names a real alternative
def test_dub_generate_uses_selected_cloning_engine_not_omnivoice(
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
):
"""A cloning-capable non-OmniVoice engine actually runs the segment."""
dg, job = dub_job_env
fake = fake_registry("fake-clone", supports_cloning=True)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-clone")
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
assert len(fake.calls) == 1
assert fake.calls[0][0] == "hola"
assert "es" in job["dubbed_tracks"]
def test_dub_generate_respects_applies_own_mastering(
dub_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
):
dg, job = dub_job_env
fake = fake_registry("fake-studio", supports_cloning=True, own_mastering=True)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-studio")
mastering_calls = []
monkeypatch.setattr(
dg, "apply_mastering",
lambda a, sample_rate=None: mastering_calls.append(1) or a,
)
asyncio.run(dg.dub_generate("jobX", _one_seg_request()))
assert len(fake.calls) == 1
assert mastering_calls == [] # studio engine's own mastering is not double-applied
# ── batch ────────────────────────────────────────────────────────────────
@pytest.fixture
def batch_job_env(monkeypatch, tmp_path):
import api.routers.batch as b
monkeypatch.setattr(b, "DATA_DIR", str(tmp_path))
async def _fake_run_transcribe_guarded(pool, fn, what=None):
# Bypass real ASR entirely — the engine-selection gate under test
# runs right after transcription, before translate/generate.
return (
[{"id": "s0", "start": 0.0, "end": 1.0, "text": "hola",
"text_original": "hola"}],
"en",
)
monkeypatch.setattr(
"services.asr_backend.run_transcribe_guarded",
_fake_run_transcribe_guarded,
)
def _fake_subprocess_run(cmd, *a, **kw):
class _Result:
stdout = b""
stderr = b"Duration: 00:00:02.00, start: 0.000000, bitrate: 1000 kb/s\n"
return _Result()
monkeypatch.setattr("subprocess.run", _fake_subprocess_run)
monkeypatch.setattr("services.ffmpeg_utils.find_ffmpeg", lambda: "ffmpeg")
def _make_job(job_id, *, voice_id=None):
return {
"id": job_id,
"status": "running",
"filename": "in.mp4",
"video_path": str(tmp_path / "in.mp4"),
"langs": ["en"], # == source_lang → translation stage is a no-op
"voice_id": voice_id,
"preserve_bg": True,
"created_at": 0.0,
"started_at": None,
"finished_at": None,
"error": None,
"progress": None,
}
return b, _make_job
def test_batch_unpinned_voice_succeeds_on_noncloning_engine(
batch_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
):
"""No voice_id pinned → any active engine (cloning-capable or not) is fine."""
b, make_job = batch_job_env
fake = fake_registry("fake-batch-nonclone", supports_cloning=False)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-nonclone")
job = make_job("jobA", voice_id=None)
asyncio.run(b._run_batch_pipeline("jobA", job))
assert len(fake.calls) == 1
assert "en" in job.get("outputs", {})
def test_batch_pinned_voice_fails_fast_on_noncloning_engine(
batch_job_env, fake_registry, monkeypatch,
):
"""voice_id pinned + a non-cloning active engine → fail before any TTS
runs, with the same actionable message shape as the dub gate."""
b, make_job = batch_job_env
fake = fake_registry("fake-batch-nonclone2", supports_cloning=False)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-nonclone2")
job = make_job("jobB", voice_id="some-voice-id")
with pytest.raises(ValueError) as exc_info:
asyncio.run(b._run_batch_pipeline("jobB", job))
detail = str(exc_info.value)
assert "fake-batch-nonclone2" in detail
assert "voice cloning" in detail
assert not fake.calls # never reached generate
def test_batch_respects_applies_own_mastering(
batch_job_env, fake_registry, no_omnivoice_model_manager, monkeypatch,
):
b, make_job = batch_job_env
fake = fake_registry("fake-batch-studio", supports_cloning=True, own_mastering=True)
monkeypatch.setenv("OMNIVOICE_TTS_BACKEND", "fake-batch-studio")
mastering_calls = []
import services.audio_dsp as audio_dsp
monkeypatch.setattr(
audio_dsp, "apply_mastering",
lambda a, sample_rate=None: mastering_calls.append(1) or a,
)
job = make_job("jobC", voice_id=None)
asyncio.run(b._run_batch_pipeline("jobC", job))
assert len(fake.calls) == 1
assert mastering_calls == []
+21 -3
View File
@@ -124,14 +124,32 @@ class _RefCapturingModel:
return [torch.full((1, int(0.5 * SR)), 0.1)]
class _FakeBackend:
"""Adapts the list-returning fake model above to the TTSBackend.generate()
contract (a single tensor, not a list) that resolve_generation_backend()
now hands dub_generate.py (issue #312 class)."""
applies_own_mastering = False
def __init__(self, model):
self._model = model
@property
def sample_rate(self):
return self._model.sampling_rate
def generate(self, *a, **kw):
return self._model.generate(*a, **kw)[0]
@pytest.fixture
def patched_generate(monkeypatch, tmp_path):
import api.routers.dub_generate as dg
model = _RefCapturingModel()
async def _fake_get_model():
return model
async def _fake_resolve_generation_backend(**kwargs):
return _FakeBackend(model)
job = {
"duration": 6.0,
@@ -148,7 +166,7 @@ def patched_generate(monkeypatch, tmp_path):
job_dir = tmp_path / "jobX"
job_dir.mkdir()
monkeypatch.setattr(dg, "get_model", _fake_get_model)
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
+264
View File
@@ -113,6 +113,66 @@ def test_tts_unknown_backend_raises():
tts_backend.get_backend_class("not-a-real-one")
# ── #981 — MLX-Audio curated-model selection ─────────────────────────────
#
# MLXAudioBackend.__init__ used to resolve its active model ONLY from
# OMNIVOICE_MLX_AUDIO_MODEL, invisible to Settings and unchangeable without
# restarting the process with an env var set. It must now mirror
# active_backend_id()'s env > prefs > default resolution.
def test_mlx_audio_model_id_resolves_via_prefs(monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
_prefs.set_("mlx_audio_model_id", "outetts")
be = tts_backend.MLXAudioBackend()
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["outetts"]
def test_mlx_audio_model_id_env_overrides_prefs(monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
_prefs.set_("mlx_audio_model_id", "outetts")
monkeypatch.setenv("OMNIVOICE_MLX_AUDIO_MODEL", "csm")
be = tts_backend.MLXAudioBackend()
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["csm"]
def test_mlx_audio_model_id_defaults_to_kokoro(monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
be = tts_backend.MLXAudioBackend()
assert be._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["kokoro"]
def test_get_active_tts_backend_reconstructs_on_mlx_model_switch(monkeypatch, tmp_path):
"""A curated-model-only change (same backend id 'mlx-audio') must
invalidate the cached instance too otherwise picking a different
curated model in Settings has no effect until an app restart."""
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_MLX_AUDIO_MODEL", raising=False)
monkeypatch.delenv("OMNIVOICE_TTS_BACKEND", raising=False)
_prefs.set_("tts_backend", "mlx-audio")
_prefs.set_("mlx_audio_model_id", "kokoro")
tts_backend.reset_active_backend()
try:
be1 = tts_backend.get_active_tts_backend()
assert be1._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["kokoro"]
# Same instance on a repeat call with nothing changed (still cached).
assert tts_backend.get_active_tts_backend() is be1
_prefs.set_("mlx_audio_model_id", "outetts")
be2 = tts_backend.get_active_tts_backend()
assert be2 is not be1
assert be2._model_id == tts_backend.MLXAudioBackend.CURATED_MODELS["outetts"]
finally:
tts_backend.reset_active_backend()
_prefs.set_("tts_backend", "omnivoice")
# ── ASR ─────────────────────────────────────────────────────────────────────
@@ -134,6 +194,53 @@ def test_asr_env_override(monkeypatch):
assert asr_backend.active_backend_id() == "pytorch-whisper"
# ── ASR selection resolution (Settings → Engines ASR picker) ────────────────
# Same env > prefs > auto-detect contract as TTS. The env var MUST keep
# winning so existing `OMNIVOICE_ASR_BACKEND` pins don't change behavior now
# that the Settings picker writes the prefs key.
def test_asr_active_backend_prefs_fallback(monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
_prefs.set_("asr_backend", "moonshine")
assert asr_backend.active_backend_id() == "moonshine"
# Env var must beat prefs.
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "pytorch-whisper")
assert asr_backend.active_backend_id() == "pytorch-whisper"
def test_asr_auto_detects_when_no_env_no_prefs(monkeypatch, tmp_path):
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
assert asr_backend.active_backend_id() in {
"whisperx", "faster-whisper", "mlx-whisper", "pytorch-whisper",
}
def test_get_active_asr_backend_follows_prefs_switch_without_restart(monkeypatch, tmp_path):
"""#981 class (fixed on the TTS side): a Settings pick must take effect on
the next transcribe, not after an app restart. get_active_asr_backend()
re-resolves the id per call, so a prefs write switches immediately."""
from core import prefs as _prefs
monkeypatch.setattr(_prefs, "_PREFS_PATH", str(tmp_path / "prefs.json"))
monkeypatch.delenv("OMNIVOICE_ASR_BACKEND", raising=False)
_prefs.set_("asr_backend", "pytorch-whisper")
assert isinstance(
asr_backend.get_active_asr_backend(), asr_backend.PyTorchWhisperBackend)
_prefs.set_("asr_backend", "moonshine")
assert isinstance(
asr_backend.get_active_asr_backend(), asr_backend.MoonshineASRBackend)
def test_asr_unknown_backend_raises(monkeypatch):
monkeypatch.setenv("OMNIVOICE_ASR_BACKEND", "not-a-real-asr")
with pytest.raises(ValueError):
asr_backend.get_active_asr_backend()
# ── LLM ─────────────────────────────────────────────────────────────────────
@@ -238,3 +345,160 @@ def test_hf_retry_is_single_shot():
with pytest.raises(RuntimeError):
tts_backend._retry_once_with_fresh_hf_client(loader, what="test")
assert len(calls) == 2
# ── #977: MLX-Audio Kokoro language-code resolution ─────────────────────────
# Kokoro's own vendored pipeline (mlx_audio.tts.models.kokoro.pipeline)
# hard-asserts `lang_code` against a fixed single-letter table. The old code
# blindly truncated a full language name — "Dutch"[:2].lower() == "du" — into
# that assert, crashing with an unreadable `(lang_code, LANG_CODES)` repr
# instead of a clean error. The resolution tests need the real mlx-audio
# package (Apple-Silicon-only) since they validate against ITS installed
# table, never a hardcoded guess; they skip cleanly where mlx-audio isn't
# installed (every non-macOS-ARM CI runner).
def test_mlx_audio_kokoro_resolves_supported_language_names():
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
resolve = tts_backend.resolve_kokoro_lang_code
assert resolve("English") == "a"
assert resolve("Spanish") == "e"
assert resolve("French") == "f"
assert resolve("Hindi") == "h"
assert resolve("Italian") == "i"
assert resolve("Portuguese") == "p"
assert resolve("Japanese") == "j"
assert resolve("Chinese") == "z"
# Some callers may already pass an ISO code — those resolve unchanged
# through Kokoro's own ALIASES table, not just our full-name map.
assert resolve("es") == "e"
assert resolve("en-gb") == "b"
@pytest.mark.parametrize("language", ["Dutch", "German"])
def test_mlx_audio_kokoro_rejects_unsupported_language_cleanly(language):
# The literal #977 report case ("Dutch") plus one more Kokoro doesn't
# support ("German") — neither's first two letters happen to alias to a
# valid Kokoro code, so both used to crash.
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
with pytest.raises(ValueError) as ei:
tts_backend.resolve_kokoro_lang_code(language)
msg = str(ei.value)
assert language in msg
assert "Kokoro" in msg
assert "English" in msg # names what Kokoro DOES support
def test_mlx_audio_generate_rejects_unsupported_kokoro_language_before_calling_model():
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
backend = tts_backend.MLXAudioBackend()
backend._model_id = backend.CURATED_MODELS["kokoro"]
backend._ensure_loaded = lambda: None # never actually load the model
def _boom_generate(**kw):
raise AssertionError("model.generate() must not run for a rejected language")
backend._model = types.SimpleNamespace(generate=_boom_generate)
with pytest.raises(ValueError, match="Dutch"):
backend.generate("hello", language="Dutch")
def test_mlx_audio_generate_passes_ref_text_through_for_cloning():
# #1012/#1013: MLXAudioBackend.generate() read voice/ref_audio/language/
# speed from kwargs but silently dropped ref_text — CSM (sesame.py) only
# builds its cloning context when BOTH ref_audio and ref_text are
# present, so cloning on CSM always raised an opaque
# "IndexError: list index out of range" deep inside mlx-audio instead of
# ever attempting the clone. Community-diagnosed with the exact fix.
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
backend = tts_backend.MLXAudioBackend()
backend._ensure_loaded = lambda: None
captured = {}
def _fake_generate(**kw):
captured.update(kw)
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
backend._model = types.SimpleNamespace(generate=_fake_generate)
backend.generate("hello", ref_audio="/tmp/ref.wav", ref_text="the reference line")
assert captured.get("ref_text") == "the reference line"
assert captured.get("ref_audio") == "/tmp/ref.wav"
def test_mlx_audio_generate_omits_ref_text_without_ref_audio():
# ref_text alone (no ref_audio) means nothing to CSM's context builder —
# don't pass a stray kwarg an engine that isn't cloning doesn't expect.
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
backend = tts_backend.MLXAudioBackend()
backend._ensure_loaded = lambda: None
captured = {}
def _fake_generate(**kw):
captured.update(kw)
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
backend._model = types.SimpleNamespace(generate=_fake_generate)
backend.generate("hello", ref_text="orphaned text, no audio")
assert "ref_text" not in captured
def test_mlx_audio_generate_design_path_unaffected_without_any_ref():
# Absorbed from community PR #1015 (MahdiHedhli) — the design/instruct
# path (no ref_audio, no ref_text at all) must stay untouched by the
# ref_text forwarding fix; neither kwarg may leak into the model call.
pytest.importorskip("mlx_audio", reason="mlx-audio is Apple-Silicon-only")
backend = tts_backend.MLXAudioBackend()
backend._ensure_loaded = lambda: None
captured = {}
def _fake_generate(**kw):
captured.update(kw)
return iter([types.SimpleNamespace(audio=__import__("numpy").zeros(4))])
backend._model = types.SimpleNamespace(generate=_fake_generate)
backend.generate("hello")
assert "ref_text" not in captured
assert "ref_audio" not in captured
def test_mlx_audio_generate_auto_language_skips_lang_code_entirely():
# Matches the "Auto" convention other engines in this file use
# (OmniVoiceBackend.generate(), _run_backend_inference) — never resolved,
# never forwarded as lang_code.
backend = tts_backend.MLXAudioBackend()
backend._model_id = backend.CURATED_MODELS["kokoro"]
backend._ensure_loaded = lambda: None
seen_kwargs = {}
def _fake_generate(**kw):
seen_kwargs.update(kw)
return iter([types.SimpleNamespace(audio=[0.0, 0.0, 0.0, 0.0])])
backend._model = types.SimpleNamespace(generate=_fake_generate)
backend.generate("hello", language="Auto")
assert "lang_code" not in seen_kwargs
def test_mlx_audio_generate_non_kokoro_model_ignores_kokoro_validation():
# Qwen3-TTS (and CSM/Dia/Chatterbox/MeloTTS/OuteTTS) don't use Kokoro's
# lang_code convention — a language Kokoro would reject must NOT be
# rejected when a different curated model is active (#977 nuance).
backend = tts_backend.MLXAudioBackend()
backend._model_id = backend.CURATED_MODELS["qwen3-tts"]
backend._ensure_loaded = lambda: None
seen_kwargs = {}
def _fake_generate(**kw):
seen_kwargs.update(kw)
return iter([types.SimpleNamespace(audio=[0.0, 0.0, 0.0, 0.0])])
backend._model = types.SimpleNamespace(generate=_fake_generate)
backend.generate("hello", language="Dutch") # must not raise
assert seen_kwargs.get("lang_code") == "du"
+31
View File
@@ -143,6 +143,37 @@ def test_classify_socks_proxy_support_missing():
assert failure.classify("ProxyError: connection refused by 10.0.0.1:8080") == ""
def test_classify_ssl_handshake_failure():
# #976: the exact error a Windows user behind a corporate/antivirus
# TLS-inspecting proxy sees on every model install — the TCP connection
# succeeds, but the handshake fails because the OS trusts the proxy's
# re-signed CA and Python's bundled certifi list doesn't. A different
# failure mode from #984's TCP-level "host unreachable" fix.
reason = (
"Install failed: Got: ConnectError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] "
"ssl/tls alert handshake failure (_ssl.c:1016)"
)
assert failure.classify(reason) == "SSL_HANDSHAKE_FAILURE"
evt = failure.build_failure(reason, stage="install", include_diagnostic=False)
assert evt["docs_topic"] == "SSL_HANDSHAKE_FAILURE"
assert evt["hint"], "the SSL-handshake class must carry an actionable hint"
# A CERTIFICATE_VERIFY_FAILED-style message (the other common corporate-MITM
# shape) must classify the same way.
cert_reason = (
"requests.exceptions.SSLError: HTTPSConnectionPool(host='huggingface.co', "
"port=443): Max retries exceeded with url: / (Caused by SSLError("
"SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate "
"verify failed: unable to get local issuer certificate')))"
)
assert failure.classify(cert_reason) == "SSL_HANDSHAKE_FAILURE"
# append_hint is the raw-string surface (setup/download.py's install SSE) —
# the detail keeps the real error AND gains the hint.
out = failure.append_hint(reason)
assert out.startswith(reason) and "truststore" in out
# A plain, unrelated connection error must NOT be mislabelled as SSL.
assert failure.classify("ConnectionError: connection refused") == ""
def test_classify_generic_still_empty():
# A genuinely unknown reason must still classify to "" (no false hint).
assert failure.classify("some totally unrelated failure") == ""
+4
View File
@@ -139,6 +139,10 @@ def test_gpu_host_keeps_vram_guidance(monkeypatch):
msg = _guidance_for(monkeypatch, "cuda")
assert "VRAM-starved" in msg
assert "set the engine to CPU" in msg
# #939: the sibling ASR timeout guard already recommends Flush/Unload —
# this message was missing it, forcing the maintainer to explain it
# manually in every report instead of the error saying so upfront.
assert "Flush" in msg
def test_probe_failure_defaults_to_gpu_wording(monkeypatch):
+43 -1
View File
@@ -14,7 +14,11 @@ import torch
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
from api.routers.generation import _sanitize_audio, _oom_friendly_reraise # noqa: E402
from api.routers.generation import ( # noqa: E402
_oom_friendly_reraise,
_safe_exc_text,
_sanitize_audio,
)
def test_sanitize_replaces_non_finite_with_silence():
@@ -257,3 +261,41 @@ def test_config_failure_does_not_swallow_real_oom():
with pytest.raises(RuntimeError) as ei:
_oom_friendly_reraise(RuntimeError("CUDA error: out of memory"))
assert "ran out of memory" in str(ei.value)
# ── #977: generic exception formatters never leak a raw container repr ─────
# mlx-audio's vendored Kokoro pipeline raises
# `assert lang_code in LANG_CODES, (lang_code, LANG_CODES)` — an
# AssertionError whose .args is a (str, dict) tuple. str(e) on that
# interpolates the ENTIRE table straight into the user-facing 500 message
# ("Underlying error: ('du', {'a': 'American English', ...})"). Any engine's
# generate() can raise something shaped like this, not just Kokoro, so the
# guard is class-level: _safe_exc_text() backs both of generation.py's
# generic (catch-all) exception formatters.
def test_safe_exc_text_plain_message_uses_house_style():
err = RuntimeError("plain readable message")
assert _safe_exc_text(err) == "RuntimeError: plain readable message"
def test_safe_exc_text_container_args_do_not_leak_raw_repr():
# Mirrors the exact #977 AssertionError shape.
err = AssertionError(("du", {"a": "American English", "b": "British English"}))
text = _safe_exc_text(err)
assert text.startswith("AssertionError")
assert "American English" not in text
assert "{" not in text and "}" not in text
assert "(" not in text and ")" not in text
def test_unrecognized_error_catchall_does_not_leak_container_repr():
# End-to-end through _oom_friendly_reraise's catch-all fallback (none of
# the specific classifiers above it match an AssertionError like this).
err = AssertionError(("du", {"a": "American English", "b": "British English"}))
with pytest.raises(RuntimeError) as ei:
_oom_friendly_reraise(err)
msg = str(ei.value)
assert "AssertionError" in msg
assert "American English" not in msg
assert "{" not in msg and "}" not in msg
+101
View File
@@ -0,0 +1,101 @@
"""
Regression tests: the mastering pre-stage must not hide a reverb.
A hardcoded Reverb inside apply_mastering() used to bake echo into every
non-raw synthesis regardless of the chosen effect preset (field reports of
echoey voices; the podcast preset even promises "no reverb", and cinematic/
warm got doubled reverb). Reverb is preset-declared only these tests pin
that contract.
Kept separate from test_effects_chain.py on purpose: that file skips
entirely without pedalboard, while the data-shape guards here must always
run. sys.path for backend imports is handled by tests/conftest.py.
"""
import builtins
import math
import sys
import pytest
import torch
from services.audio_dsp import (
EFFECT_PRESETS,
MASTERING_CHAIN,
apply_mastering,
)
def _stage_types(chain):
return [fx["type"] for fx in chain]
def _make_test_audio(duration_s=1.0, sample_rate=24000) -> torch.Tensor:
"""Create a test audio tensor with a simple sine wave."""
t = torch.linspace(0, duration_s, int(duration_s * sample_rate))
return torch.sin(2 * math.pi * 440 * t).unsqueeze(0) # 440 Hz sine, mono
class TestMasteringChainHasNoHiddenReverb:
def test_mastering_chain_contains_no_reverb(self):
"""The recurrence guard: nobody re-adds a reverb outside the preset system."""
assert "reverb" not in _stage_types(MASTERING_CHAIN)
def test_mastering_chain_keeps_highpass_and_compressor(self):
"""Removing the reverb must not gut the rest of the pre-stage."""
types = _stage_types(MASTERING_CHAIN)
assert "highpass" in types
assert "compressor" in types
class TestPresetReverbContract:
@pytest.mark.parametrize("preset_id", ["broadcast", "podcast"])
def test_no_reverb_presets_stay_reverb_free(self, preset_id):
"""podcast's description literally promises "no reverb"."""
assert "reverb" not in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
@pytest.mark.parametrize("preset_id", ["cinematic", "warm"])
def test_user_chosen_reverb_survives(self, preset_id):
"""Presets that deliberately declare reverb must keep it."""
assert "reverb" in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
class TestApplyMasteringFunctional:
def test_returns_same_shape_and_device(self):
pytest.importorskip("pedalboard")
audio = _make_test_audio()
result = apply_mastering(audio, sample_rate=24000)
assert isinstance(result, torch.Tensor)
assert result.shape == audio.shape
assert result.device == audio.device
def test_no_echo_tail_bleeds_into_silence(self):
"""A burst followed by silence must stay silent after mastering.
Fails with the old hidden Reverb (its tail rings past the burst);
passes with highpass + compressor only.
"""
pytest.importorskip("pedalboard")
sr = 24000
burst = _make_test_audio(duration_s=0.25, sample_rate=sr)
audio = torch.cat([burst, torch.zeros(1, sr)], dim=1) # + 1 s silence
result = apply_mastering(audio, sample_rate=sr)
# Skip 50 ms after the burst so the filters settle; a reverb tail is
# far louder and longer than that.
tail = result[:, burst.shape[1] + int(0.05 * sr):]
assert tail.abs().max().item() < 1e-3
def test_passthrough_when_pedalboard_missing(self, monkeypatch):
"""Graceful degradation: no pedalboard, audio returned unmodified."""
real_import = builtins.__import__
def no_pedalboard(name, *args, **kwargs):
if name == "pedalboard" or name.startswith("pedalboard."):
raise ImportError("pedalboard unavailable (simulated)")
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", no_pedalboard)
monkeypatch.delitem(sys.modules, "pedalboard", raising=False)
audio = _make_test_audio()
result = apply_mastering(audio, sample_rate=24000)
assert torch.equal(result, audio)
+39
View File
@@ -100,6 +100,45 @@ def test_list_loaded_asr_row_is_honest(monkeypatch):
assert asr.get("note") # explains the disabled unload button
def test_list_loaded_attributes_resident_tts_to_its_engine(monkeypatch):
# Field report: OmniVoice stays resident in VRAM after switching to
# voxcpm2, and the panel offered no hint it wasn't the routed engine.
class _Model:
_asr_pipe = object()
def parameters(self): raise StopIteration
monkeypatch.setattr(mm, "model", _Model())
monkeypatch.setattr(mm, "_diar_pipeline", None)
# String-target setattr: other suites pop+reimport services.* modules
# mid-run (see module docstring), so the collection-time `tb` alias can go
# stale — patch the module object _active_tts_id late-imports at call time.
monkeypatch.setattr("services.tts_backend.active_backend_id", lambda: "voxcpm2")
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
assert rows["tts"]["engine_id"] == "omnivoice"
assert rows["tts"]["is_active_engine"] is False
# ASR isn't competing with the TTS selection — must not be mislabeled.
assert "is_active_engine" not in rows["asr"]
monkeypatch.setattr("services.tts_backend.active_backend_id", lambda: "omnivoice")
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
assert rows["tts"]["is_active_engine"] is True
def test_list_loaded_attribution_failure_degrades(monkeypatch):
# Attribution is advisory: a raising prefs layer must not break the
# panel, just leave the active flag unknown.
class _Model:
_asr_pipe = None
def parameters(self): raise StopIteration
monkeypatch.setattr(mm, "model", _Model())
monkeypatch.setattr(mm, "_diar_pipeline", None)
def _boom():
raise RuntimeError("prefs unavailable")
monkeypatch.setattr("services.tts_backend.active_backend_id", _boom)
rows = {m["id"]: m for m in ml.list_loaded()["models"]}
assert rows["tts"]["is_active_engine"] is None
def test_facade_unload_unknown_raises():
with pytest.raises(ValueError):
_run(ml.unload("bogus"))
+47
View File
@@ -0,0 +1,47 @@
"""Regression test for issue #974.
The nemo-parakeet install hint used to tell users to run
`pip install nemo_toolkit[asr]` directly into OmniVoice's shared venv.
nemo_toolkit[asr]==2.7.3 hard-pins transformers>=4.57,<4.58, which is
UNSATISFIABLE alongside the app's own transformers>=5.3 requirement (needed
by omnivoice/models/omnivoice.py for HiggsAudioV2TokenizerModel). A user who
followed the old hint ended up with a backend that wouldn't even start
(ImportError: cannot import name 'HiggsAudioV2TokenizerModel').
nemo-parakeet has no isolated-venv option yet (unlike dots-tts /
moss-tts-v15 / confucius4-tts, which do), so the hint must not imply a safe
one-line fix it must say plainly that installing into the shared venv
will break the backend.
"""
from __future__ import annotations
from services.asr_backend import _INSTALL_HINTS, list_backends
def test_nemo_parakeet_hint_does_not_recommend_shared_venv_install():
hint = _INSTALL_HINTS["nemo-parakeet"]
# The literal old, destructive recommendation must never reappear.
assert "pip install nemo_toolkit[asr]" not in hint, (
f"nemo-parakeet install_hint regressed to the shared-venv-breaking "
f"bare pip install: {hint!r}"
)
def test_nemo_parakeet_hint_warns_about_transformers_conflict():
hint = _INSTALL_HINTS["nemo-parakeet"]
assert "transformers" in hint
lowered = hint.lower()
assert "conflict" in lowered or "break" in lowered
def test_nemo_parakeet_hint_does_not_imply_isolated_venv_exists():
"""Unlike dots-tts/moss-tts-v15/confucius4-tts, nemo-parakeet has no
isolated-venv env var yet the hint must not invent one."""
hint = _INSTALL_HINTS["nemo-parakeet"]
assert "OMNIVOICE_NEMO_PARAKEET_DIR" not in hint
def test_nemo_parakeet_hint_surfaced_in_list_backends():
rows = list_backends()
row = next(r for r in rows if r["id"] == "nemo-parakeet")
assert row["install_hint"] == _INSTALL_HINTS["nemo-parakeet"]
+5 -1
View File
@@ -81,7 +81,11 @@ def test_design_save_creates_row_when_model_unavailable(iso, monkeypatch):
assert row["kind"] == "design"
# Sample is pending — no rendered identity wav was forced at save time.
assert not row["ref_audio_path"]
assert json.loads(row["vd_states"]) == _VD
# #983: vd_states is completed to all 6 known categories before persisting
# (missing ones default to 'Auto') — _VD only sets 3, so the stored value
# is a superset of it, not an exact match.
stored = json.loads(row["vd_states"])
assert stored == {**_VD, "Style": "Auto", "EnglishAccent": "Auto", "ChineseDialect": "Auto"}
def test_all_auto_design_is_saveable(iso, monkeypatch):
+30 -1
View File
@@ -170,12 +170,41 @@ def test_design_create_renders_sample_and_stores_params(app_client, fake_render)
assert body["kind"] == "design"
profile = client.get(f"/profiles/{body['id']}").json()
assert profile["kind"] == "design"
assert json.loads(profile["vd_states"]) == _VD
# #983: the server now completes vd_states to all 6 known categories
# (missing ones default to 'Auto') before persisting — _VD only sets 3,
# so the stored value is a superset of it, not an exact match.
stored = json.loads(profile["vd_states"])
assert stored == {**_VD, "Style": "Auto", "EnglishAccent": "Auto", "ChineseDialect": "Auto"}
assert profile["seed"] == 42 # deterministic identity sample
wav = os.path.join(cfg.VOICES_DIR, profile["ref_audio_path"])
assert os.path.exists(wav) and os.path.getsize(wav) > 0
def test_design_normalizes_partial_vd_states_to_all_categories(app_client, fake_render):
"""#983: a design profile must never persist with a partial vd_states shape.
A client (older frontend build, hand-edited payload, third-party API
caller) that only sends a subset of the 6 known category keys used to be
saved as-is selecting that profile later handed the frontend an
incomplete vdStates object, crashing DesignMethodPanel's render
("Cannot read properties of undefined (reading 'replace')"). The server
now fills every missing category with 'Auto' before persisting, so the
stored vd_states is always complete regardless of which client wrote it.
"""
client, _ = app_client
r = client.post(
"/profiles",
data={"name": "Partial", "kind": "design", "vd_states": json.dumps({"Gender": "male"})},
)
assert r.status_code == 200, r.text
profile = client.get(f"/profiles/{r.json()['id']}").json()
stored = json.loads(profile["vd_states"])
assert set(stored) == {"Gender", "Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect"}
assert stored["Gender"] == "male"
for cat in ("Age", "Pitch", "Style", "EnglishAccent", "ChineseDialect"):
assert stored[cat] == "Auto"
# ── Migration 0005 ───────────────────────────────────────────────────────────
def _run_alembic(direction: str, db_path: str, target: str = "head"):
+21 -3
View File
@@ -154,6 +154,24 @@ class _FakeModel:
return [torch.full((1, n), val)]
class _FakeBackend:
"""Adapts the list-returning _FakeModel above to the TTSBackend.generate()
contract (a single tensor, not a list) that resolve_generation_backend()
now hands dub_generate.py (issue #312 class)."""
applies_own_mastering = False
def __init__(self, model):
self._model = model
@property
def sample_rate(self):
return self._model.sampling_rate
def generate(self, *a, **kw):
return self._model.generate(*a, **kw)[0]
@pytest.fixture
def patched_generate(monkeypatch, tmp_path):
"""Patch api.routers.dub_generate so `_stream` runs hermetically:
@@ -162,8 +180,8 @@ def patched_generate(monkeypatch, tmp_path):
model = _FakeModel()
async def _fake_get_model():
return model
async def _fake_resolve_generation_backend(**kwargs):
return _FakeBackend(model)
job = {
"duration": 2.0,
@@ -173,7 +191,7 @@ def patched_generate(monkeypatch, tmp_path):
job_dir = tmp_path / "jobX"
job_dir.mkdir()
monkeypatch.setattr(dg, "get_model", _fake_get_model)
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
+57
View File
@@ -269,6 +269,63 @@ def test_preflight_network_handles_offline():
assert _probe_network(host="10.255.255.1", timeout=0.3) is False
def test_preflight_network_unreachable_is_warn_not_blocker():
"""A dead network must NOT hard-block the wizard (restricted-network
first-run, e.g. China where huggingface.co is blocked): the check is a
warning and the aggregate `ok` is unaffected by it."""
from api.routers.setup import wizard as setup_mod
with patch.object(setup_mod, "_probe_network", return_value=False):
body = client_factory().get("/setup/preflight").json()
net = next(c for c in body["checks"] if c["id"] == "network")
assert net["status"] == "warn", net
assert "continue" in (net["fix"] or "").lower()
# ok must still equal "no fail among checks" — network can't be the fail.
any_fail = any(c["status"] == "fail" for c in body["checks"])
assert body["ok"] is (not any_fail)
def test_preflight_network_probes_configured_mirror():
"""With HF_ENDPOINT set, the probe targets the mirror host — not the
hardcoded official host that may be blocked on the user's network."""
import os
from api.routers.setup import wizard as setup_mod
seen_hosts: list[str] = []
def fake_probe(host="huggingface.co", port=443, timeout=2.0):
seen_hosts.append(host)
return True
with patch.dict(os.environ, {"HF_ENDPOINT": "https://mirror.example.test"}), \
patch.object(setup_mod, "_probe_network", side_effect=fake_probe):
body = client_factory().get("/setup/preflight").json()
net = next(c for c in body["checks"] if c["id"] == "network")
assert "mirror.example.test" in net["label"]
assert net["status"] == "pass"
assert "mirror.example.test" in seen_hosts
def test_preflight_network_suggests_reachable_mirror():
"""Official endpoint blocked but hf-mirror.com reachable → the fix names
the mirror and the check carries mirror_reachable=True for the wizard's
quick-pick affordance."""
from api.routers.setup import wizard as setup_mod
def fake_probe(host="huggingface.co", port=443, timeout=2.0):
return host == "hf-mirror.com"
with patch.object(setup_mod, "_probe_network", side_effect=fake_probe):
body = client_factory().get("/setup/preflight").json()
net = next(c for c in body["checks"] if c["id"] == "network")
assert net["status"] == "warn"
assert net.get("mirror_reachable") is True
assert "hf-mirror.com" in (net["fix"] or "")
# ── RAM thresholds ───────────────────────────────────────────────────────
def test_preflight_ram_fail_threshold():
+143
View File
@@ -0,0 +1,143 @@
"""A quit mid-preload must not report a clean shutdown while a GPU-pool
thread is still running (#1000 class).
Field report: a backend log showed three rapid restart cycles, each ending
with "Shutdown: done." immediately followed by a "Model loading failed:
Could not import module 'AutoFeatureExtractor'" error — transformers' own
generic lazy-import wrapper, not a real dependency problem. The real cause:
`preload_task` (and the optional `capture_preload_task`) were created at
startup but never referenced in the shutdown block, so `idle_task`/
`worker_task` got cancelled-and-awaited while the preload task was simply
abandoned the process declared "done" while a background GPU-pool thread
was still mid-`import`, and got torn down by interpreter finalization under
it.
`_cancel_and_await_tasks` is the extracted, directly-testable shutdown
helper the full `lifespan()` context manager touches too much startup
machinery (DB init, gallery init, MCP session manager) to drive directly in
a unit test (this suite's own test_mcp_mount.py notes exactly this: running
the full lifespan contaminates other tests' event loops).
"""
from __future__ import annotations
import asyncio
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "backend"))
from main import _cancel_and_await_tasks # noqa: E402
def _run(coro):
return asyncio.run(coro)
def test_a_task_that_finished_before_cancel_keeps_its_result():
"""An early-stage task (mirrors preload still importing, not yet deep in
blocking weight-load work) that completes on its own before the shutdown
helper even reaches it must not be treated as an error `.cancel()` on
an already-done task is a no-op, and its real result survives. This is
the fix: previously preload_task was never referenced in shutdown at
all, so this case (the common one most quits don't land mid-import)
was never even checked."""
finished = []
async def _quick():
await asyncio.sleep(0.01)
finished.append("done")
async def _scenario():
t = asyncio.create_task(_quick())
await asyncio.sleep(0.05) # long enough for _quick() to fully finish
assert t.done()
await _cancel_and_await_tasks(t, timeout=1.0) # must not raise on a done task
_run(_scenario())
assert finished == ["done"]
def test_none_entries_are_skipped_without_error():
"""capture_preload_task is None when OMNIVOICE_PRELOAD_CAPTURE_ASR=0 —
the helper must not crash on a mix of real tasks and None."""
async def _noop():
return None
async def _scenario():
t = asyncio.create_task(_noop())
await _cancel_and_await_tasks(t, None, timeout=1.0)
_run(_scenario()) # must not raise
def test_a_task_stuck_past_the_bound_times_out_without_hanging():
"""A task that never yields back (mirroring a GPU-pool thread stuck in a
blocking native call) must not hang shutdown forever the bound is the
backstop, same as the pre-existing idle_task/worker_task pattern."""
async def _wedged():
await asyncio.sleep(10.0)
async def _scenario():
t = asyncio.create_task(_wedged())
await asyncio.sleep(0.01)
await _cancel_and_await_tasks(t, timeout=0.2)
import time
start = time.monotonic()
_run(_scenario())
elapsed = time.monotonic() - start
assert elapsed < 2.0, f"shutdown helper did not bound its wait: took {elapsed:.2f}s"
def test_multiple_tasks_are_all_cancelled_before_any_await():
"""Cancel-then-await (not cancel-then-immediately-await-one-at-a-time) —
every task gets its cancellation requested up front, so a slow task
earlier in the list can't delay a later task's cancel signal."""
cancelled_order = []
async def _tracked(name, delay):
try:
await asyncio.sleep(delay)
except asyncio.CancelledError:
cancelled_order.append(name)
raise
async def _scenario():
t1 = asyncio.create_task(_tracked("slow", 5.0))
t2 = asyncio.create_task(_tracked("fast", 5.0))
await asyncio.sleep(0.01)
await _cancel_and_await_tasks(t1, t2, timeout=0.5)
_run(_scenario())
assert set(cancelled_order) == {"slow", "fast"}
def test_production_shutdown_wait_is_generous_enough_for_a_cold_import():
"""Post-merge code-review finding (Greptile, PR #1002): the original 3s
bound left a real residual window cancelling the asyncio task doesn't
stop the underlying OS thread, so a cold transformers import taking
longer than the bound could still let shutdown report "done" while that
thread was alive, the exact #1000 class again just with lower odds.
Python can't forcibly kill a running thread, so no finite bound
eliminates this outright this pins the production call site to a
materially more generous wait (20s, not 3s) rather than letting a future
edit quietly shrink it back down without deliberate consideration.
Source-level guard, not a live-timing test: driving an actual >3s cold
import through this suite would make it slow and environment-dependent
for no real benefit.
"""
import re
src = open(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"backend", "main.py")).read()
call = re.search(
r"await _cancel_and_await_tasks\(\s*idle_task,\s*worker_task,\s*preload_task,"
r"\s*capture_preload_task,\s*timeout=([\d.]+),?\s*\)",
src,
)
assert call, "production shutdown call site not found in main.py"
assert float(call.group(1)) >= 15.0, (
f"shutdown wait bound regressed to {call.group(1)}s — see PR #1002 review history "
"before shrinking this"
)
+21 -3
View File
@@ -52,6 +52,24 @@ class _FakeModel:
return [torch.full((1, n), 0.25)]
class _FakeBackend:
"""Adapts the list-returning _FakeModel above to the TTSBackend.generate()
contract (a single tensor, not a list) that resolve_generation_backend()
now hands dub_generate.py (issue #312 class)."""
applies_own_mastering = False
def __init__(self, model):
self._model = model
@property
def sample_rate(self):
return self._model.sampling_rate
def generate(self, *a, **kw):
return self._model.generate(*a, **kw)[0]
async def _fake_stretch(wav, target_samples, sr):
"""Stand-in for the ffmpeg atempo pipe: deterministic linear interp."""
if target_samples <= 0 or wav.shape[-1] == target_samples:
@@ -67,8 +85,8 @@ def patched_generate(monkeypatch, tmp_path):
model = _FakeModel()
async def _fake_get_model():
return model
async def _fake_resolve_generation_backend(**kwargs):
return _FakeBackend(model)
job = {
"duration": 4.0,
@@ -78,7 +96,7 @@ def patched_generate(monkeypatch, tmp_path):
job_dir = tmp_path / "jobX"
job_dir.mkdir()
monkeypatch.setattr(dg, "get_model", _fake_get_model)
monkeypatch.setattr(dg, "resolve_generation_backend", _fake_resolve_generation_backend)
monkeypatch.setattr(dg, "_get_job", lambda job_id: job)
monkeypatch.setattr(dg, "_save_job", lambda job_id, j: None)
monkeypatch.setattr(dg, "DUB_DIR", str(tmp_path))
+82
View File
@@ -21,6 +21,8 @@ from services.speaker_clone import (
MIN_SLICE_DURATION_S,
_pick_reference_slices,
extract_speaker_clones,
refine_ref_text,
refine_ref_texts,
)
SR = 16000
@@ -124,3 +126,83 @@ class TestExtractSpeakerClones:
# or every real turn boundary would be flagged.
from services.segmentation import SPEAKER_GAP
assert 0 < ADJACENT_TURN_GUARD_S < SPEAKER_GAP
class _FakeASR:
"""Stands in for the active ASR backend's .transcribe() — no model, no
network. `chunks_by_path` maps a ref_audio path to the canned chunk list
that path's re-transcription should return."""
def __init__(self, chunks_by_path=None, raises_for=()):
self.chunks_by_path = chunks_by_path or {}
self.raises_for = set(raises_for)
self.calls = []
def transcribe(self, path, *, word_timestamps=True):
self.calls.append(path)
if path in self.raises_for:
raise RuntimeError("simulated ASR failure")
return {"chunks": self.chunks_by_path.get(path, []), "language": "es"}
class TestRefineRefText:
# Issue #1004: the ASR segment's `text` field and its `[start, end]`
# timestamps routinely drift (a trailing word audible in the slice but
# missing from the text, or vice versa) — pairing a mismatched (ref_audio,
# ref_text) breaks zero-shot TTS prompt priming badly enough that the
# clone can speak the reference text verbatim instead of the target text.
# Re-transcribing the actual written clip guarantees the pair matches.
def test_replaces_mismatched_text_with_the_actual_clip_transcript(self):
asr = _FakeASR(chunks_by_path={
"/tmp/ref.wav": [{"text": "hola"}, {"text": "que tal"}],
})
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="mismatched source text")
assert out == "hola que tal"
assert asr.calls == ["/tmp/ref.wav"]
def test_falls_back_to_original_text_on_asr_failure(self):
asr = _FakeASR(raises_for={"/tmp/ref.wav"})
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
assert out == "original text"
def test_falls_back_to_original_text_on_empty_transcript(self):
# A clip ASR can't get any text out of (e.g. near-silent) shouldn't
# wipe out a usable original — empty is worse than stale.
asr = _FakeASR(chunks_by_path={"/tmp/ref.wav": []})
out = refine_ref_text("/tmp/ref.wav", asr, fallback_text="original text")
assert out == "original text"
def test_no_asr_backend_is_a_strict_no_op(self):
# Preflight ASR load failure, or any other reason the caller has no
# backend to hand in — never a crash, never blocks the original path.
out = refine_ref_text("/tmp/ref.wav", None, fallback_text="original text")
assert out == "original text"
class TestRefineRefTexts:
def test_refines_every_entry_in_place_and_returns_the_dict(self):
asr = _FakeASR(chunks_by_path={
"/tmp/spk1.wav": [{"text": "hola amigo"}],
"/tmp/spk2.wav": [{"text": "buenos dias"}],
})
clones = {
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "stale 1"},
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
}
out = refine_ref_texts(clones, asr)
assert out is clones # mutated in place, returned for call-and-reassign
assert clones["Speaker 1"]["ref_text"] == "hola amigo"
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
def test_a_failing_entry_does_not_affect_the_others(self):
asr = _FakeASR(
chunks_by_path={"/tmp/spk2.wav": [{"text": "buenos dias"}]},
raises_for={"/tmp/spk1.wav"},
)
clones = {
"Speaker 1": {"ref_audio": "/tmp/spk1.wav", "ref_text": "kept on failure"},
"Speaker 2": {"ref_audio": "/tmp/spk2.wav", "ref_text": "stale 2"},
}
refine_ref_texts(clones, asr)
assert clones["Speaker 1"]["ref_text"] == "kept on failure"
assert clones["Speaker 2"]["ref_text"] == "buenos dias"
+31
View File
@@ -0,0 +1,31 @@
"""The conftest torch-dtype guard resets a leaked default dtype between tests.
The CI "flaky trio" (test_effects_chain / test_generation_audio_guard /
test_persona_bundle) failed intermittently on CI-Linux with signatures that
all trace to one leak: some earlier test leaves
``torch.set_default_dtype(torch.float16)`` behind. Reproduced locally with a
simulated polluter ``torch.tensor([0.1, ])`` under fp16 yields exactly the
0.0999755859375 CI observed, and Pedalboard refuses fp16 audio outright
("only supports 32-bit and 64-bit floating point"), silently returning
unmodified audio for every preset so their outputs compare identical.
These two tests are order-dependent BY DESIGN (pytest runs tests within a
file in definition order): the first leaks, the second proves the autouse
guard in conftest.py reset the leak before the next test began.
"""
import torch
def test_a_deliberate_dtype_leak():
# Simulates the CI polluter. The conftest guard must clean this up (and
# emit a UserWarning naming this exact test as the offender).
torch.set_default_dtype(torch.float16)
assert torch.get_default_dtype() is torch.float16
def test_b_next_test_starts_back_at_float32():
# If the guard is ever removed/broken, this fails — and so, eventually,
# does the flaky trio on CI, much less legibly.
assert torch.get_default_dtype() is torch.float32
# The exact fp16 signature the trio's CI failures showed, as documentation:
assert torch.tensor([0.1]).item() != 0.0999755859375
Generated
+12 -1
View File
@@ -3207,7 +3207,7 @@ wheels = [
[[package]]
name = "omnivoice"
version = "0.3.11"
version = "0.3.14"
source = { editable = "." }
dependencies = [
{ name = "accelerate" },
@@ -3247,6 +3247,7 @@ dependencies = [
{ name = "torchaudio", version = "2.8.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'linux' and sys_platform != 'win32'" },
{ name = "torchaudio", version = "2.8.0+cu128", source = { registry = "https://download.pytorch.org/whl/cu128" }, marker = "sys_platform == 'linux' or sys_platform == 'win32'" },
{ name = "transformers" },
{ name = "truststore" },
{ name = "uvicorn" },
{ name = "webdataset" },
{ name = "websockets" },
@@ -3328,6 +3329,7 @@ requires-dist = [
{ name = "torchaudio", marker = "sys_platform != 'linux' and sys_platform != 'win32'", specifier = ">=2.4" },
{ name = "torchaudio", marker = "sys_platform == 'linux' or sys_platform == 'win32'", specifier = ">=2.4", index = "https://download.pytorch.org/whl/cu128" },
{ name = "transformers", specifier = ">=5.3.0" },
{ name = "truststore", specifier = ">=0.9" },
{ name = "unidecode", marker = "extra == 'eval'" },
{ name = "uvicorn" },
{ name = "webdataset" },
@@ -6234,6 +6236,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/20/63/8cb444ad5cdb25d999b7d647abac25af0ee37d292afc009940c05b82dda0/triton-3.4.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7936b18a3499ed62059414d7df563e6c163c5e16c3773678a3ee3d417865035d", size = 155659780, upload-time = "2025-07-30T19:58:51.171Z" },
]
[[package]]
name = "truststore"
version = "0.10.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" },
]
[[package]]
name = "typer"
version = "0.24.1"