Refreshes README and linked docs with accurate installation, platform, privacy, API, and model-license guidance; documents local gigastt; and pins OpenAI-compatible ASR traffic to the configured secure origin. Closes#1736.
* feat(settings): compute-device override — auto | CUDA | ROCm | XPU | MPS | CPU
Auto-detect stays the default; the override kills the 'auto-detect picked
wrong' issue class. Applied at the single choke point (_probe()'s family
selection) so routing, get_best_device(), and every badge inherit it.
Resolution: OMNIVOICE_DEVICE env > Settings pick (prefs.json) > auto (#981
pattern). An override can steer, never invent hardware: a family the host
lacks is noted and ignored; cpu is always honorable. Applies at next
backend start (host caps are immutable per process — same restart contract
as the rest of the Performance tab, RestartBadge shown).
GET/PUT /api/settings/compute-device (admin-gated) reports resolved vs
applied so the panel shows restart-required truthfully and disables itself
under an env pin instead of pretending.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): entry for the compute-device override (#1557)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): harvest — the override reaches CT2 ASR, full i18n, honest edge states
- _ctranslate2_cuda_ok() and the ASR sidecar now gate on the probe's family,
so a cpu pin (or ROCm host) can never hand CTranslate2 a CUDA device —
the override reaches every CT2 loader through one shared gate
- override_ignored exposed by the API and shown by the panel (env pin naming
a device this machine lacks: auto is in effect, restart won't change it)
- all 8 panel strings + 5 device-family labels translated into all 21
locales; failed saves keep their error visible through the re-sync
- test isolation: cleanup drops OMNIVOICE_DEVICE before re-probing so no
overridden caps leak into later tests; panel tests wait for loaded state
- xpu/intel search keywords; oxfmt formatting
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(device-override): round 2 — fail-safe probe fallbacks, complete i18n, combined pin state
- a broken capability probe now means CPU everywhere (CT2 gate + ASR
sidecar) — never a torch-derived guess that would bypass a cpu pin or
re-open #1529 on ROCm; regression test added
- env-pinned AND not-detected shows both facts in one subtitle
- device_load_failed/perf_save_failed translated into all 21 locales;
CJK/th/vi/ar strings no longer say literal 'Auto'
- test_ctranslate2_never_gets_cuda_on_a_rocm_build pins the probe family
(it was order-dependent on the lru_cache before)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(test): pin the probe family in the faster-whisper OOM-fallback test
Same class as the rocm-build test: it mocked torch but not the probe the
new override gate consults first, so on a cpu-family CI host the CUDA
fallback chain under test was unreachable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* docs(engines): a guide for every engine + index; fix two engine-metadata bugs
21 new pages under docs/engines/ (10 TTS, 10 ASR, index README) — every
registered engine now has one: what it's for, platform support, model env
vars, quirks with issue refs. Linked from both READMEs' engine sections.
Code fixes found while verifying facts against the registries:
- KittenTTS docstring claimed default voice 'Jasper'; the code default is
expr-voice-2-f
- the isolated-ASR sidecar read only ASR_MODEL_FW while the download
preflight read ASR_MODEL_FASTER — set one and the other quietly used a
different model; both now resolve ASR_MODEL_FW-override → ASR_MODEL_FASTER
- moonshine's install hint named 'useful-moonshine', a package the backend
never imports; now moonshine-onnx / moonshine-voice
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(changelog): entries for the engine guides + sidecar model fix (#1556)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(engines): second-harvest fixes — TLS guidance, matrix/code alignment, CN counts
- README matrix aligned to gpu_compat (the code is the source of truth):
CosyVoice macOS is CPU not MPS, IndexTTS and GGUF gain their real
CUDA/CPU/MPS cells
- gpt-sovits guide: prefer https/tunnel for non-loopback servers, plaintext
warning; first-use download guidance on both OmniVoice pages
- preflight empty-env fallback matches the sidecar (ASR_MODEL_FASTER='' no
longer resolves a different repo)
- nano installs via uv pip; kitten log level wording; index links install
guides incl. the Gatekeeper step; README_CN engine counts 16/11
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(readme-cn): the all-engines-local claim now excludes the remote client
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
An RX 7900 XTX in the :rocm image crashed ASR init with 'CUDA driver
version is insufficient for CUDA runtime version' (#1529): ROCm torch
answers torch.cuda.is_available() and hands out 'cuda' device strings,
but whisperx/faster-whisper run on CTranslate2, whose CUDA runtime is
NVIDIA-only. Same class as the Apple/#1127 lesson, on the AMD axis.
- _ctranslate2_cuda_ok(): 'cuda' for CTranslate2 only when torch is a
real CUDA build (torch.version.hip is the honest tell); ROCm hosts
take CPU int8 instead of a native crash.
- _auto_detect(): on a ROCm-GPU host prefer pytorch-whisper — a pure
transformers pipeline riding torch itself, so it actually uses the
HIP GPU while CTranslate2 engines would idle on the CPU.
Fail-before/pass-after: 4 new tests fail on the old device pick/order.
Fixes#1529
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor(launchpad): quieter, borderless design refresh
The launchpad carried decoration from an earlier direction: icon chips,
corner-hung count badges, a permanently visible filled arrow, uppercase
mono card titles, and a dotted stipple divider — plus a frame that had
been invisible since the app-wide border tokens were zeroed.
Rework it around what the borderless direction actually implies:
- Feature tiles get a whisper-faint surface instead of a dead frame, and
read as three bands (bare glyph + count / title + arrow / description).
`--card-hue` is spent sparingly — the glyph at rest, the surface, count
and arrow only once raised. Titles move to sans sentence case; counts
are plain tabular numerals. Lift softened 4px -> 2px, coloured glow ->
neutral shadow, plus an explicit focus ring and a staggered entrance.
- Hero drops the boxed "646" pill and the filled A/B-Compare button for
quiet type, with a hairline standing in for the separation.
- Section labels trade the dotted stipple for a single fading hairline;
rows are transparent until hover and reveal "Open" on hover/focus (it
stays in the DOM, so AT and keyboard always reach it).
- Hero, tiles, recent files, callout and project lists now share one
1180px column — previously only the top half was capped, so lists ran
edge-to-edge on a wide display while the deck stayed centred.
Two bugs found and fixed while doing it:
- Buttons that had `border border-solid border-transparent` removed fell
back to the UA default border and rendered a visible 1px outline. They
now carry `border-0` explicitly.
- `.lp-animate` used `animation-fill-mode: both`, so after the entrance
it kept owning `transform` — and animation-origin declarations outrank
normal ones, which silently killed the card hover lift. Now `backwards`,
which still holds the from-state through the stagger delay.
Also drops CSS the page has not rendered since #904: the cursor-spotlight
layer, the breath ring, and the per-card waveform strip.
Verified with headless renders at 1600/1280/940 and the empty state.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(dictation): decode Wayland portal signals and show the capture pill
The GlobalShortcuts portal declares Activated/Deactivated as
(o session, s shortcut_id, t timestamp, a{sv} options). We decoded the
timestamp as u32, so zbus rejected every signal with
Signature mismatch: got `(osta{sv})`, expected `(osua{sv})`
and the press was dropped as an invalid signal. Registration succeeded
and the desktop even reported the bound chord back, so the hotkey looked
wired up while doing nothing at all — on every Wayland compositor, for
the whole life of the feature (#1490). Decode the 64-bit timestamp, and
keep the 32-bit spelling as a fallback so a non-conforming portal
degrades to working rather than to silence.
With presses arriving, the second half of the failure showed: nothing
had shown the widget window since it became a hidden recorder host, so a
capture ran with no pill on screen — and a mic or Accessibility failure
rendered into a window nobody could see. Add show_dictation_pill, which
bottom-centres the capsule on the monitor under the pointer and shows it
without taking focus (Windows keeps SW_SHOWNOACTIVATE so paste still
lands in the user's document), and call it from the widget for every
state but idle. Wayland denies clients their own placement, so the
compositor picks the spot there; the pill still appears.
dispatch_dictation_capture now logs whether a press was emitted or
queued — a press that reaches Rust and produces nothing was otherwise
indistinguishable from one the compositor never delivered.
Tests: portal signals decode at both timestamp widths (the 64-bit case
fails before this change with the exact production error); pill
placement centres, respects a second monitor's origin, and clamps rather
than going off-screen; the widget shows for a state needing the user,
stays hidden while idle, and never shows for a press that arrives while
dictation is disabled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: sync in-progress workspace changes
Uncommitted work already in the tree, checkpointed so the branch matches
the local machine:
- Remote GPU workers: join-from-the-app flow, one-time secrets, QR join
codes, a Compute control in the status bar, and the device-list
Workers panel (#1516)
- Model Catalogue workspace, with Settings pointing at it
- Settings sidebar search and keyboard navigation
- Demo assets for dubbing, dictation and voice design, plus the scripts
that render them
- Backend: validation-error handling, ASR request-path degradation, and
the accompanying tests
- CHANGELOG entries for the above and for the Wayland dictation fix
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): follow Engines to the Model Catalogue, and green the sweep
- test_supertonic3 asserted the license gate points at "Settings" while
the engine now names Model Catalogue → Engines, which is where the
accept button actually lives. The assertion follows the move; what it
pins is unchanged — the hint must name a place the user can reach it.
- Carries the CJK allowlist entries for the rendered dub bundle (#1517)
and the regenerated route snapshot for /workers/agent (#1516), both of
which this branch inherits from the workspace sync.
- docs/install/linux.md: the dictation capsule is bottom-anchored
everywhere except Wayland, where the protocol gives applications no
say in their placement. Documented rather than left as a surprise
(CodeRabbit).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: stop a flaky dependency fetch from failing green runs
en-core-web-sm resolves to a direct GitHub release URL, and github.com
intermittently answers `http2 error: refused stream before processing
any application logic`. uv's own three retries all land within the same
few seconds and fail together, so the whole job dies on a dependency
that has nothing to do with the change under test — it cost #1518 and
#1517 an otherwise-green run tonight.
Two changes: back off between whole `uv sync` attempts, which is what
actually clears it, and pass --no-sync to the pytest steps. `uv run`
re-resolves the environment before running, so every test step was a
fresh chance to hit the same fetch even though the install step had
already synced — that is exactly how #1518 failed, in the isolated
backend/tests step, with all 5467 tests already passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: one retry seam for every uv sync, not just the job that failed last
en-core-web-sm resolves to a direct GitHub *release* URL rather than a
package index, and github.com intermittently answers `http2 error:
refused stream before processing any application logic`. uv's own
retries all land inside the same ~10 seconds and fail together, so a job
dies on a dependency unrelated to the change under test. Tonight that
cost four otherwise-green runs across #1515, #1517 and #1518 — and the
first fix only covered the Tests job, so the next failure simply moved
to Smoke (Linux), which syncs separately.
The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh
backs off between whole attempts (15s, 45s, 90s) and every workflow that
syncs now goes through it — ci.yml (tests + the platform matrix),
release.yml, security.yml, evals.yml. It still fails loudly after four
attempts, so a genuinely broken lockfile is not disguised as a flake.
The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke
matrix has always set, which is part of why it was the one that kept
dying; it has them now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(ci): pin the Intel-Mac contract by intent, not by command spelling
test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted
the literal line `run: uv sync --extra pockettts`, so routing every sync
through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The
contract it exists to protect is that the pockettts extra installs ONLY
on backend_supported legs — which the regex now pins, while leaving how
the sync is invoked free to change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: keep every uv run out of the resolver, and bound the retry budget
CodeRabbit, #1517:
- `uv run` re-resolves before running, so the smoke suite, the
worker-artifact tests, the release test run and the eval run were each
a fresh chance to hit the flaky direct-URL fetch outside the retry
loop. All of them pass --no-sync now; the environment is already
synced by the step that owns the retries. security.yml's
`uv run --with pip-audit` is deliberately left alone — it layers an
ephemeral package rather than running the project's own tests.
- The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a
120 s timeout on the smoke matrix). Three attempts and 60 s of total
backoff outlast the refusals actually observed while staying well
inside the jobs' timeout-minutes.
- The Intel-Mac contract test pinned the smoke command literally too, so
--no-sync tripped it exactly like the sync line did. Same fix: assert
the contract (smoke runs only on backend_supported legs), not its
spelling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decode MLX Whisper audio once through VoiceStudio's validated bundled ffmpeg and reuse the waveform for forced alignment. This restores source-install transcription on Apple Silicon without requiring system ffmpeg.\n\nThanks @gambletan!
CTranslate2 (the engine under WhisperX and faster-whisper) requires cuDNN 8 while torch ships cuDNN 9. When the side-loaded cuDNN 8 is absent it does not raise — it __fastfail()s, killing the whole backend with 0xC0000409 and no traceback, so the shell restarts it and the next attempt dies the same way.
The defect was that the backend computed the answer and discarded it: the preload passed silently on a missing directory and on every OSError, then called into a library that treats the same condition as fatal. The answer is now kept, and the CTranslate2 engines report themselves unavailable so auto-detect falls through to pytorch-whisper.
Two more instances of the same class went with it: the crash-isolated ASR sidecar is a child process that never preloaded at all (failing every transcribe, quietly), and the preload only searched <project root>/.venv, missing any other interpreter.
Conservative by design — a false positive costs WhisperX's forced alignment, so ROCm is excluded via torch.version.hip before cuda.is_available(), which is True on HIP builds.
14 regression tests; backend suite 4374 passed.
Renames what users see. The app, the installers, the window title, the
docs and all 21 locales now say VoiceStudio, with "(previously
OmniVoice-Studio)" noted near the title of each doc surface so people
recognise it.
Deliberately NOT renamed, because renaming any of them silently breaks
an existing install — there is no legacy-path fallback anywhere in this
codebase:
- bundle identifier com.debpalash.omnivoice-studio (MSI UpgradeCode,
macOS TCC grants, managed venv, WebView localStorage, the
single-instance lock)
- data directories OmniVoice / .omnivoice and omnivoice.db
- the ~150 OMNIVOICE_* environment variables
- the X-OmniVoice-* HTTP headers (a wire protocol)
- the published Docker image paths
- the OmniVoice ENGINE, which is a model name and not this product
tests/test_identity_paths_survive_the_rename.py pins every one of those
so a future well-meaning sweep cannot orphan a user's library.
Linux .deb users install a new package name and should apt remove
omnivoice-studio; that note is in the changelog.
The TRANSFORMERS_IMPORT hint and the ASR pipeline error told users to reinstall torch + torchaudio + transformers. torchvision — the package whose ABI mismatch actually produces this exact lazy-import wording (#1357's torchvision::nms, wrapped into "Could not import module 'AutoFeatureExtractor'") — was the one package the advice omitted. Following it to the letter left the broken package untouched (#1376).
Both surfaces now name the mismatch as a cause and prescribe the pinned reinstall with literal versions (desktop installs ship no deploy/, so the constraint-file form fails there) targeting the venv explicitly. A lockstep test asserts the exact command on every advice surface against deploy/torch-constraints.txt, so a pin bump stays red until the advice matches. Docs gain the same-wording-different-cause section (1a-bis).
- fallback preflight pins the candidate backend id (Greptile) — also the
CI empty-cache failure: deep-import fall-through tests get the
asr_model_installed fixture
- locale ratchet: improvement warns instead of failing CI (CodeRabbit)
- stream-exit ASR unload runs on the GPU pool fire-and-forget instead of
blocking the event loop (CodeRabbit)
- E741 rename; M4A ftyp sniff scope documented (CodeRabbit)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Harvested and verified every CodeRabbit/Greptile finding from PRs #1175,
#1189, #1192, #1195: 16 real ones fixed (fallback ASR preflight bypass,
VRAM release on stream exit, typed 409 parity, uv env independence,
path-privacy in errors, MCP clone_voice hardening, CaptureWidget WS
guard, test hygiene), 4 refuted with evidence, rest documented as
deliberate design or deferred.
Deterministic CI replaces hand-enforcement: tests/test_changelog_style.py
(quiet one-liner format) and tests/test_locale_parity.py (21-locale
key/placeholder lockstep with a ratchet baseline) — the latter surfaced
and fixes 151 already-broken locale strings. CodeRabbit/Greptile carry
the house rules via .coderabbit.yaml + greptile.json; CLAUDE.md gains
the harvest-before-merge and never-accept-as-is rules.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- validate managed binaries before exec; 0-byte GGUF placeholders fail
actionably instead of "Exec format error" (#1172)
- KittenTTS: tokenizer-measured chunking to the ONNX 512-token cap;
clear 400 for unspeakable input (#1173)
- clean SIGTERM during weight load: shutdown-aware loader, benign
cancelled-load classification, lifespan hardening, scoped log
silencers (transformers load + alembic fileConfig) (#1174)
- broken ASR deep-imports (lightning_fabric) mark the engine
unavailable with a repair hint and fall through (#1185)
- uv cache + managed Python follow the chosen install drive on
Windows (cherry-picked cross-drive class fix + spaces/D: tests) (#1186)
- adaptive silence-removal ladder for quiet clone references; localized
actionable error for truly silent clips, all 21 locales (#1188)
- CHANGELOG: consolidated Unreleased into the quiet one-liner style
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The curated default `sherpa-parakeet-tdt-v3` installs cleanly, loads without
error, and returns an empty token list for clear speech — while whisper and
zipformer transcribe the same bytes. Ruled out: quantisation (fp32 fails too),
sherpa-onnx version (1.13.3 and 1.13.4), decoding method (greedy and
modified_beam), and model_type (explicit and auto-detect). The defect is inside
sherpa-onnx's NeMo-TDT decoder and cannot be fixed here by configuration.
The previous commit made that survivable — a silent model falls back to the
capture ASR engine for the session. This makes it stop recurring.
Hard-coding a different default per OS would be a guess: there is evidence for
Windows only, and blanket-changing the curated pick could regress a platform
that works. So the app observes rather than predicts. When a session hears
speech-level audio and the model returns nothing, that model is demoted ON THIS
MACHINE and is no longer auto-selected; dictation routes to the capture ASR
engine instead. Self-correcting wherever the breakage actually is, a no-op
everywhere it isn't, and no round trip repeated on every later session.
Demotion is never a dead end: explicitly choosing a model in Settings clears it,
so a sherpa upgrade that fixes the decoder is reachable and the user stays in
charge.
Verified end-to-end on a clean backend: with parakeet pinned and no demotions,
one speech session demoted it ("demoted on this machine" logged), after which
dictation_model_id() returns None (capture ASR engine) while the pref is
untouched — and re-picking the model restores it. 12 tests cover the round
trip, per-model scoping, and the user-regains-control path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Only the TTS model (~2.4 GB) is required on first run; ASR models are
per-platform curated picks (curated_on in models.yaml) installed on demand.
Every transcription surface returns a typed asr_model_missing error with a
one-click download CTA instead of silently pulling multi-GB Whisper weights.
Settings -> Models is a grouped, platform-aware catalog. New guided
permissions UX (wizard System Check + Settings -> Permissions + mic
pre-flight) with native mic-state checks and OS settings deep-links. New
parakeet-mlx engine brings Parakeet TDT v3 to Apple Silicon (language-gated
capture preference so multilingual dictation never regresses). Docs:
expressive-speech page, Flush/Unload + CPU-fallback triage, clone-length FAQ.
Hardening: preflight fails open for custom model pins, ROCm curation no
longer inherits NVIDIA picks, Windows mic probe reads the NonPackaged
consent key, CaptureWidget setup race fixed, offline-cache CI simulation
fixes so empty-cache runners stay green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(engines): contain SystemExit at the pool boundary — a CLI-shaped dependency killed the backend (#1133)
Auto-report #1133 (8GB M1, v0.3.21, engine mlx-audio, exit code 1 at 21s
uptime) carried the whole story in its stderr tail: mlx-audio's Kokoro
pipeline uses misaki's G2P, whose __init__ runs spacy.cli.download() IN
PROCESS when en_core_web_sm is missing. spaCy's downloader is written as a
CLI: with no pip in the venv (uv-managed venvs ship none), its error printer
calls sys.exit(1). SystemExit is not an Exception, so every except Exception
on the path waved it through; it rode the executor future into the event
loop, where uvicorn treats SystemExit as "shut down" — backend dead.
Class fix, not a spacy special-case: _contain_system_exit() wraps every
callable dispatched through run_on_gpu_pool_guarded (all engine loads AND
generates funnel through it, #1033) and asr_backend.run_transcribe_guarded,
converting SystemExit into a RuntimeError that names the real failure mode.
Any engine dependency written as a CLI is now covered on both the TTS and
ASR sides.
Not done here (follow-up candidates): pre-provisioning en_core_web_sm for
the Kokoro/mlx-audio path so the download never triggers, and/or shipping
pip into the managed venv. Both are provisioning decisions; this PR makes
the failure survivable and honest first.
Tests: SystemExit from a pool job -> RuntimeError naming SystemExit(code),
executor still usable afterwards; same for the transcribe guard. Both fail
with the containment reverted. Full suite: 3016 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(engines): containment helper moves to a leaf module (CodeQL cyclic-import)
utils/containment is stdlib-only, so model_manager and asr_backend both
import it at module top with no cycle — the call-time back-import CodeQL
flagged is gone.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(asr): Apple Silicon dubbing ran Whisper on the CPU — pick the engine by hardware
_auto_detect() probed WhisperX first, unconditionally, with no device check. WhisperX
is always installed, so it always won — and WhisperX (like faster-whisper) is CTranslate2,
which has NO Metal backend. On every Mac, dub transcription therefore ran whisper-large-v3
on the CPU while the GPU sat idle. The MPS branch below it was unreachable in practice.
Measured on an M2, one 30 s dub chunk of large-v3:
WhisperX (CPU) 90.4 s <- 3x SLOWER than realtime
MLX (GPU) 20.5 s
MLX (GPU) + forced align 20.3 s <- ~4.4x faster, identical word timings
That is a 16-minute video taking ~48 minutes and looking like a hang. It is also why the
slowest chunks exceeded OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S (120 s) and were ABANDONED
after 2 attempts (chunks 33/34 in the reported run), losing transcript outright — while
#730's advice blamed a "VRAM-starved GPU" on a machine with no VRAM.
The engine pick is now device-aware: Apple Silicon gets MLX, everywhere else is unchanged
(on CUDA, WhisperX already uses the GPU and remains the default).
Crucially this does NOT buy speed with lip-sync accuracy. WhisperX's value is its wav2vec2
forced alignment (±10-30 ms word boundaries vs Whisper's ±100-300 ms), and dub lip-sync
depends on it. Alignment takes *plain segments*, so it is independent of whichever engine
produced them: it is extracted into forced_align() and layered on MLX's GPU transcript.
Verified — the boundaries agree with WhisperX's (multiple: 0.62 vs 0.71; different: 1.57 vs
1.55) and every word carries timing. Alignment prefers MPS (20.3 s vs 28.4 s, byte-identical
timings) and falls back to CPU rather than silently dropping to loose timestamps.
Also fixes the ETA, which was pure fiction: TranscribeOverlay estimated
`ceil(duration/60)*3 + 8` seconds — an assumption of ~20x-realtime transcription. For a
16-minute video it predicted 56 s against a real ~48 min, then clamped to "~0s remaining"
with the bar frozen at 95% for the rest of the job. The backend already streams a real
progress fraction (dub_core.py emits `progress` on every `segments` event) and the UI simply
ignored it. It now extrapolates from the observed rate, and shows nothing until it has a
rate to extrapolate from.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style: oxfmt
Formatting only — no behaviour change. CI's format:check gate (not run locally
before the push) rejected the two new/edited dub files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measured on a 16 GB M2: a generate on omnivoice (~2.8 GB core) followed by a
generate on mlx-audio left BOTH resident (footprint 3.9 → 4.3 GB) because the
OmniVoice core lives in model_manager.model while every other engine lives in
engines._ENGINE_INSTANCES — two caches that never coordinated, and the latter
was never unloaded. That accumulation is a direct contributor to the memory
pressure behind the "Can't reach the local backend" OOM deaths.
- services/engine_memory.py: evict_other_tts_engines(keep_id) unloads every
OTHER resident TTS engine before the incoming one loads — spans both stores
(the OmniVoice core under its async lock, and the instance cache). No-op when
nothing else is resident, so steady-state single-engine use pays nothing; only
a real switch evicts. Default on; OMNIVOICE_SINGLE_ENGINE_RESIDENT=0 to keep
several warm. Wired into the /generate path right after the engine resolves.
- TTSBackend.unload() (the ABC default) now actually frees the held model: it
clears _MODEL_ATTRS (_model/_tts) and empties the device cache. Every
in-process engine but OmniVoice previously inherited a NO-OP unload(), so an
engine switch dropped the instance ref but left its model for GC with the GPU
cache un-emptied. One change fixes all of them and is future-proof.
- FasterWhisperBackend.unload() cleared self._asr — an attribute it never
assigns — so its model in self._model was never freed. Fixed.
Live-verified: omnivoice → mlx-audio now DROPS footprint 2300 → 1541 MB (core
evicted) instead of climbing to 4305 with both resident. 7 new unit tests
(eviction spans both stores / keeps the active engine / no-op when disabled /
a failing unload doesn't abort the sweep / ABC unload frees + is idempotent),
order-independent. Backend suite green.
Refs the 16 GB OOM class (#1076#1092#1093#1101)
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Four "Can't reach the local OmniVoice backend" reports (#1076/#1092/#1093/#1101)
all died at the same moment: during a generate, on a 16 GB machine. Measuring it
(phys_footprint, not RSS — RSS badly undercounts MPS unified memory) showed the
generate was never the problem: it costs ~116 MB. The problem is the BASELINE —
the backend sits at ~6.2 GB *idle*: TTS 3.8 GB plus ~2 GB of warm capture ASR.
The TTS model has always been idle-unloaded (model_manager.idle_worker). The
capture/dictation ASR singleton never was — one dictation warmed it and it
stayed resident for the life of the process. So the app dutifully freed 3.8 GB
of TTS while silently holding 2 GB of ASR forever, and on a 16 GB Mac that
baseline plus the app plus macOS is enough for the OS to kill the backend
mid-generate. That death surfaces as the "can't reach the backend" error — the
class #1102 made honest and this fixes at the source.
- asr_backend.release_idle_capture_backend(idle_s): unloads the warm capture
singleton once it's been unused that long; no-op under a live lease, when
nothing's loaded, or when recently used; never raises (idle_worker calls it
on a loop).
- capture_lease(): pins the singleton for a live dictation session's whole life
(the stream holds the backend without re-resolving it, so the reaper must not
unload the model mid-sentence); wrapped around both sherpa handlers in
capture_ws. Releasing restarts the idle clock.
- Both capture getters stamp _touch_capture() so any handout resets the clock.
- idle_worker runs the reaper each tick with the same idle timeout the TTS model
uses, then free_vram().
Cost: a ~1.4 s model re-warm on the next dictation after a full idle timeout —
the same bargain the TTS model already makes. 8 new unit tests (releases when
idle / never while leased / never when recently used / lease released on raise /
nested refcount / failing unload still drops the ref / no-op when empty).
Backend suite 2905 passed; verified live against the running backend.
Fixes the crash class behind #1076#1092#1093#1101
Co-authored-by: mergetest <nizam4103@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The openai-compat-asr backend (#877) shipped with settings routes but no
discoverable activation path: the config panel hid in Settings → Models,
its hint text claimed "there's no in-app engine picker for ASR yet"
(stale — the matrix has one), and there was no way to check a server
actually answers before pointing a dub/dictation run at it.
Configure → test → activate now live on one screen, Settings → Engines:
- The ASR family tab mounts the config panel (URL / model / optional
API key) below the engine matrix; saving refetches the matrix via a
new reloadToken prop so the engine row flips unavailable → available
and its "Use" button appears without a manual refresh.
- New "Test connection" button + loopback-gated
POST /api/settings/asr-openai-compat/test: saves first (same
stale-config contract as /llm-providers/{id}/test), then probes
GET {base_url}/models — no audio leaves the machine. The structured
verdict maps to localized, actionable messages: latency + whether the
configured model is listed on success; classified auth_failed /
http_error / timeout / unreachable / ok_no_models failures. detail is
core.scrub-ed; the key is never logged or echoed.
- Engine reads persisted config fresh per transcribe (regression test) —
config changes need no backend restart. Never default-active: ASR
auto-detect only picks local engines.
- i18n for every new string (en.json); no hardcoded CJK; identical
behavior on macOS/Windows/Linux (pure HTTP + React).
- Docs-sync: docs/engines/openai-compatible-asr.md rewritten around the
one-screen flow with LM Studio / llama.cpp / Groq / OpenAI examples
and the privacy note; README engine table cell updated.
Verified end-to-end against a fake OpenAI-compatible server: UI drive
(configure → test → row flip → Use) plus a real transcription through
the backend's /v1/audio/transcriptions immediately after a config
change, no restart.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Three-part fix for the v0.3.5-comparison report:
1. Perf: since v0.3.6 (#308), a clone reference without a stored
transcript triggered a FULL ASR model load + transcribe on every
/generate — get_active_asr_backend() builds a fresh whisper backend
per call. Measured live: 92.7s wall vs 14.9s of actual TTS. Now the
first auto-transcript is persisted onto the (unlocked, clone-kind)
profile row, and transcribe_reference caches results by audio
content hash (bounded LRU, no model/VRAM held), so the cost is paid
once per clip, not per request. User-typed transcripts are never
overwritten; locked/design profiles are excluded from the persist.
2. Clear History: the workspace UX overhaul (#374) moved history into
the right-side WorkspaceHistory panels and dropped the old Sidebar's
clear-all control (the Sidebar is now hidden in every mode). Both
the Voice and Dub panels get a scoped Clear History button wired to
the existing DELETE /history and /dub/history endpoints, with the
same confirm dialog the Sidebar used.
3. Auto-play: the finished-render playback (playBlobAudio) has no
on-screen player and the only stop lived in the Voice ActionBar's
CTA morph — unstoppable from the Dub workspace, profile pages, or
after navigating away. A global PlaybackStopPill now appears for any
'output' playback on every page. The existing Settings → Appearance
"Auto-play preview" pref (#667) now also gates the generate path,
as its label always promised (default ON — no behavior change).
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
First slice of the community's two-track proposal for #877: a generic
OpenAI-compatible ASR backend that works TODAY, without waiting on
transformers to ship a direct Qwen3-ASR integration (tracked separately,
still blocked upstream). Points OmniVoice's transcription at any server
exposing POST /v1/audio/transcriptions — a self-hosted Qwen3-ASR/
FunASR/SenseVoice server, or OpenAI's own API.
- New OpenAICompatASRBackend (backend/services/asr_backend.py): a pure
network client, no local model, no install. Prefers
response_format=verbose_json for real per-segment timestamps,
degrades to plain text (matching MoonshineASRBackend's shape) when a
minimal server rejects that format. Never leaks a raw SDK/httpx
exception to the caller (#977 convention) — wraps network/auth
failures in a clean, actionable RuntimeError naming the server.
- Settings persist via the same encrypted-secret convention as
services/llm_providers.py (settings_store.set_secret for the API key
— Fernet-encrypted, never a .env row, never echoed back; get_text/
set_text for base_url/model). New GET/PUT /api/settings/
asr-openai-compat, loopback-gated like every other settings route.
- Frontend: a small settings panel (Settings → Models) mirroring
HFMirrorPanel's exact structure. No ASR engine picker exists yet for
ANY ASR backend (only TTS has one) — activating this engine still
needs OMNIVOICE_ASR_BACKEND=openai-compat-asr; documented plainly
rather than pretending otherwise.
- README's ASR Engines table (9 → 10 engines) and docs/features.yaml's
drift-checker inventory updated; the '9 engines, all fully local'
claim corrected since this one genuinely isn't.
- docs/engines/openai-compatible-asr.md: setup steps + an explicit
privacy note (unlike every other ASR engine, audio leaves the
machine to whatever server is configured).
Regression tests: tests/test_asr_openai_compat_877.py (12 tests) —
is_available() gating, verbose_json + plain-text response adaptation,
network-failure error hygiene, SDK retry disabling, and the settings
endpoints' persist/mask/clear-vs-unchanged semantics.
Fixed two real full-suite-only failures found during verification (not
brushed aside): the API route inventory snapshot needed regenerating
for the two new routes, and this file's own tests had a module-
staleness bug — a collection-time settings_store import went stale
relative to a test-time-fresh fixture when another test elsewhere in
the ~2400-test suite reimports the module — fixed by making
settings_store itself a fixture resolved at test-run time, same
lesson already applied to tests/test_mm2_lifecycle.py earlier this
session.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The Engines page told users to run `pip install nemo_toolkit[asr]` for
the NeMo Parakeet ASR engine. nemo_toolkit[asr]==2.7.3 hard-pins
transformers>=4.57,<4.58, which is unsatisfiable alongside OmniVoice's
own transformers>=5.3 requirement (needed by
omnivoice/models/omnivoice.py for HiggsAudioV2TokenizerModel). A user
who followed the hint ended up with a backend that wouldn't start
(ImportError: cannot import name 'HiggsAudioV2TokenizerModel').
_INSTALL_HINTS["nemo-parakeet"] in backend/services/asr_backend.py now
states plainly that installing into the shared venv will break the
backend, names the transformers conflict, and tells users to use a
separate/dedicated Python environment instead — without implying a
safe one-line fix or an isolated-venv env var exists (unlike
dots-tts/moss-tts-v15/confucius4-tts, nemo-parakeet has no isolated
venv option yet; that's a separate, larger follow-up).
Also adds one sentence to docs/install/troubleshooting.md's existing
"engine venv clash" section (#11) pointing at the same class of issue
on the ASR side, and a regression test asserting the hint never again
contains the literal bare `pip install nemo_toolkit[asr]` string.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* fix(engines): snapshot lazy registry keys so /engines can't 500 under concurrency
`list_backends()` runs in a FastAPI threadpool and iterates the lazy TTS/ASR
registries via `items()` → `__iter__`, which held a *live* `dict.__iter__(self)`
open across each engine's slow `is_available()` probe. Meanwhile the lazy
`__getitem__` resolves a deferred entry by mutating the dict (`self[key] = cls`).
A second concurrent `/engines` request (or any ASR op) materializing the lazy
`faster-whisper-isolated` entry therefore changed the dict size mid-iteration:
RuntimeError: dictionary changed size during iteration
asr_backend.py:1729 list_backends → _REGISTRY.items()
asr_backend.py:1665 __iter__ → for k in dict.__iter__(self)
Both `_LazyRegistry` (TTS) and `_LazyASRRegistry` (ASR) now snapshot their live
keys up front with `list(dict.__iter__(self))` — consumed atomically under the
GIL — so a concurrent lazy insert can no longer trip the iteration. The slow
per-engine probes then run over the snapshot, not the live iterator.
Deterministic fail-before/pass-after regression for both registries:
tests/backend/services/test_lazy_registry_concurrency.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(changelog): add the /engines concurrency fix under [Unreleased] (#940)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
P0 — Refinement blocked every dictation final with no timeout. With refinement
auto:true and a slow/dead LLM endpoint, maybe_refine ran unbounded and blocked
the final send in all three capture_ws handlers (~51s measured; the pill hung
"Transcribing…" until the widget's 15s fallback fired). Fix the class: a hard,
env-tunable budget (OMNIVOICE_REFINE_TIMEOUT_S, default 4s) via a new
maybe_refine_async — a slow/dead endpoint now falls back to the unrefined (but
polished) text within the budget and can NEVER delay the final beyond it. The
LLM HTTP call is bounded to the same budget so the orphaned worker unwinds
instead of holding a connection for the client's full 45s. Refinement is now
also fully best-effort in the legacy handler (it can't turn a good final into
an error frame).
P1 — REST /transcribe lacked polish parity. capture.py never applied
polish_text, so REST returned raw "…test" while the WS returned "…test."
Apply text_polish.polish_text to `text` and `refined_text` (segments stay raw),
so the widget POST fallback and MCP/CLI callers match the live socket.
P1 — The #888 "instant first dictation" preload was a no-op. The preload called
warmup() only `if hasattr`, but SherpaDictationBackend had none, and the WS
handlers built a FRESH backend per session so a warm singleton wasn't reused.
Add SherpaDictationBackend.warmup() (builds the recognizer) and share one warm
recognizer per model id across sessions (get_sherpa_dictation_backend, same
invalidation + a shared lock as the capture singleton); each session keeps its
own decode stream. First dictation no longer pays the 1.3–2.5s load.
P1 — llm_ready is a lie (feeds the P0). It only means "an endpoint is
configured", so a placeholder key reads as ready. The P0 timeout makes a dead
endpoint harmless; add last_refine_status so RefinementPanel flags a
configured-but-failing LLM and links to LLM Providers → Test.
Regression tests (fail-before/pass-after): slow-LLM WS final arrives < budget;
maybe_refine_async hard timeout + status; REST polish parity + refined_text
polish; warmup builds the recognizer and a second session reuses it; the panel
honesty note. Backend refinement/capture_ws/capture/sherpa suites, CJK + route
inventory gates, full vitest (733), lint (0 errors) and format all green.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Residual A — the chunked dub-stream had a PARALLEL wedge mechanism (its own
ping-loop timeout, its own _reset_pool_on_wedge, a dead-end "Try restarting
the server" message). A wedged chunk now routes through the SAME
run_transcribe_guarded bound+reset as the whole-file paths (#731/#851): the
guard resets the poisoned pool once per wedged attempt (no double-reset on
retry) and the user sees the actionable ASRTimeoutError. The reset logic is
extracted to asr_backend.reset_pool_after_wedge — one shared mechanism, so
the semantics can't drift again. run_transcribe_guarded also gains a
timeout_env param so chunk errors name OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S
instead of the whole-file knob.
Residual B — the crash-isolated ASR sidecar (#393, faster-whisper-isolated)
is wired as an explicit ESCAPE HATCH, not a default:
- selectable end-to-end: Settings engine list gets an explanatory
install_hint; honest gpu_compat ("cuda","cpu" — it wraps the same
CTranslate2 engine as faster-whisper); get_active_asr_backend now hands
back a process-wide singleton for subprocess-isolated backends (a fresh
instance per request would leak atexit hooks and respawn the sidecar —
reloading its model — on every transcribe).
- on the SECOND consecutive guarded timeout-with-reset in one session
(resets aren't recovering the hang; the wedged thread keeps its VRAM),
the error the user sees + the log recommend switching to the isolated
engine in Settings → Engines. Never auto-switched (owner rule: no silent
behavior divergence); a completed transcribe resets the streak.
Tests (fail-before/pass-after verified against origin/main): wedged-chunk
SSE integration (reset count + actionable error + recommendation surfaces),
consecutive-timeout streak (fires at 2, resets on success, suppressed when
already on the isolated engine), timeout_env parametrization, shared-reset
helper, isolated backend in list_backends with hint + honest availability,
singleton caching, gpu_compat matrix entry. Docs: troubleshooting §14 gains
the chunk knob + escape-hatch guidance.
Closes the residuals tracked on #730.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
A whisperx/CTranslate2 transcribe can hang hard on some Windows+CUDA setups and
never return. ASR shares the small (1-2 worker) _gpu_pool with TTS, so one stuck
worker starved every other request — the next TTS generate then surfaced as
"Can't reach the local backend" though the process was alive (#720/#721/#723).
Two parts:
- Bound the three remaining unguarded whole-file transcribe paths (dub
whole-file dub_core.py, batch.py, live-dictation capture_ws.py) with
run_transcribe_guarded, matching the dub-QC/dictation/OpenAI paths that were
already bounded by #656.
- On timeout, run_transcribe_guarded now calls executor.reset() when the pool
supports it (_ResilientGpuPool, already built for the model-load-timeout case
in #589/#599): the wedged worker is abandoned and the next submit gets a fresh
one, restoring capacity without an app restart. Best-effort — a plain
ThreadPoolExecutor (tests) just gets the bound + actionable error.
Regression tests: pool.reset() is invoked on timeout; a non-reset pool still
bounds cleanly.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On hardened kernels / newer glibc (e.g. WSL2 glibc 2.43) CTranslate2's shared
object is rejected at load with 'libctranslate2…cannot enable executable stack'
— an OSError, not ImportError. The WhisperX/faster-whisper is_available() probes
only caught ImportError, so the OSError escaped and crashed the ASR/dub
preflight ('ASR backend initialization failed: …').
- Both probes now also catch the non-ImportError load failure and REPORT
(False, 'failed to load …') instead of raising — a probe must never raise.
- _auto_detect() routes every probe through a never-raising _probe_available()
so no exploding probe can crash engine selection; it falls through to
pytorch-whisper (transformers, no CTranslate2), which works on CUDA/CPU.
Regression tests cover the raising probe, the .so-load OSError surfacing as
unavailable, and auto-detect falling back to pytorch-whisper.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(dictation): live local dictation via sherpa-onnx + Voice settings panel
Add a sherpa-onnx ASR engine alongside the existing Whisper/NeMo dictation
path, powering a genuinely live experience: as you speak, words type straight
into the focused field (streaming partials via a new simulate_type command,
self-correcting with backspaces) and commit per pause.
Backend:
- SherpaDictationBackend + sherpa_dictation registry of the 7 models (Parakeet
TDT v3/v2, streaming Zipformer EN/ZH/bilingual, Paraformer bilingual, Whisper
Tiny) from csukuangfj/* int8 HF repos; CPU provider for cross-platform parity.
- /dictation/models + /dictation/prefs router; get_capture_asr_backend() honors
the selected dictation model. get_active_asr_backend() (dub transcription) and
the legacy WebM/Opus capture path are untouched.
- True streaming over /ws/transcribe (OnlineRecognizer: live partials +
per-endpoint finals); offline models surface partials via short re-decode.
Frontend:
- New "Voice" settings panel (enable, Toggle/Hold mode, model picker with
offline/streaming/recommended badges + per-model download/delete).
- Live word-by-word typing via simulate_type (enigo) with prefix-diff delta and
backspace correction; paste fallback retained, no double-insertion.
Deps: sherpa-onnx>=1.13.3 (+ sherpa-onnx-core); uv.lock regenerated, Docker
frozen-install verified. API route-inventory snapshot updated. 40+ new tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(dictation): register sherpa-onnx-asr engine in README + features inventory
Fixes the docs-drift CI guard: the new sherpa-onnx-asr ASR engine existed in
the registry but not in docs/features.yaml or README. Adds the live-dictation
engine row to the ASR Engines table, bumps the engine counts (8→9), and adds
the inventory entry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(changelog): fold live-dictation into the [0.3.8] section
main is 0.3.8 (untagged), so the dictation feature belongs in that release, not
a separate [Unreleased] block. Merge the two Added lists under one [0.3.8],
refresh the headline to lead with live dictation, and correct the capture
description to reflect live word-by-word typing (not paste-on-pause).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A Windows/CUDA user (Vietnam) hit "Can't reach the local backend" only when
dubbing/transcribing. Their log proves the backend started fine — model loaded,
preload complete, 25 models — and the log ends right after
`whisperx transcribing …tmp.wav`. The backend was alive; the *transcription*
stalled (large-v3 ASR contending with the resident TTS model for VRAM on an
8 GB-class GPU), which the UI surfaces as an unreachable backend.
Root cause (class, not instance): the chunked dub pipeline already bounds each
chunk (OMNIVOICE_TRANSCRIBE_CHUNK_TIMEOUT_S), but the *whole-file* transcribe
paths ran unbounded:
- dub QC re-transcribe (dub_export)
- dictation (capture)
- OpenAI-compat /audio/transcriptions
A slow/stuck transcribe on any of these hung the request AND held a GPU-pool
worker — indistinguishable from a dead backend.
Fix: add run_transcribe_guarded() in services/asr_backend.py — a shared
asyncio.wait_for wrapper (ASRTimeoutError, a TimeoutError subclass) with a
generous env-tunable bound (OMNIVOICE_ASR_TRANSCRIBE_TIMEOUT_S, default 300 s).
On timeout the request returns 504 with actionable guidance (backend is alive;
free VRAM / pick a smaller ASR model / use CPU; restart to clear the stuck
worker) instead of hanging forever. Wired into all three whole-file paths.
Docs: new troubleshooting §14 — "Can't reach the local backend during
transcription/dubbing" — explains it's ASR weight/VRAM pressure, not a network/
mirror problem, and corrects the misconception that a "Network → Restricted/Global
mirror" Settings toggle exists (the Network control is LAN sharing). Serves the
#602/#585/#567 "can't reach backend" cluster.
Test: backend/tests/test_asr_transcribe_timeout.py — slow fn raises ASRTimeoutError
with the actionable message, fast fn passes through, subclass-of-TimeoutError so
the openai_compat broad catch still maps to 504.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WhisperX (the default ASR) aborts transcription with zero segments on Windows
only, surfacing "Lazy import of LazyModule(...speechbrain.integrations.k2_fsa...)
failed" (#630) or its generic wrapper "Transcribe stream dropped..." (#611, #647).
Root cause is in speechbrain 1.x. It exposes optional integrations (k2_fsa,
numba losses, spacy/flair nlp) as LazyModule redirects in sys.modules. Stray
introspection during whisperx.load_model (pyannote -> speechbrain) — PyTorch's
op-registration machinery, pickling, a dir()/hasattr walk — touches one of these
redirects. speechbrain suppresses such inspect-triggered imports via a guard, but
the guard checks filename.endswith("/inspect.py") with a hardcoded POSIX
separator. On Windows the frame filename uses backslashes, the guard misses, the
redirect actually imports k2_fsa -> import k2 -> k2 not installed -> ImportError
that bubbles out and kills ASR. macOS/Linux use forward slashes, so the guard
fires and the feature works — a Windows-only break of a cross-platform default
(P0 parity).
Fix the whole class (every optional-integration redirect, not just k2) by
re-implementing LazyModule.ensure_module with a separator-agnostic basename check
(normalise both "\\" and "/"), applied right before whisperx loads. Idempotent;
a no-op on macOS/Linux and when speechbrain is absent; genuine missing-dep
accesses from real user code still raise ImportError — only inspect-triggered
spurious imports are suppressed, now on every platform.
Regression test fakes the importer frame with Windows- and POSIX-style inspect.py
paths plus a real-caller path, so it pins the behaviour on any CI host (fails
before the fix on the Windows-path case, passes after).
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When WhisperX (or any ASR backend) failed to load its model, the transcribe
SSE stream dead-ended on a generic "Transcribe stream dropped … Likely ASR
backend failed to load" message with no actionable cause.
Two root causes, both fixed:
1. WhisperX loads lazily inside transcribe(), so a load failure (faster-whisper
weights, CTranslate2/cuDNN mismatch, torch-2.6 weights-only VAD regression)
was buried in per-chunk errors and retried on every chunk. Added
ASRBackend.ensure_loaded() (no-op default; WhisperX triggers its lazy
loader) and call it in the transcribe pre-flight so the genuine cause
surfaces once, up front, as a structured error event.
2. The pre-flight and audio-load error paths closed the SSE stream with a bare
`error` and no terminal `done`, so the browser's native EventSource
connection-drop could race and win against the structured error — discarding
the real cause. Every terminal error now emits `done`, and the frontend
latches the structured cause so a connection drop can't overwrite it with
the generic message.
Adds a fail-before/pass-after regression test driving the stream's async
generator through the ASR-load-failure path; updates the existing #516 fake
backend to the new ensure_loaded() contract; CHANGELOG ### Fixed entry.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#551: both CTranslate2 ASR backends request compute_type="float16" on CUDA with
NO fallback. On GPUs without efficient fp16 (older Maxwell/Pascal, GTX 16xx) or a
CTranslate2/cuDNN binary mismatch, WhisperModel/whisperx.load_model raise a
ValueError at construction — which escaped the existing OOM-only `except
RuntimeError`, so every chunk failed and the user got "Transcription produced no
segments". Add a per-device compute_type fallback chain (cuda: float16 →
int8_float16 → int8; cpu: int8 → float32) to both backends + the ASR sidecar,
alongside (not replacing) the existing OOM→CPU path, with an ASR_COMPUTE_TYPE
override for exotic hardware (documented in README).
Also in the same ASR-robustness pass:
- #549: PyTorchWhisperBackend._ensure_pipe wraps the transformers pipeline load
and re-raises an actionable error (reinstall transformers / use faster-whisper)
instead of a bare "Could not import module 'AutoFeatureExtractor'".
- #516: the /dub/transcribe SSE generator is wrapped so it can NEVER close
without a terminal event — any unanticipated exception now yields a structured
`error` (with build_failure's hint) + `done`, turning "stream dropped, likely
ASR failed" into the real cause + Retry.
- failure.py: COMPUTE_TYPE_UNSUPPORTED + TRANSFORMERS_IMPORT classes so the
no-segments toast is actionable.
Tests (fail-before/pass-after): float16-unsupported → int8 for both WhisperX +
FasterWhisper; a generic non-OOM RuntimeError still raises; classify() maps the
two new classes; the SSE stream always terminates with error→done. 7 + 1 passed,
17 in the failure suite (no regression).
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WhisperX transcription called `whisperx.load_audio()`, which shells out to a
literal `"ffmpeg"` resolved against the OS PATH. On Windows that resolves to a
WindowsApps alias stub or a corrupt/wrong-arch binary — passing `which` but
exploding at spawn with `[WinError 193] %1 is not a valid Win32 application`.
whisperx only catches `CalledProcessError`, so the spawn-time `OSError` escaped
and the dub/batch path reported the opaque "Transcription produced no segments".
#377 added ffmpeg validation but only for the dub-export path; the transcription
path never went through the validated resolver. Since WhisperX is a default ASR
engine, this is a P0 platform-parity break (works on mac/Linux, fails on Windows).
Fix: decode the audio ourselves in `WhisperXBackend.transcribe` via
`find_ffmpeg()` (which `-version`-probes each candidate and returns the bundled
imageio-ffmpeg / Tauri sidecar) and hand WhisperX the array — bypassing the bare
PATH lookup entirely. This is more robust than a PATH-prepend, which couldn't
fix the imageio case (its binary is named `ffmpeg-<plat>-vN.exe`, not `ffmpeg`).
If no runnable ffmpeg exists, raise a clear, locale-independent error instead of
"no segments". Fixes both the dub and batch transcription paths.
Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(routing): wire effective-device + routing_status into /engines (#21 PR 3/5)
Surfaces the PR-1 probe + resolver through the engine registries so the
matrix UI (PR 5) and the no-silent-fallback gates can consume it.
- `engine_routing.routing_fields()`: shared helper returning the three
serialization-ready keys, centralizing the scrub rule — routing_reason is
scrubbed via `core.scrub.scrub_text` only when truthy, so a None reason
stays JSON `null` (never coerced to "").
- TTS/ASR `list_backends()` each gain `effective_device` / `routing_status` /
`routing_reason`, computed from a SINGLE `detect_host_caps()` call per
request (host caps are constant per process). ASR is brought to full TTS
parity: it now also carries `install_hint` / `last_error` / `isolation_mode`
and a SCRUBBED `reason` (closing a pre-existing ASR token-leak gap) — an
identical 11-key shape across families. ASR also gains the same
is_available()-raises resilience TTS has (degrade to available:false, never
500).
- LLM `list_backends()` reaches 11-key parity too but emits literal
`effective_device:"network"` / `routing_status:"n/a"` / `routing_reason:null`
(NOT via resolve_routing — LLM runs no local GPU model). `LLMBackend.gpu_compat
= ()`. "network" is a label, not a probe — nothing here touches the network.
- `select_engine` host-routing gate: refuses a pick whose `routing_status` is
`unavailable` on this host (400 with an actionable detail), while ALLOWING
`cpu_fallback` (it runs, just slower). LLM is never gated. Defensive `.get`
so legacy payloads still select. New typed `SelectEngineResponse`.
Tests: 11-key shape across all 3 families, well-formed tts/asr routing keys
(+ None-not-"" contract), LLM network/n/a labels, select gate (block
unavailable / allow cpu_fallback / never-gate LLM). Updated the registry
exact-shape test for the 3 new keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(cjk): allowlist docs/specs/ in the hardcoded-CJK guard
PR #429 merged the longform design specs, which legitimately quote functional
CJK (test-fixture descriptions, CosyVoice speaker IDs, multilingual sample
text). The CJK guard scans every tracked file, so those docs turned main red.
Specs are documentation, not shipped UI strings — allowlist the docs/specs/
prefix, matching the individually-allowlisted docs already in the set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Builds on the device probe from PR 1. Backend-only; the routing keys are
wired into /engines in PR 3.
- #390 closed: MLXAudioBackend / MLXWhisperBackend now call the shared
`core.device_caps.mlx_supported()` gate FIRST, before importing the
package. On Linux/Windows/mac-Intel they report unavailable and never
advertise a usable `mps` route, even with a stray mlx wheel installed.
Replaces the ASR backend's ad-hoc inline MPS check with the one shared
rule. (The Wave-4.4 OSError/RuntimeError import-guard is preserved — it
now lives behind the platform gate; its test forces the gate open so the
guard stays the path under test.)
- `ASRBackend` ABC gains `gpu_compat: tuple[str, ...] = ("cpu",)` mirroring
TTSBackend, and each subclass declares its real targets:
whisperx/faster-whisper → (cuda,cpu); mlx-whisper → (mps,cpu);
pytorch-whisper → (cuda,mps,cpu); nemo/funasr → (cuda,cpu);
moonshine → (cpu,). Inert until PR 3 serializes them.
- IndexTTS2 declares `gpu_compat = ("cuda","cpu")` so it stops advertising
the inherited CPU-only default.
- ROCm is deliberately NOT claimed for any ASR engine (or for IndexTTS2):
CTranslate2 has no upstream HIP build, and an unverified `rocm` claim
would route ROCm hosts to a broken GPU path — strictly worse than the
honest `cpu_fallback` the resolver already emits ("declares CUDA only;
ROCm not in its compat set"). The per-engine TTS ROCm audit is a tracked
follow-up that will verify each path before claiming it.
Tests: MLX gate regression (both backends, on/off Apple), ASR gpu_compat
tuples + no-false-rocm invariant, IndexTTS2 override; existing MLX
import-guard test updated for the new gate ordering.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(asr): crash-isolated faster-whisper subprocess backend (Wave 4.2)
Native ASR engines (faster-whisper / CTranslate2) can segfault on GPU
teardown — a process-level crash that kills the whole backend. Running the
engine in a child process turns that into a failed job: the sidecar dies,
the parent raises a decorated error (engine id + device), and the next
request respawns a fresh sidecar.
- services/subprocess_asr.py: SubprocessASRBackend reuses
SubprocessBackend's wire protocol + lifecycle — including
respawn-on-dead-process (_spawn relaunches when the child isn't alive) and
GPU-slot acquire/release — adding a 'transcribe' op (the TTS 'generate'
surface is stubbed). IsolatedFasterWhisperBackend wraps faster-whisper
using the PARENT venv (already a dep — only the process boundary is new);
opt-in via OMNIVOICE_ASR_BACKEND=faster-whisper-isolated.
- engines/_asr_sidecar/main.py: the faster-whisper runner (stdlib wire
protocol; torch/CT2 import lazily so the ready handshake fits the timeout).
- engines/_echo/main.py: a 'transcribe' echo op so the round-trip + crash
recovery are testable without a real engine.
- asr_backend._REGISTRY is now a lazy dict (mirrors the TTS registry) so the
isolated backend lists/resolves without importing the subprocess stack
unless selected.
Tests (echo sidecar, stdlib-only): round-trip, single long-lived sidecar
across calls, crash-mid-transcribe → decorated error + backend healthy +
next call respawns, registry exposure, generate-not-supported.
Spec 7 / parity program Wave 4.2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(asr): deterministic crash test + drift marker for lazy ASR registry (Wave 4.2 CI)
CI surfaced two issues:
- The echo crash test relied on the crash-AFTER-reply hook, whose reply
may still reach the parent (timing-dependent) — and a leaked
OMNIVOICE_ECHO_CRASH from a sibling subprocess test poisoned the
non-crash tests. Fix: a deterministic OMNIVOICE_ECHO_CRASH_NO_REPLY hook
that exits BEFORE replying (guaranteed dead pipe → decorated error), and
the asr fixture clears both crash envs so the round-trip/two-call tests
can't inherit a leak.
- check-docs-drift's _ASR_MARKER didn't match the new lazy registry line
(_LazyASRRegistry({); updated the marker + the self-test fixture.
Verified the no-reply crash hook by driving the sidecar directly
(reply=None, exit 1); drift self-test + real-repo check green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(asr): allowlist the 'segments' op so transcribe replies aren't dropped (Wave 4.2 CI)
The parent's PARENT_INBOUND_OPS frozenset gated inbound sidecar frames but
never included 'segments' — the ASR transcribe reply op. _recv() dropped the
frame as disallowed, tail-recursed, hit EOF, and returned None, so every
transcribe surfaced as a bogus 'sidecar crashed mid-transcription'. TTS
('audio') was allowlisted; ASR ('segments') was missed. Add it (and list
'transcribe' in the informational SIDECAR_INBOUND_OPS), update the exact-shape
allowlist test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
MLXWhisperBackend / MLXAudioBackend is_available() caught only ImportError.
In a PyInstaller bundle mlx's native dylib/metallib can fail to load even
when the package imports, raising OSError/RuntimeError — which would
propagate and crash the registry scan instead of reporting the backend
unavailable. Broaden to (ImportError, OSError, RuntimeError) so the picker
falls back cleanly. 6 tests across all three exception types.
The capture ASR path already prefers MLX Turbo on Apple Silicon
(get_capture_asr_backend), so this hardening is the remaining slice of
Spec 6 / Wave 4.4.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Voice cloning without a transcript fell through to OmniVoice's built-in
load_asr_model() — a transformers pipeline() load of
whisper-large-v3-turbo that fails outright on transformers 5.3 — even
when whisperx / faster-whisper / mlx-whisper were installed and working.
The dub pipeline already used the registry; the /generate clone path
never did.
- services/asr_backend.py: new transcribe_reference() resolves the
active registry backend (honoring auto-detect order and the
OMNIVOICE_ASR_BACKEND override), extracts text from either result
shape (top-level "text" or whisperx-style segments), and degrades to
None on any failure so the model fallback behaves exactly as before.
When the registry itself resolves to pytorch-whisper it defers to the
model's lazy load instead of building a second pipeline.
- api/routers/generation.py: transcript-less references get transcribed
in the GPU pool before inference.
- tests/test_transcribe_reference.py: covers both result shapes,
failure degradation, and the pytorch-whisper deferral.
The remaining half of #308 — pytorch-whisper itself being incompatible
with transformers 5.3 when it truly is the last resort — is tracked in
the issue.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Palash Debnath