On a Tesla T4 the backend exited during the first /generate with no
traceback and no HTTP response, leaving the client with
RemoteDisconnected and every later call with ConnectionRefused. Three
separate defects combined, which is why none of the reporter's
workarounds helped.
1. torch.compile(mode="reduce-overhead") captures CUDA graphs. T4
(sm_75) passed the existing arch gate, so capture was attempted and
aborted the process from inside the native CUDA library — below the
interpreter, where neither the #278 eager-fallback wrapper nor any
except clause can see it. The compile mode is now resolved per GPU:
Ampere (sm_80) and newer keep the cudagraph mode, older cards drop to
the non-cudagraph "default" mode and keep their compiled Inductor
kernels. Fails open on any probe error, so no GPU that works today
loses the optimization. OMNIVOICE_FORCE_CUDAGRAPH=1 restores it.
2. should_torch_compile() never read TORCH_COMPILE_DISABLE. main.py sets
it on win32, build_engine_env injected it into subprocesses, and
docs/install/windows.md tells users to export it — but the in-process
gate ignored it, so the reporter exported the documented variable and
still got "torch.compile applied". The gate now honours
TORCH_COMPILE_DISABLE / TORCHDYNAMO_DISABLE / TORCHINDUCTOR_DISABLE on
every platform, and an env opt-out on the parent propagates to engine
subprocesses. The settings DB path is logged alongside the toggle:
the reporter had three omnivoice.db files and edited one the backend
never opened.
3. Settings -> Performance -> "Disable torch.compile" was rendered
disabled outside Windows in both the Tauri and Electron UIs, so the
one control that would have stopped this was unreachable for the
affected Linux user. The toggle is now live on every platform, and
build_engine_env honours it everywhere rather than only on win32.
Also arms faulthandler before torch is imported, so a fatal native
signal writes the faulting thread's Python stack to backend_err.log
instead of the process vanishing silently. This does not prevent a
crash; it makes one diagnosable. OMNIVOICE_DISABLE_FAULTHANDLER=1 skips
it.
Tests fail before / pass after, verified by stashing the source and
running the new tests against unfixed code. The crash test kills a real
child interpreter with a real SIGSEGV and requires a named Python frame
in the output. test_torch_compile_path_gate's fixture now clears the
compile-disable env vars: main.py setdefaults them on win32, so on a
Windows runner they leaked into os.environ and decided those tests.
Not verified on real hardware — no Turing GPU available. The sm_80 floor
is inferred from the crash report and from docs/hardware-notes-tesla-t4.md,
which already flagged cudagraphs on T4 as attempted by default and never
evaluated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
XNU stops signalling a process as soon as it starts exiting but posts
NOTE_EXIT later in the same exit. A KILL that lands in that window gets
EPERM while the exit probe still reads "alive", so stopping a child that
TERM had just ended could fail with "Operation not permitted". This
flaked the macOS run of contained_exit_probe_preserves_a_live_child.
The EPERM branch now re-probes for up to 250 ms before treating the error
as a live, unsignalable root. Live roots, reaped roots and probe failures
are still errors.
The engine matrix (five columns, three-line rows, every chip on every row)
becomes a shadcn table with three columns — Engine · Runs on · Status — and
one primary action per row (Use / Install). Everything else lives in a
detail panel for the selected row: GPU compatibility chips, isolation,
hints and reasons, health and self-test probes, one-click install
progress, setup snippet, disk usage, docs, license, the curated-model
picker, and now the engine's downloadable WEIGHTS.
Weights belong to their engine: every models.yaml entry names the backend
ids that load it (`engines:`), the detail panel lists and installs them
(EngineWeights, on the model store's install/cancel/remove flow via the
extracted useModelDownloads hook), and the sherpa-onnx engine shows its
dictation-model picker there. The page's "Downloaded weights" list and
recommendation card are gone; only weights no engine owns (speaker
diarisation) remain in a small "Other weights" list. A backend test pins
the mapping: every entry has an `engines` list and every id is a real
backend.
- useEngineInventory: the matrix's state machines extracted verbatim
(shared/local fetch, residency, health/self-test cooldowns, install
poller with overlap guard + epoch, disk-usage generations, license).
- Row status phrases: GPU active / CPU fallback / CPU / Available /
Needs setup / Installing… / failed; routing "unavailable" never reads
Ready. Group captions keep "Ready to use" / "Add more engines".
- Engine titles read "Engines" (each locale's own word); backend
"Model Catalogue → Engines/Models" messages and docs updated to the
new structure.
- Dead matrix CSS (phone-tier grid) removed; scopeReco and RecoBanner gone.
- SetupSummary: an installed engine whose routing is "unavailable" reads
Needs setup, not Ready (select is refused for it too); a failed /engines
or /dictation/models fetch renders as an error with Retry instead of
posing as "Off" / "Needs setup".
- Bulk installs (summary, model store, recommendation card) wait for every
request to settle before re-enabling, and report which repos failed —
one early rejection can no longer re-arm the button mid-flight.
- The weights list stays mounted across family switches (hidden under LLM)
so download progress and Retry/Dismiss state survive navigation.
- Settings search: Hugging Face mirror terms route to Network; the legacy
"models" tab id resolves to Storage. "Manage models" opens the TTS tab.
- Locales: uk "Рушії", zh-TW "引擎", vi "Engine" for the Engines heading.
- Docs name the family tab wherever the instruction depends on it.
The Model Catalogue put the same decision on two axes: an Engines pane with
TTS/ASR/LLM tabs and a Models pane with TTS/ASR/Dictation/Diarisation
sections, dictation shown in both, plus storage stats, the HF token and the
voice-preview toggle parked on the model list. Settings → Voice still carried
Engines and Models entries that only pointed back here.
Now the page reads top-down: a SetupSummary (speech, transcription,
dictation, language model — engine, device, one status word, Change), the
engine list for one family, and that family's downloadable weights under it
(TTS under TTS; offline ASR, streaming dictation and diarisation under ASR;
nothing for LLM, whose engines bring their own). One storage line points at
Settings → Storage.
- ModelStoreTab takes a `family` and scopes sections and the recommendation
preset to it (scopeReco); stats strip, HF-token toolbar and previews
panel removed from it.
- Settings: Engines/Models categories and CataloguePointer removed; models
directory → Storage, HF mirror → Network (both restart-flagged), voice
previews → Storage. "Manage models" in disk usage opens the catalogue.
- Store: openCatalogue takes a family (pane key tolerated, ignored);
pendingCatalogueTab gone.
- Engine matrix title is now the locale's plain "Engines".
- i18n: catalogue.* summary keys in all 21 locales; pane/pointer keys dropped.
- Docs: "Model Catalogue → Engines" is "Model Catalogue"; "→ Models" is
"→ Downloaded weights".
Lands #1998 by @yangfan-yf-yf. Correct finding: the PowerShell example I added
in #1993 generated the administrator key with `python -c`, and the whole point
of the Docker path is that the host does not need Python. On a Windows host
without it, the very first line of the setup fails.
One thing on top. The key is also accepted as an `?api_key=` query parameter
(core/auth.py), and raw Base64 carries `+`, `/` and `=`. A `+` in a query
string decodes to a space, so a user who pasted such a key into a URL would get
a silent mismatch with nothing to explain it. The Bash line next to it uses
`secrets.token_urlsafe` and never had this shape, so the two now agree:
trim the padding, map `+` to `-` and `/` to `_`.
Verified in Windows PowerShell 5.1 (5.1.26100): the block parses and runs, and
the key is 43 URL-safe characters — the same shape `secrets.token_urlsafe(32)`
produces. validate-install-docs.py and both docker/changelog test files pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
The code half of #1931 landed already: `torchaudio.set_audio_backend()` is
guarded, so the torch 2.9.x upgrade a Blackwell card needs no longer trades one
`ml_imports` crash for another. Two things were still missing.
The changelog said the upgrade was documented. It was not — nothing in docs/
mentions sm_120, Blackwell, or the 50-series at all, so a user hitting a native
access violation inside `import torch` had the issue thread and nothing else.
troubleshooting.md now carries it: why the pinned torch 2.8.0 cannot work
(no sm_120 kernels in the wheel — not a setting, not a workaround), the trio
that has to move together, the verification command that proves the kernels
arrived, and the fact that the change is to the repo's own pins so a later pull
will undo it.
The part most likely to be missed is that there are TWO pin lists.
`constraint-dependencies` governs `uv sync`/`uv lock`/`uv run`;
deploy/torch-constraints.txt governs the `uv pip install` paths, which ignore
project-level uv settings. Editing one leaves the other behind, which is what
`RuntimeError: operator torchvision::nms does not exist` looks like from the
outside. Both are named.
The second gap: nothing protected the guard. CI runs the pinned torch 2.8.0,
where `set_audio_backend` still exists, so deleting the `hasattr` as a
"simplify this no-op" cleanup would pass every test in the suite and restore a
hard startup crash for every RTX 50-series user. tests/ now walks the backend
AST and fails on any reach for a torchaudio API that 2.9 removed unless
something proves it is there — a `hasattr`/`getattr` check or a `try`. Fails
with the guard removed, passes with it.
Not addressed here, because it is already fixed: the reporter's third
observation, that launching through the desktop shell hung inside `import
torch`'s native init, is the OpenBLAS/stdin-pipe deadlock closed under #1952.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
@yangfan-yf-yf pushed two more commits to #1987 after the first pass landed.
Two things in them were worth taking:
- an explicit `compose pull` step, so the platform override is proven before
`up -d` rather than discovered when the pull inside it fails; and
- a PowerShell form. An ARM64 Windows host cannot use `export`, and the
surrounding page only ever shows Bash — so the guidance did not actually
reach the users most likely to need it.
Not taken: the same commits also moved `--platform linux/amd64` into the
default `docker pull` / `docker run` quick start. That is a no-op for the
amd64 majority and contradicts the Architecture section directly above, which
introduces the flag as the conditional ARM64 step. The canonical command stays
the one almost everyone should run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
Lands #1987 by @yangfan-yf-yf, which closes#1921.
The published images are linux/amd64 only, and the quick start reached image
resolution before saying so — an ARM64 user met "no matching manifest for
linux/arm64/v8" with no explanation. Verified against docker.yml, which says
so in its own comment: "only building linux/amd64".
One gap in the original: the platform override was documented for docker pull
and docker run, but Compose has no per-command --platform flag, so the
recommended Compose command still resolved the missing ARM64 manifest and
failed exactly as before. DOCKER_DEFAULT_PLATFORM covers it, with the same
caveat the rest of the section makes — emulation, not native support, and only
the CPU profile makes sense under it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
Closes#1847.
The splash is the only surface with a Show/Copy affordance for these lines, and
it unmounts the moment the stage flips to ready — so on a successful first run
the whole install log was gone for good, with no completion pause and nowhere
to retrieve it. A user who wanted to check what had just been installed, or
attach it to a bug report, had nothing.
The lines are written to bootstrap.log beside backend.log now, so everything
about a run is in one directory and a bug report does not have to hunt in two.
Truncated once per process rather than appended forever: a bootstrap is a
single episode and the useful question is always "what happened this time".
That also bounds the file across repeated retries without needing a hook on
every restart path. The docs say so, and say to copy it first if you need a
superseded attempt.
Best effort throughout — a log that cannot be written must never take the
bootstrap down with it, and a test pins that it does not.
The counter half of this issue (Activity frozen at 200) was already fixed on
main by #1918; I verified that before starting rather than assuming the whole
issue was open.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
Refs #1931.
torchaudio 2.9 removed set_audio_backend(). soundfile has been the only backend
since 2.0, so the call was already a no-op — but unguarded it raises
AttributeError inside the ml_imports startup phase, and a failure there takes
the whole backend down: the desktop app sits on "starting backend" forever and
/health stays 503.
The group hitting it is not hypothetical. RTX 50-series (Blackwell, sm_120)
cards have no kernels in the pinned torch 2.8.0, so those users MUST move to
torch 2.9.x, which brings torchaudio 2.9 with it. Being forced to upgrade and
then crashing on a line that does nothing is the whole defect.
This does NOT raise the torch pin. Doing that changes the CUDA build on every
platform, in Docker and in CI, so it is the owner's call rather than something
to slip into a bug fix — the issue stays open for it. What lands here is the
half that is safe: the guard, plus a troubleshooting section with the exact
upgrade recipe and the command to confirm the card is visible, so an affected
user has a supported path today.
The guard is tested at the source level: reproducing it needs a real torchaudio
2.9 in the environment, which the pinned test env does not have. One test also
pins that the guard actually WRAPS the call, since a hasattr elsewhere in the
file would satisfy a naive substring check while the real call stayed bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
On Windows with a non-UTF-8 system code page, a venv path containing non-English characters (commonly a CJK username) killed the interpreter during startup, before any VoiceStudio code ran — Python reads .pth files in the active code page, and uv's editable install writes the project path there in UTF-8. The backend could never start, and the setup screen only said it had stalled.
A new or genuinely broken environment now builds at an ASCII-safe short path; an existing environment that starts cleanly is never relocated. When no safe path can be produced, the app says so before downloading rather than after. The failure message names the cause and a remedy that works for the install mode in use, since portable installs ignore the setting managed installs use.
Fixes#1783.
The compute-time error told users to raise a generation timeout that had no control anywhere in the app — the only knob was an environment variable, and on Windows the docs explicitly warn against the usual way of setting one. Both budgets are now editable in Settings under Performance & Device, persisted and applied on the next start.
Two defects found in review and fixed here rather than shipped: an explicit universal budget silently overrode a separately saved CPU budget, so the CPU row would have looked like it worked and done nothing; and a value already set in the environment shadowed the saved preference while the panel still reported success. A shadowed row now says so instead. Long-input warnings also fire on Apple Silicon, which gets the accelerated budget and was the device in one of the duplicate reports.
Fixes#1787. Closes the reports tracked in #1774 and #1778.
Synchronize VoiceStudio release metadata, lockfiles, installers, container references, documentation, and the dated v0.5.2 changelog after all planned fixes landed.
Refreshes README and linked docs with accurate installation, platform, privacy, API, and model-license guidance; documents local gigastt; and pins OpenAI-compatible ASR traffic to the configured secure origin. Closes#1736.
Closes #1716.\n\nDefines supported, best-effort override, unverified, and unsupported architecture evidence; marks RX 6700 XT/gfx1031 over WSL2 ROCDXG as unverified; and requires routing, utilization, and CPU-fallback evidence before claiming acceleration.
Closes#1713
Adds a separately identified per-user MSI and updater channel, non-administrator install/uninstall verification, and fail-closed WebView2 handling for current-user installs.
Run uvicorn directly under the dev wrapper so a worker crash cannot hide behind a live reload parent. Preserve Python source reloads, restart isolated crashes with bounded diagnostics, and keep persistent crash loops loud.
Closes#1690.
Prepare the tested main branch for the v0.5.1 patch release with synchronized version sources, mirrors, lockfiles, release notes, and install guidance.
Closes#1687.
Fail before backend/window startup when Linux source hosts lack Enigo’s libxdo linker input or WebKit’s GStreamer audio sink. Print an exact distro package command, sync source-build docs, and lock the probes with deterministic tests.
Closes#1680Closes#1682
Palash Debnath